From 21349c93879addd8dfbf9ce2128726c5840297e9 Mon Sep 17 00:00:00 2001 From: Daniel Rosel Date: Tue, 13 Jan 2026 16:10:10 +0100 Subject: [PATCH] agentic givergence, finally --- paper/src/chapters/03-methodology.tex | 13 ++++++++++--- paper/src/chapters/mdp_agent.pdf | Bin 0 -> 10743 bytes 2 files changed, 10 insertions(+), 3 deletions(-) create mode 100644 paper/src/chapters/mdp_agent.pdf diff --git a/paper/src/chapters/03-methodology.tex b/paper/src/chapters/03-methodology.tex index 2454643..7b4d3f4 100644 --- a/paper/src/chapters/03-methodology.tex +++ b/paper/src/chapters/03-methodology.tex @@ -185,15 +185,22 @@ To develop a robust pricing agent, we require a simulation environment capable o \subsubsection{GOFAI-Based Separability} -We employ Good Old-Fashioned AI (GOFAI) heuristics to generate initial weak labels for separability. We define a set of rule-based predicates $\phi_j: \tau \to \{0, 1\}$ to partition the dataset $\mathcal{D}$ into high-confidence sets $\mathcal{D}_H$ and $\mathcal{D}_A$. We construct distinct MDPs per each behavioral profile of humans and agents and from those we establish $D_{KL}$. +We employ Good Old-Fashioned AI (GOFAI) heuristics to generate initial weak labels for separability. We define a set of rule-based predicates $\phi_j: \tau \to \{0, 1\}$ to partition the dataset $\mathcal{D}$ into high-confidence sets $\mathcal{D}_H$ and $\mathcal{D}_A$. We construct distinct MDPs per each behavioral profile of humans and agents and from those we establish $D_{KL}$. From initial findings we compute a KL divergence of $\approx 2.0236$ across transition probabilities between states which can be seen in \ref{fig:human_mdp_viz} and \ref{fig:agent_mdp_viz}. \begin{figure}[ht] \centering \includegraphics[width=0.8\textwidth]{chapters/mdp_human.pdf} - \caption{Markov Decision Process visualization illustrating the behavioral transition dynamics for human and agent actor profiles. The state space and transition probabilities are learned from observed session trajectories to enable generative contamination.} - \label{fig:mdp_viz} + \caption{Markov Decision Process visualization illustrating the behavioral transition dynamics for human actions.} + \label{fig:human_mdp_viz} \end{figure} +\begin{figure}[ht] + \centering + \includegraphics[width=0.8\textwidth]{chapters/mdp_agent.pdf} + \caption{Markov Decision Process visualization illustrating the behavioral transition dynamics for \textbf{agent} behavior profiles. The state space and transition probabilities are learned from observed session trajectories to enable generative contamination.} + \label{fig:agent_mdp_viz} + \end{figure} + \subsubsection{Transition Probability Estimation} For both subsets, we model the session dynamics as a Markov Decision Process (MDP) and estimate the transition kernel $\mathcal{T}$. The probability of transitioning to state $s'$ given state $s$ is estimated via maximum likelihood: \begin{equation} diff --git a/paper/src/chapters/mdp_agent.pdf b/paper/src/chapters/mdp_agent.pdf new file mode 100644 index 0000000000000000000000000000000000000000..0566be9e926934d4b553b90623800f40acac21ea GIT binary patch literal 10743 zcmbVy1z23mvTpD|un=4a2oQV*cXtWy5Oi<}Fu1$BgeEqc-E{ep;YR2Dtl!J;a*$*@1x^1g*Uw%hz*vZ_X0{lGd#mQtg=uTQTZ|72V6 zST3~ULweF9-S}+t=Dekyh2yfTyBq`kiiRJnRaVhwxZ0EplcoWsn@4LmswH!ZsPB z)F$aAO^ZtuC*b#%zEiyW2#RR0gBzFELV#dP9l6~?!&(G;)JIBi8hJgw*0-^-&@SOs z6-E#u@1Ri>imf@Yuq65fass(s>R7-2(9*kzYr#M|ljA!j)rxpJ>VLFY|DMoFpwZCl zg=*o-F`^LKL~LbsQ0xg3F9KbKROd9U&vZ1#&iBDDviooHUUtO8@p8Tt`)XhWbM+aW zp_(uh587|YfTZ=T<#o0Hv2){R-|p}TbdUUx!-9r7{CYBo9L2Beu+H6E3Aa5wNwU^? z?ZyviS>(wQ$+PS{zvc_DkY}txwPPra2_5c>4L9ebU~FJO3Fr(1-t` zxXc<{N{DWOPai^ebJNVS&Ui}N0WZD`YSnt=R31-e>(kS>0=jk`N6Y>j1#AAP{*N>n_@zwYb}-#ob_+0Op#3?~{xGa~sFaSC%%sl8D^MLMe?D?|#0Cc$?PYdAU772+2I zmmCe)!bQx@mcqslM5^gNn7mtwVLC&IL3}~&lIlc7Rc}GRX0(bFpm-~uO6N|~o0CY` zfUUNlsGfoGp=#2$Mn^$0W{NtJX`~8tchoVP7XjKKsEC9^8uC5X=>LlKMJ_DUL@NR{ z&>;Y;FqZ&V6Q0jQm6Z$q`en632L9`+q0TeKu55reBh?}ziO5SWhLICZ-%OP9Ynbbe zi#26i>q+|I6$e)StM?(o;QkNO(!Ou;^pxA+`G&~!YEp(|PK1c_UUEiE62$AL=BG$%1 zJOc*pb11gd944b_LwhfTQ3idGwCG&#F{fo`Lz+v>=Kw>##UcXl05;HON(g&KgiWSl zzch!hr~?i3Sk6NWkf=1oL@|`JzGA78LRm$td_`zpcxy$-06<|E{{d}yhFXF=owo%| z{*7Cb?MF+hBh?P#91eR15|Iy$XdHjc6?F6Q+gJ{w$LcVf7e0GH@tki22Dn|3vPqs+ zy3>((hegIX*DE8ofwS|rHz#v2bp;GsuuD}n$3rlIOW503me!Uj_OrK(oB5er0=?QJ zYgS27G_miO`P05T-{8;=w^QGTieEr0Uxl7YwVN3MJji2#So-7Qxg$Mp26jJ9_1tW; z^kJ9sNF^0W+mkki+|!&B&Ip{nmJ*Vk}}%a zCvSjXMTW(mQon&H%cx07dM>W&N4f=X)1deSy9qW(s+zN{;w_5yFzLLFT+TVpLp13m zjbSu@@YAjN0bD>z>7bpm!jU8?k~dBtD~qun(|2>Gy-l`AmiSP^f z%00Zcvm=ogyp&75^wy~FTzMe}eCcPG1Ty)^ zgeiW*F-@@N?AS&OVeOOi5?H;vE)KQbr|nrze1z%Vq2xgVTO0kEQlFlllksx~e!f5D zU=Rlz%QM4sllkvl{X090Izz;jAWxYYz{B%QczQAF1D_fIMqxn!2*7Bl|8#xI*H0aP zTp7h|tRes4Hl_#C0|D%R<@+b5r_Z0tIQ~`!U{rSb2nI098JH_UtbVKh^Q6G}mn7qR z0O)szn3gyLcf6LfYSL-D?E?>mlfFlnidH3k9qw$4G`1c z^Z4&hEYEiNgA)Yc;A8>*+Yx5b<&`t$Ou|;P!=5AWjmqK_T0uaLHLD`|xa(NOixYG( z9T1J2*1J5M+*ANV2pVRUo-S;5l|I@<-8&oC7tckAkX&M@&)ZMN$gk-e9KsmE>CqLu5dR`=dm*&Qo);^i*!U_Xd? zcSTo7&`;FHkF#VP$BJY~{ut@Di5OD-$Ud_o@Ku6=Y-?rGS$eC~F{!E}1=^mYn~cPQ zr?y~{nO z9-?*Gl!@)!E+eECAF&Omgd4y-{cmoKVuhGh}wG?0dYH6IqCLxnrdLfw!Wio=k zDQ1H_d_$~Ro8MZiHtrVB=Va@%g_>txAY1S;L^~^~PI^h2`Wr?7iVJT`$84WM04n|n zK=dl61NZf5nf2>y&U;;A5B<)o?^oiF=RK_V_ItOTt@4Kz;L)1AW+d@0S`v2F^qnsO zuKdk%t#+qF`}|%gM6VrCUmN`5ya^9-w20C|DA!T6fiYj+%LQOl&h&O=^8^3gl3~3Y#et=V4Er*p+Ka<@YO1iEp}?W9g7FDyU^k zvw!6_$_RL=ym6qno%~J$Tcd1%ElivONreMT`GbC_=QE6p+2PEGa9Xx;H2Lk$V_E4*-W&{)TT=^9;htFI^^7=ldHEO4f)QY2B+el z+D&gu%FvS=V3$xWrn3`rI~{!ifth(3aR*=1ULk0TUVV--q3M$A5Mfp9QEXUjpH?wa zJ<_bHvpq*@mDK3I5Au9lNJ-ZFGE)S-0JB7X^XBZl(cuw`&BXOg%-;TmuV}}M3&_p`J3yM)W-yP2` z9j|x2^@z)t*>DVPydr!cHW$s`&EL0CvG&-B`Q=t~a1%qyycP&VTd(rIlW63}mH*iU z=71tqnL~~VJg6CxkL?}s59BdLy?q<|ZD8FBZ~RMeutD+)Ne1DG*c4)C7H50kn5k;} z%PySBeK9$&(TF8txUfMhuG_j6^$h;JD{7Cx-t19}X~gnh<(uUuhD0Do951m?*UZq; z&|WO8x+Aq)w`Xw-{Yl@F?vi2~3=}EoGvR6KF}xwV*zgaJW%{2MQwL12?-$=N5?RdD zlb>u7w7QAlAe3lxXQ{2eQjx0>?*8nM*F zHR0eODq@!9G}PdMW-sl1R*o2LI0ltU0;ymhB}&2Et;z{p`*cUp6;<}(9h&02U9L7M zJwD-Dcy!h`Z{R>uoW=EJg#V$_@YSd;@+_KD=a>$ei_{0D58}j8H}x`0;h0!?nb_pL zO2YglJACYg{+_G3h$gb?DU1iLH919Ohf5RA=id5b{9b32`0>+w2v@8}OO+;0XM7wp zkJsB?BLg=&R(B)~T7_HR71(ViV$O|bo1wcg$5XWXr(F}w)^KKc+GaQ$#@X7evK zz=l`BW`hA%>nBo>>4?5<50EfcVQ_M~a!ToML-Eh>Aq*04SdFm;z5|oLE2hNO2ys9U z%K;!xxKrzHMYL6awjXM%#?`Ckw2lZ@qZztin~yu=wjU$Xovy|PJA zTRPz+NwTyt{#G5(ZG|ml8JfJgxI>TYAa{DwBix-mFd&A-ugu->&GP(f1zwHJdRFYl(K)#(#?<=+@<46N!%y88N87@j}$!zc=$5pBV1^DM75}u%IfhU zF@0Yn#TaL)%@x${kAsgClN9lOM;rYHaYmpapg-{<-AJ14N~R}vwsMI3t7sISPk4-u znp$sF15|1t+L%{dpS@<_y8`6zA@Df*Eyj(LE9R90WJ%KTU$w$#vm$*rV9|9CRgm{D zFS^geaK;rFKeJ7X(Nf;he!)FBQgIK}l^p0L_ZDM)4-+w?UZ8en=1Jcl7PS@LFNnWE zm*5m2#}**h6N{eS&|wUJX;X&7_tg;mz4qs?T@r#MXF%Md#_GnlS8 z0Fj^VO|;ET#2AEpK6OOSM(Ri+sBtYnc`aXGGFaESwC^!c#yExLWNjky(vZuGqecIF zBDvV-+nWh0&tC*_Onr=(srS_8%-*a!Ne*9wR+Bo%8MQ1EvB@`OeZ6ayJN&Oobn$T& zk1O>~PmM$#h6E?YWscD(Yd)Z@nqQG{lzBb+_}#Wlw0`NIDH%7qAME!!*sXUg&fks- z1Ex%w$vh^M>>s6!_VV~2iDaDXsos7`#4})EjIO#jnD9if@=nNUe~pu}NDXBXGa%Z` zFy#7*=`cEOwTQPlP#dNRQiwQ8l8_z#({^!Cq)zmlIPdiG0qe~!!cMGUB$NB~bVZ}` z8R#-MpR3ewzIHtLbMuOyXu?}*leX%gS7P-LmtPP6>2ELTYwI228_7F*Tas=ZBMDv&+27w?rV`xOkXBv z23QiWRmDwdt9qo4{3{%$l;dJ7a}suUPDHyV$# zisOxI7IVa8V^6r{ z6WRi8oy0S<$=;`qUz!MQV>&e{)hPNgietfM@CjB% z2{3X*MH>`tAx!~w2P;*fsao~6E`+f?9P!NM_etX~{DKTUkH5q+W)7yJN(s7+K_SL| zTGCIsuZvk+i?Y95H-)b?gJ)0lF=FhOf`PE4Nx8h*UJvXiuOC14Tfo$4^G|Qc5gdNA zYfXApGSa@Qg*9VSyyd8yQ*bl2e@+Q|(>g})z+#WLu#>cZ5rwdRvM0d9^9}sV8E&1_ zrIpvCb;VpBMC@PJoNO=Nj3k2I5Ecl#K}ZfjrebbBD20KR z?Cr72#r)L8lFlfP)3LyQ!y**d>NPcN{5(X&)5o*f<$IE@`^i*X1fuqQ@Z$o12L$l-0d>iX~@N`S)i&-9M`}-XBlst-q%QHauy`Zu=s;yb;tq;C|r7drXrw*&)_ zx8B|*eHCZ2nxMH0q8N9Y-H#L5*6*g4%33db8&Vi#?g>9YFsMFt;F0;$v zQ=(NGb5^9)OGVQ5@-Do&Fupxi68jY@09OKc7+_wV--9Cu(*3SM5Pwl}aae8c=ak>@y>hW}aVB+(@6vx3reIV9hy5TbW&!V9Bye+g?RcXNzAUYx6x1%X1|&3YMYmJnOBaK{qQAS z68DmbEK3YN3R)tE8sZ{5N&PZ~EroSozch)Fk9+c-cwm#OKZo#)ZYIu~)v3H+5Z)U% z`E0ldZW%7_U?AwWwq9QUu-Y)Sj_BG;Y!nWr2kF6iud7~&JD8s{FU)o46V=Q3& zh%KjB_`HN~iBfG`6*&KW-tTATW%%XCLbfKeYs68+BgAFI$B>*i{oB0T&D$Q^_uFwJ zRNNnX#ZXccfyZZN#Pm#~FJLmQR|L3ZvqQ++3%NXJBB?Ucy-d$8BU)WNE|Wza?rK_JQ}%@xyBP^42&R@+&qSySob%DOSiW_g%w?K|I4tqR^S zwD^6*LBTkpWsny~CHw#z&VWY36m{6r4x!eDP}yK~*c8_@9j%u})jG5kc@=n7ZaK+) zi`I2n9qa#2bY@8Po~O}a-~JNo8|&w*n`IjhR@}PNb=^k>RSD+wMX%Fek!lu~2Oh@T zdHFK3ZO&Svmj2rV$DhS84{W0;?FgbkuXbE-Fc$}PHuw;Y;a>R~`Z5MFSue9Kwa1Y| zC?nRL*W2ns*Snp19xr>_O67LWsv=svH~R$YgEy^iE?eF5NhwGj*B657V{!H+ALD*K z?vn)SA}YP&KUw9&eT0$}gD3CHMI~EVstXRLr;;Sa&cc60zvgFXf21OziBqYwPutL; zBhmXwswl1+TfFgdnxBeb_<`Md{N3!?YvwIa0WLGPk@aw)vyd3xJHi?2uJTb^Pn4Rd z=zy(jS?6WC(azfit`0%Y>49PH$msXDgJ;K{YZglv40v)Q(d%Mq&2pNyOEPwFgCg1< z_?+T=S)bjxj2E3Q7e-d?FfqeL|O+8YOWyDROK#d$U1(@!}FhdoHMI{E0jH za}dg|%s|yIfrPxMN@Y&W1?%hTWX9C27t%Xzwge}?U~Xcs>TT@j-2I7rw{b_ENoM%p z_hdJ*zT3sTvkScY7(oa{juAUYVDuR{Y|we$5;N2@9w9H0QJWLXVdg(qCxmzTlMH;0 z#0TfMRTXg3!;hIInCmx~syUVEQABoqX_&dN<28d!BUI)W?=;H}%WbpFP`aWtYe-}% zfnXF$1|_8$SI(&x2%c{T8b-2nEXOW8%?P#hTHjg>NpuWpysr_q2E zSf-Y)im-YCa|tir8{xqTu_zc4h@t7VK2G6E#7l@u$1gDslYmN9j{b z{5u~tgVY?Z3sCbR9CIfCZkIP)#Qtx%nkHWIkFSCe?yl=1?q^|ot&X#C?>w-Y;^BJ@ zj@QO%F0)X@i&lb)Q7=1XI4V9Ei!-erL!1SDNHzrLYgnf9IyX{NC)u$8A2c?nN+~tPk}vvyaWtHM&6_ZYnj5Z!QnZE1PK!e&&3pJaJ1; zm?P#s*v%118Q`bwtkiKIJQbU^!*=wV8E|aWtL-`J_)!8HYdWK|nAYEpl&U$SJ6PyC znoaL_RPI2HSdO%_8uho*(y`RSx7=@9%w627ZO~)pbG65EFMW}I(*C{pSclfKu+Sg{ zVyanKmS^N#INqKfDwqwEzOZ8?S6DWVh*DhE4z~|huVw3-Ui7cF2rR1>RA+`&U6(ET zu9twiJg&cq#I04(iNyVF?0PfexT^x=QiPX|)+hIZtwz}=H|5%uVA8^!h_B+1lCFSv zFmXq|u`YHhJ>ZAz-P&iFnmP<@3wK#c<@KDs6nPc6u%&&fhR;AkNA`oF=QUiHud+WzgJd;WyE^HC)Jf4*V-_t3!^XvGP>fCy`@h ztpcb)&ghtzUy$M!fgy2oIY#e11i-l$$8913yk9!HZi_9mpsy@28kdni!+tK0s=P)l zmhafEh{XeXrUpf1psx1}Sf(QMnBA%4^~u4Z3kSre3*l{!~lMQ#imwd~DiO=u5C&KkECcb|%-)7Y9tnq2Osk%kgn2(~k(J zjLJu6-HJ=STz^Vz#xLA{LoB*b08?A$+~X@H&XI1xacqZ0|0t1_xRt{ zcIYS$ns)-xoAzlTGHC5+6n02G@)Kcb9`dy$deTLc@`FN-s1LV?=E6w4v)uT@be$y? z_O}Z+wh41?sj>(~knGymiNEks`k)xHaYU(qnoiGC-w#hB#AMdbCt~GIH)>2)5Ahw_ zX{=K^$vG0Q2ue^_)|!$=K9D)-Zbudz;&-6Qtf(c2<;7qV*0}9JfcQcIs@;y7CrS5U*xb%_+6yaDOc z1xJNP1s8>~Vy$IMLTinh-fI7-b~|yvX62> zS8pQ8v(LKAx;-8*UMUurw|0lMsWwErhGn8k&1{%fX7Ej-lUc!xl4?uNAE5{4!o(xt zJiL6Y%_c5){5wJY(WJIvmqT7q{G|i|A@9Pn<|ml?iK|zP8fD`D%`W~`i1UP2F|&gH z-o(G4z-LtN4;bzl!xI798`_zDgxJ{qfwVo5-s@XEVRV9G$_lEQv_fW(KM+M#u${gV z*v`zDUeU%%-}-NEF?}mDOBcYqzZn6~oPgh~)PHbU>YLaDn3?_-5fZX-25153K^)8g zdM0L80Emf$9l*)N0nqu)BMH&BG&2;mHn9W)fWIk%_J-gmyzzI+0{RHlTnE<~LK*fI`mCyL)v;Y6QiGFKPaxj4W z#x9lZ9Kg@!`%M+nw+H`5LjQN>dWJNGP4(^mGKP%)pKTC`^Y?iFz9Rt8zY3^Ct)FV9 z%&biSjA~}qg4XtCe-VDmshJr;Ozod+3j815%pi6K4t5Zb^VvbznAsSZn1L*8oc~Xc z|0ei9Df|EFV+B4-v2wC8fH*-cKsEpy3y=ZC%*GDl1hBFJ8Gx)DtWT5wCzbQ5{*052 zgZA`1B=5f)xie zK2yqt6XVU(k$P1huVgj1ZfV{Sx6HvV8GS#=zOi*kcI&D@))KSWU*c+RcVtbi!biXi zAi3iSl7R@~h_fCIuXJ`4rwY%bk?|8aRF#1qPc+WPvx38#ORFFb`)CU%a$!|7i&1^WX0F--iFQ znBdRg^M5KNP%?7`1DO5@P-*}z5P%85^hf+tw6TEzKz|ogFiKh*+Wd#(}he0ii(R00p2u6N*d~3h}}e-GV<_zUd{|2WZ62 zU4Q{m7p53``_R`D5I|G9K~h`t@f+3smoARr4TGZF=bsGJzme*x3E BqGtdA literal 0 HcmV?d00001