[From Bill Powers (941019.1500 MDT)]
Bruce Abbott (941018.1030 EDT)]
Guess what -- your program compiled and ran on my Turbo Pascal 5.5
compiler! 5.5 was the first TP version that included objects. So if you
don't get fancier than what 5.5 can handle, you can send me object-
oriented source and I can sent you plain vanilla code, and we can run
each others' programs. We seem to have very similar concepts of how to
program this kind of simulation. That will save us loads of time and
argument.
Our ways of computing the schedule are the same, except that your Random
function seems to return a number between 0 and 1 and mine is strictly
integer, so I have to say "if random(10000) < k" where k is 10,000 times
the probability we want.
I'm glad you didn't use a COD or a time-out for eating, and had an
infinite hold (I've been learning a few things from Sam Saunders). I
would like to see a simple model running before we add any frills.
My question about the hold condition was not whether the schedule
stopped running, but whether the "enabled" condition of a key could be
reset by anything but a peck on that key that produces a reinforcer,
once established. I deduce from your program that the vSetUp condition
is turned off only when a reinforcer is delivered. That's what I meant.
Mine works the same way.
It shouldn't be too hard for you to lift my control-system model out of
the code I sent you and plug it in in place of the constant-probability
organism model.
···
=======================================================================
My model, with random switching between keys, can produce results
essentially the same as yours: the generated reinforcements are in the
same ratio as the programmed probabilities, and the key-presses are
close to equal on the two keys. In the control system model, this occurs
when the reward size and input time constant are set so the error is
very large; the model is getting lots less input than it wants. This
makes it press the key quite fast, so that the delivered response rate
is set primarily by the schedule. You accomplish this in your model by
making the probability of a response high enough.
With equal key-presses on the two sides, of course we don't get matching
because b1/(b1+b1) is about 0.5, and with a 3:1 probability ratio the
obtained reinforcement proportion is r1/(r1 + r2) = 0.75. When I adjust
my model so it is controlling well, I get the opposite of matching: on
the "bad" key, the control system presses faster, trying to overcome the
shortfall of input. The program below is initialized to show this
effect. This makes the behavior ratio change upward from 0.5, because
we're getting fewer keypresses on the good side and more on the bad
side. Actually, I'll bet that by adjusting the schedule and the reward
size we could demonstrate this effect in a real animal.
Right now it seems to me that the only way to get matching of behavior
ratio and reward ratio is to give the rate-of-pecking output function a
long time constant, and use the same error signal that drives the
pecking output function to vary the switching in an E. coli-like way. If
the error is small, switching is delayed, and if it is large, switching
is done sooner. The pecking rate must change slowly relative to the
switching rate, so about the same behavior ratio is maintained on both
sides. This will bias the number of pecks toward the high-reward side,
and by adjusting parameters we could probably get a match.
But now I'm wondering why we want to see matching. We can probably
achieve it, but there are lots of other parameter settings under which
the model will behave quite understandably without matching. I'm
beginning to suspect that in real experiments, it's necessary to play
with reward size and schedules to get matching. That is, if this
particular PCT model is anywhere near right. Is there some deep
theoretical signficance to matching? Do animals always display matching,
under all 2-key conditions? More important, how _well_ do they show
matching when they do show it?
What data should our simulations show on the screen?
I need some descriptions of how birds actually behave in situations like
this. Right now I'm working from imagination.
Here's an updated version of my model, with more realistic parameter
settings and the pecks shown individually. The scheduled probability is
computed every 10th iteration, so the value for k has to be adjusted
accordingly. The display is shown at higher resolution -- at 10
iterations per second, the display is 1 minute wide (it wipes out and
resets when the screen limit is reached).
Press space to switch between graphical and text displays, and Esc to
quit.
program varint1;
uses dos,crt,graph;
const
maxkey = 2;
var
reinfsize: array[1..maxkey] of real;
k: array[1..maxkey] of integer;
enabled: array[1..maxkey] of boolean;
peck,graphdisplay: boolean;
numpecks,numreinf: array[1..maxkey] of longint;
q1,o1,e1,o1max,r1,p1,g1: real;
maxx,maxy: integer;
ch: char;
decay1: real;
i,x,key,oldkey,t: integer;
clock: longint;
re1,re2,b1,b2: real;
graphmode,graphdriver: integer;
procedure setgraphics;
begin
graphdriver := 0; graphmode := 0;
detectgraph(graphdriver,graphmode);
initgraph(graphdriver,graphmode,'');
graphmode := getmaxmode;
setgraphmode(graphmode);
maxx := getmaxx; maxy := getmaxy;
clearviewport;
graphdisplay := true;
end;
procedure showdata;
begin
gotoxy(1,key); write('key ',key,' Num reinf = ',numreinf[key]:10);
gotoxy(1,3+key); write('key ',key,' Num Pecks = ',numpecks[key]:10);
gotoxy(35,1);
re1 := re1 + 0.1*(numreinf[1] - re1); { smooth for display}
re2 := re2 + 0.1*(numreinf[2] - re2);
b1 := b1 + 0.1*(numpecks[1] - b1);
b2 := b2 + 0.1*(numpecks[2] - b2);
if ((re1 + re2) > 0.0) and ((b1 + b2) > 0.0) then
write('r1/(r1+r2) = ',re1/(re1+re2):5:3,' b1/(b1+b2) =
',b1/(b1+b2):5:3);
end;
procedure showgraph;
begin
putpixel(40+x,maxy - round(4.0*q1),lightgreen);
putpixel(40+x,maxy - round(4.0*r1),lightgreen);
putpixel(40+x,maxy - numreinf[key] div 50,lightmagenta);
putpixel(40+x,maxy - numpecks[key] div 1000,yellow);
putpixel(40+x,15*key,white);
putpixel(40+x,maxy, lightgray);
inc(x);
if x >= maxx - 40 then
begin
clearviewport;
setcolor(lightgreen);
outtextxy(0,0,'r and q1');
setcolor(yellow);
outtextxy(100,0,' numpecks/1000');
setcolor(lightmagenta);
outtextxy(300,0,' numreinf/50');
setcolor(white);
outtextxy(450,0,' key');
x := 0;
end;
end;
procedure initialize;
var i: integer;
begin
for i := 1 to maxkey do
begin
enabled[i] := true;
reinfsize[i] := 4.0;
numreinf[i] := 0;
numpecks[i] := 0;
end;
k[1] := 20;
k[2] := 120;
decay1 := 1 - 1/30000.0;
o1max := 100.0;
g1 := 0.04;
r1 := 100.0;
q1 := 0.0;
o1 := 0.0;
key := 1;
clock := 0;
re1 := 0; re2 := 0; b1 := 0; b2 := 0;
end;
begin
t := 0; x := 0;
clrscr;
setgraphics;
initialize; ch := chr(0);
while ch <> 'q' do
begin
if (clock mod 10) = 0 then
for i := 1 to maxkey do
if not enabled[i] then enabled[i] := random(10000) < k[i];
q1 := q1 * decay1;
e1 := g1*(r1 - q1);
if e1 < 0.0 then e1 := 0.0;
o1 := o1 + e1;
peck := o1 > o1max;
if peck then
begin
if graphdisplay then
begin
setcolor(white); line(x+40,maxy,x+40,maxy-20);
end;
o1 := 0.0;
peck := false;
inc(numpecks[key]);
if enabled[key] then
begin
enabled[key] := false;
inc(numreinf[key]);
q1 := q1 + reinfsize[key];
end;
end;
if graphdisplay then
begin
if((clock mod 10) = 0) then showgraph;
end
else if ((clock mod 100) = 0) then showdata;
if (clock mod 5) = 0 then
if random(10000) < 10 then
begin
oldkey := key;
repeat
key := 1 + random(maxkey);
until key <> oldkey;
end;
inc(clock);
if keypressed then
begin
ch := readkey;
if ch = ' ' then
begin
graphdisplay := not graphdisplay;
if graphdisplay then
begin
setgraphmode(graphmode);
clearviewport;
setcolor(lightgreen);
outtextxy(0,0,'r and q1');
setcolor(yellow);
outtextxy(100,0,' numpecks/1000');
setcolor(lightmagenta);
outtextxy(300,0,' numreinf/50');
setcolor(white);
x := 0;
end
else
begin
restorecrtmode;
clrscr;
end;
end;
end;
end;
restorecrtmode;
closegraph;
end.