[Bruce Abbott (941103.2000 EST)]
Sam(uel) Saunders (941101.1715 EST)
One thing that seems to me to be missing in the considerations of both Bill
and Rick, on the one hand, and Bruce, on the other, is the stimulus side of
the "three term contingency". In the E-coli, and in many other situations,
there are both stimulus and "reinforcing" aspects to the same event. An
alternative model might be the following:
Two 'stimulus contexts':
A. Increasing gradient
B. Decreasing gradient
In A, a 'flip' will be followed by either an equal (increasing gradient)
or worse (decreasing gradient) consequence, and so would be expected to
decrease in probability.
In B, a 'flip' will be followed by either equal (decreasing gradient) or
better (increasing gradient) consequence, and so should increase in
probability.
Sam, you're right, I'd overlooked the stimulus aspects of the two gradients.
Thorkdike's Law of Effect states as follows:
Of several responses made to the same situation, those which are
accompanied or closely followed by satisfaction to the animal will,
other things being equal, be more firmly connected to the situation, so
that, when it recurs, they will be more likely to occur; those which are
accompanied or closely followed by discomfort to the animal will, other
things being equal, have their connections with that situation weakened,
so that, when it recurs, they will be less likely to occur.
(Thorndike, 1911)
Below is a variant of my e. coli simulation that implements this suggestion.
Speed of travel is constant, but there are two parameters that control (oops,
influence) the probability of tumbling in the presence of positive versus
negative nutrient gradients. The simulation begins with both probabilities
set relatively low and equal, but experience alters those probabilities. One
fun experiment is to begin with the probability of tumbling greater in the
presence of positive gradients than in the presence of negative ones-- at
first e. coli avoids the nutrients, (usually going off screen for a good
while), but eventually "gets it right."
Rather than hearing from our PCT friends about how this is, after all, just
another control system, I'd like to hear them explain the way or ways in which
this model fails to properly implement TRT (and I do admit to the possibility
of error on my part). Be sure to keep strictly within definitions provided by
the law of effect as stated above, for it is the very essence of TRT.
Tom: the problem you mentioned with the earlier e. coli demos is that I forgot
to include the CloseGraph statement at the end of the program. I didn't
notice its absence because it caused no problems when run in my Borland Pascal
7.0 environment.
Bruce
{***********************************************************************
* E. COLI "REINFORCEMENT" SIMULATION 3 *
* *
* Programmer: Dr. Bruce B. Abbott *
* Psychological Sciences *
* Indiana U. - Purdue U. *
* Fort Wayne, IN 46805-1499 *
* (219) 481-6399 *
* Language: Borland/Turbo Pascal 7.0 *
* Created: 11/03/94 *
* *
* This program implements a "selection-by-consequences" reinforcement *
* simulation of the "tumble-and-swim" behavior of e. coli. It *
* assumes that the level of some biochemical in e. coli determines *
* probabiity of a tumble in the presence of rising nutrient levels *
* and that a different biochemical determines the probability of a *
* tumble in the presence of a dropping nutrient levels. *
* *
***********************************************************************}
program Ecoli3;
uses
CRT, Graph, GrUtils;
const
TWOPI = PI * 2;
ENDSESSION = 10000;
var
MaxX, MaxY: integer;
NutrX, NutrY, X, Y: integer;
NutMag, NutCon, dNut: real;
EcoliX, EcoliY,
Speed, Angle, DecayRate, LearnRate,
pTumbleGivenSplus, pTumbleGivenSminus, pMax, pMin: real;
Ch: char;
Clock: longint;
procedure InitScreen;
begin
ClrScr;
InitGraphics;
MaxX := GetMaxX; MaxY := GetMaxY;
Rectangle(0, 0, MaxX, MaxY);
OutTextXY(MaxX div 2 - 170, Y+5,
'E. COLI SIMULATION: REINFORCEMENT MODEL');
OutTextXY(MaxX - 200, 50, ' dNutrient');
OutTextXY(MaxX - 200, 60, 'p(Tumble|S+)');
OutTextXy(MaxX - 200, 70, 'p(Tumble|S-)');
OutTextXY(20, MaxY-50, 'Press ESC to Quit...');
end;
procedure ShowReal(x,y: integer; v: real);
var s: string;
begin
str(v:8:4, s);
setfillstyle(0,0);
bar (x,y,x+textwidth(s),y+textheight(s));
outtextxy(x,y,s);
end;
procedure Tumble(var Angle: real);
begin
Angle := TwoPi * Random;
end;
procedure InitSim;
begin
Randomize;
NutrX := MaxX div 2;
NutrY := MaxY div 2;
EcoliX := 50.0;
EcoliY := 50.0;
X := Round(EcoliX);
Y := Round(EcoliY);
Rectangle(NutrX-2, NutrY-2, NutrX+2, NutrY+2);
Speed := 1.0;
DecayRate := 0.25;
LearnRate := 0.00005;
pMax := 0.500; { Maximum tumble rate }
pMin := 0.001; { Minimum tumble rate }
pTumbleGivenSplus := 0.050; { initial tumble rates }
pTumbleGivenSminus := 0.050;
NutMag := 100.0; { max concentration }
repeat Tumble(Angle) until (Angle < PI/2);
Clock := 0;
end;
function NutConcen(X, Y: real): real;
{ Nutient concentration at point X, Y: environment function }
var
Dist: real;
begin
Dist := Sqrt(Sqr(X - NutrX) + Sqr(Y - NutrY));
NutConcen := NutMag / (1 + 0.001*(Sqr(Dist)));
end;
procedure StepEColi;
var
NewNut: real;
begin
EcoliX := EcoliX + Speed * cos(Angle);
EcoliY := EcoliY + Speed * sin(Angle);
X := Round(EcoliX);
Y := Round(EcoliY);
PutPixel(X, Y, white);
NewNut := NutConcen(EcoliX, EcoliY);
dNut := DecayRate * dNut + (NewNut - NutCon);
if dNut > 0 then { rising nutrient levels }
begin
if pTumbleGivenSplus > pMin then
pTumbleGivenSplus := pTumbleGivenSplus - LearnRate;
if (Random < pTumbleGivenSplus) then Tumble(Angle);
end
else { stable or falling nutrient levels }
begin
if pTumbleGivenSminus < pMax then
pTumbleGivenSminus := pTumbleGivenSminus + LearnRate;
if (Random < pTumbleGivenSminus) then Tumble(Angle);
end;
NutCon := NewNut;
showreal(maxx - 100, 50, dNut);
showreal(maxx - 100, 60, pTumbleGivenSplus);
showreal(maxx - 100, 70, pTumbleGivenSminus);
end;
var i: integer;
z: real;
begin
InitScreen;
InitSim;
repeat
inc(Clock);
StepEcoli;
outtextxy(1,1, '');
if Keypressed then Ch := ReadKey;
until (Ch = #27) or (Clock >= ENDSESSION);
if Ch <> #27 then Ch := ReadKey;
RestoreCRTMode;
CloseGraph;
end.