[From Bill Powers (941030.1845 MST)]
Bruce Abbott (941029.1700 EST) --
Got your program to run with no trouble; grUtil is now a unit in my
TP7.0.
Me:
Obviously there can be no effect on the current behavior by
its consequence, the reinforcement that has not occurred yet and may
or may not occur. Only after the behavior has taken place (swim for
some length of time and then tumble) and the consequence occurs can
the consequence be known
I disagree with you here. Not swim AND THEN tumble, but merely swim.
The behavior on which reinforcement would act in this context is
forward motion along a given trajectory.
Let's examine your code, which I reproduce here with its parts labelled
according to PCT:
procedure StepEColi;
var
NewNut: real;
begin
EcoliX := EcoliX + Speed * cos(Angle); |
EcoliY := EcoliY + Speed * sin(Angle); |
X := Round(EcoliX); |
Y := Round(EcoliY); ENVIRONMENT
PutPixel(X, Y, white); |
NewNut := NutConcen(EcoliX, EcoliY); |
ยทยทยท
-----------------------------------------------
PERCEPTUAL FUNCTION, PERCEPTUAL SIGNAL(dNut)
dNut := DecayRate * dNut + (NewNut - NutCon);
-----------------------------------------------
if dNut <= 0 then COMPARATOR
Tumble(Angle); OUTPUT FUNCTION, OUTPUT QUANTITY (ANGLE)
-----------------------------------------------
NutCon := NewNut; {program process}
end;
OK, what is the "consequence" and what is the "behavior" here? I think
we can probably agree that dNut is the consequence; it's a perceptual
representation of the time rate of change of nutrient concentration just
outside the thingie. What is the behavior? I have a little problem with
this because it seems to be the execution of a tumble the instant that
dNut falls below its reference level of zero. It's hard to see how the
behavior could be affected by the consequence -- how could the behavior
change if it is always the execution of a tumble? About the only
dimension in which this behavior could change would be in rate, or in
number of tumbles needed to get within some distance of the goal.
So I added a termination condition, ..." or (Dist < 3)", and made Dist a
global variable.
Then I supplied a routine for showing real numbers on the screen:
procedure ShowReal(x,y: integer; v: real);
var s: string;
begin
str(v:6:2,s);
setfillstyle(0,0);
bar (x,y,x+textwidth(s),y+textheight(s));
outtextxy(x,y,s);
end;
And I changed the procedure for stepping:
======================= added
if (dNut <= 0) or (random(1000) < 100) then
begin
Tumble(Angle);
NumTumbles := NumTumbles + 1.0;
end;
showreal(maxx - 100,20,NumTumbles);
showreal(maxx - 100,40,dNut);
Also, you will notice, I added a disturbance to the output, by creating
random tumbles independently of the value of dNut. If the consequence
were actually controlling the tumbles, disturbing the tumbles should
result in a change of dNut that would oppose the change in tumbling, and
restore it to (or toward) the undisturbed state.
As far as I can see, that doesn't happen. When you put in the
disturbance, the number of tumbles needed to get near the goal goes way
up (a factor of 10 with the numbers I used), and dNut increases much
more slowly.
On the other hand, when you add a disturbance to dNut (instead of the
above disturbance -- remove it):
====== disturbance
dNut := DecayRate * dNut + (NewNut - NutCon) - 0.01;
the value of dNut just continues to rise as the target is approached,
much as without the disturbance.
This is all very rough and ready, and I'll try to do a more careful job.
But at the moment it looks as if dNut is controlled, and the rate of
tumbling is not controlled.
----------------------------------------------------------------------
Best,
Bill P.