[From Bill Powers (2000.12.08.0959 MST)]
Attached is a program called "furnace4.exe" which will run on a PC. It will
show up where attached files show up on your system. Use Start-Run-Browse
to find it (make sure the right drive is typed into the window, such as c:)
and double-click on it. The source code is appended at the end of this post.
This program is called Furnace4 for the reason that it started out as a
program to implement the furnace control system that was talked about some
time ago. But forget that association. It's now set up to show an organism
moving around in a cage looking for food (or maybe water). In the cage, a
rectangle on the right side of the screen, there are six randomly placed
circles, each containing a randomly-selected amount of food (or water)
between 1 and 100 units. When the organism is in motion (see later), it
moves in straight lines until it encounters a wall of the cage, after which
it turns to a new randomly-selected direction (one that leaves it still
moving inside the cage). I'll assume food is involved, in the following.
The organism is always producing repetitive actions which, when food is
present, will deliver it one unit of food for each action (fixed-ratio 1).
However, no food is delivered unless the organism is located inside one of
the green food circles.
When food is delivered, it bumps up the value of a controlled variable
(like stored enegy) by 5000 units. Between deliveries, the controlled
variable decreases in value by an amount determined by a smoothed random
disturbance which is always negative or zero. This is left over from the
furnace model in which variations in outside temperature affected the
inside temperature. It works just as well for a generic disturbance such as
energy losses caused by variable activities other than those shown on the
screen. This disturbance is plotted in white on the left side of the
screen; when it is going downward, it is getting larger (its negative
effect on the controlled quantity is increasing).
When the organism is inside a food circle that still contains some food, it
emits actions at a rate proportional to the error signal. Each action (such
as a lever press) is shown by a short vertical yellow line. The greater the
error signal, the faster the presses. The error is the difference between
the reference level for (let us say) stored energy and the actual amount of
stored energy that exists. The actual stored energy error is shown by a
trace which is normally green, but turns red when the negative-going error
is larger than a threshold amount of about -0.5 units. When the error is
large and actions are occurring rapidly, the negative error rises rapidly
toward zero, increasing only slightly between the big decreases due to food
delivered by the actions. As the error gets smaller, the actions slow down
and eventually we reach the point where the losses due to the disturbance
equal the amount gained from the actions. As the disturbance varies, of
course, the rate at which the actions vary also changes, so as to keep the
error reasonably small (near the white line representing zero error).
The speed with which the organism moves depends on the error when the error
is greater than 5 units. This is larger than the maximum error caused by
the largest amount of disturbance, so when the actions are successfully
producing food, the speed of movement remains at zero. However, when a food
circle runs out of food (the food circle turns red), the error begins to
increase in the negative direction, and it soon exceeds 5 units. At that
point the speed begins to increase in proportion to the error. The organism
moves in straight lines as described, bouncing off the walls of the cage at
random angles, and moving faster and faster as the stored-energy error
increases.
When the organism encounters a food circle containing food (green), its
actions produce food which causes the error to decrease rapidly. Since the
speed of movement is proportional to the error, the organism slows as the
error decreases. If the amount of error is brought below 5 units before the
organism leaves the food circle, motion stops and the actions continue to
provide food until the food runs out in that circle.
By these mechanisms, the organism moves into food circles, stops until it
has produced and eaten all the food in that circle, and then moves on to
the next circle that contains food, in a random search pattern.
When the last food circle is empty and all are red, the error gets larger
and larger and the speed keeps getting greater, up to a point. When the
error reaches 30 units, further increases begin to slow the speed. This is
a crude representation of what happens as the stored energy drops toward
zero. The error gets larger and the speed continues to drop, until the
speed goes to zero. A legend appears in the cage: "STARVED TO DEATH".
There are many details in this model that could be refined. The search
pattern could be made to work like the E. coli method of reorganization.
The food ingested could go into an actual energy store, which would be
depleted at some basic metabolic rate and an additional rate proportional
to the speed of movement (or the square of the speed). The behavior could
be multidimensional instead of consisting only of emitting a single action
at a variable rate. For greater realism, the rate of emitting actions could
be made on-off, so the organism is either not acting or is acting at some
maximum possible rate. I'm sure there are many other details that need
attention.
However, the basic principle I propose is illustrated here well enough, I
think, to make the point. The search pattern is driven by errors larger
than those that are found when there is food available. When food is
available, behavior is driven by the stored energy error in the normal
(lower) range, and it keeps the error smaller than the amount required to
initiate the search again. Increases in food ingestion rate always
_decrease_ the behavior rate, and never increase it, while food is being
produced. So this is clearly not a reinforcement model.
Appended below is the source code in Turbo Pascal. If you compile this
using your own copy of Turbo Pascal 7.0 on a fast machine, I recommend that
you ask me for the Turbo Pascal Library file, Turbo.tpl, which I have
modified as necessary to avoid a fatal error during initialization of the
"delay" function. Without this modified library file, no programs will run
on a machine with a clock speed faster than around 100 to 200 MHz. I hasten
to disclaim authorship of this fix; I found it on the web. I don't have a
fix for other versions of Turbo Pascal.
Enjoy.
Bill P.
Furnace4.exe (56 Bytes)
···
==========================================================================
program Furnace4;
uses dos, crt, graph, grutils;
const triglevel: real = 100.0;
maxplaces = 6;
type foodplacetype = record
x1,y1: real;
foodquantity: integer;
end;
var foodplace: array[1..maxplaces] of foodplacetype;
contvar, per, ref, error, output, dist, timing: real;
xx, timeoff, noff, count: integer;
xmax, ymax, xmin, ymin, width, radius, foodradius: integer;
first, failed,searching,acting: boolean;
x,y,dx,dy,speed,dir: real;
procedure initfood;
var i: integer;
begin
for i := 1 to maxplaces do
with foodplace[i] do
foodquantity := random(100); {food between 0 and 100 units}
end;
procedure initprogram;
var i: integer;
begin
initgraphics;
noff := 10;
timeoff := noff;
timing := 0.0;
failed := false;
searching := false;
dist := 0.0;
contvar := 68.5;
ref := 70;
count := 100;
radius := 3;
width := 200;
speed := 1;
dir := pi/4.0;
xx := 0;
xmax := hsize - radius - 1; xmin := xmax - width - radius - 2;
ymin := 0; ymax := ymin + width + radius + 2;
rectangle(xmin, ymin,
xmax+1+radius, ymax+1+radius);
x := 25.0; y := 5.0;
dx := 0.5; dy := 0.9;
foodradius := 20; { radius of food circle}
for i := 1 to maxplaces do
with foodplace[i] do
begin
x1 := xmin + width * random; { locate places where food available}
y1 := ymin + width * random;
end;
initfood;
end;
procedure showfoodplace;
var i: integer;
begin
for i := 1 to maxplaces do
with foodplace[i] do
begin
if foodquantity > 0 then setcolor(lightgreen)
else setcolor(lightred);
circle(round(x1), round(y1), foodradius);
setcolor(white);
end;
end;
function findfoodplace: boolean; {return true if in food circle}
var i: integer;
found: boolean;
xloc,yloc: real;
begin
xloc := xmin + x;
yloc := ymin + y;
found := false;
i := 1;
while (i <= maxplaces) and (not found) do
begin
with foodplace[i] do
if not found then
begin
if (sqrt(sqr(xloc - x1) + sqr(yloc - y1)) < foodradius)
and (foodquantity > 0) then
begin
found := true;
dec(foodquantity); {reduce quantity of food available}
end;
inc(i);
end;
findfoodplace := found;
end;
end;
procedure search;
var newx,newy: real;
begin
setcolor(black);
circle(xmin + radius + round(x),ymin + radius + round(y),radius);
newx := x + speed*cos(dir);
newy := y + speed*sin(dir);
while ((newx > width) or (newx < 1)
or (newy > width) or (newy < 1)) do
begin
dir := 2*pi*random; {move in random direction after hitting edge}
if dir > 2*pi then
dir := dir - 2*pi;
newx := x + speed*cos(dir);
newy := y + speed*sin(dir);
end;
x := newx; y := newy;
setcolor(white);
circle(xmin + radius + round(x),ymin + radius + round(y),radius);
showfoodplace;
end;
procedure control;
const lasterror: real = 0.0;
var errorchange: real;
begin
per := contvar;
lasterror := error;
error := 10*(ref - per);
if error < 50 then speed := error/50 {SLOW DOWN WHEN ERROR DECREASES}
else
begin
speed := 3.0 - error/50; { SLOW WHEN LOSING STORED NUTRIENTS}
if speed <= 0 then
begin
outtextxy(round(xmin + width/4),
round(ymin + width/2),
'STARVED TO DEATH');
outtextxy(hsize - 200, vsize - 20,'PRESS ANY KEY TO EXIT');
end;
end;
if speed > 3.0 then speed := 3.0;
if speed < 0.0 then speed := 0.0;
errorchange := errorchange + 0.1*(error - lasterror - errorchange);
if error < 0.0 then error := 0.0;
searching := (error > 5.0) {and (errorchange > 0.0)};
if searching then search;
timing := timing + error;
if timing >= triglevel then
begin
if not failed then output := 5000;
timing := 0.0;
setcolor(yellow);
if not failed then line(xx div 10,vsize - 250, xx div 10, vsize - 300);
failed := not findfoodplace;
end
else output := 0.0;
contvar := contvar + 0.00005*(output - dist);
end;
procedure showdata;
var color: word;
y: integer;
begin
if failed then color := lightred else color := lightgreen;
putpixel(xx div 10,vsize - round(50.0*(contvar - 65.0)), color);
putpixel(xx div 10, vsize - 250 + round(dist), white);
inc(xx);
if xx >= 10*(hsize - 210) then
begin
xx := 0;
setviewport(0,0,xmin - 1, vsize - 1,true);
clearviewport;
setviewport(0,0,hsize - 1, vsize - 1,false);
rectangle(xmin, 0,
xmax+1+radius, ymax+1+radius);
line(0,vsize - 250,xmin,vsize - 250);
end;
end;
begin
randomize;
initprogram;
line(0,vsize - 250,hsize - 210,vsize - 250);
repeat
control;
dist := dist + (3000.0*(random - 0.49) - dist)/300.0;
if dist < 0.0 then dist := 0.0;
showdata;
delay(3);
until keypressed;
readkey;
closegraph;
end.
=============================================================================