[From Bill Powers (970620.1322 MDT)]

I have created a monster. Gary Cziko asked for a program illustrating

reorganization, so after looking over some of my previous attempts I

decided to start over and write another one, based on my experience with

the others. While I was at it, I included something that had come up in

previous discussions, an arbitrary environment in which lots of variables

interact with each other, and where the effects of the outputs of the

control systems enter this collection at random, and the variables to be

sensed are also selected at random.

In the present version, there are 15 control systems operating, which

perceive variables made up of weighted sums from 12 sensor inputs each, and

which produce (single) output signals. These control systems act on 75

environmental variables.

The environment is structured as follows. There is an output vector with a

length equal to the number of environmental variables, loaded with randomly

selected pointers to the control system outputs; thus each output variable

is affected by a randomly-selected output from the control systems. There

is also a square matrix of randomly-selected numbers between 1 and -1,

which establishes a contribution from each environmental variable to each

other environmental variable.

On each iteration, all the environmental variables are set to zero. Then

the outputs of the control systems are added to each variable according to

the output vector, and finally the square matrix is scanned, each

environmental variable in turn having the weighted value of all the other

environmental variables added to it. The order of scanning, of course,

makes a difference in the final result. This makes the values of the

environmental variables depend on the control system outputs in a very

strange way, which I haven't bothered to characterize yet. The idea was

just to provide a very complicated environmental feedback function, with

all kinds of internal relations that are beyond my immediate grasp.

The connections from the environment to the sensors of the control systems

are also selected at random. Each of the multiple sensors at the input to

each control system is either connected or not connected to each of the

environmental variables, in a three-dimensional matrix of randomly-selected

boolean variables, each with about a 30% chance of being TRUE.

Once these environmental vectors and matrices have been loaded (part of the

initialization), they can be saved in a file so the same environmental

setup can be used again and again. Right now, however, a new environmental

setup is created each time the program is run.

Also as part of the initialization, a set of reference signal values is

chosen at random in the range from -100 to 100.

The control systems are all integrating control systems with an output

integration factor of 0.001. The perceptual signal in each system is a

weighted sum of 12 sensor inputs, derived through the input Boolean matrix

from the environmental variables. The weights are what get reorganized.

In each input function there are 12 sensors and 12 weights, and also 12

"DeltaWt" variables that get added to the existing weights on every

iteration. Reorganization consists of selecting a new set of 12 DeltaWt

variables at random, with values between -1 and 1.

The square of the error signal in each control system is compared with its

value from the past iteration. If the squared error is increasing, a

reorganization takes place: a new set of DeltaWt variables is calculated.

The DeltaWt variable is multiplied by the absolute value of the error

signal and also by a general SPEED constant (set to 1e-6), and whether

there is a reorganization or not, the product is added to the existing

weight. The weights given to the sensor inputs are thus always changing,

but they change more slowly as the absolute error signal becomes smaller.

Believe it or not, this set of control systems usually converges to the

point where the errors in all the systems are nearly zero -- sometimes zero

to three decimal places. During convergence there are often episodes where

the errors get larger again and fluctuate, but they almost always manage to

get smaller again. In the present version, the reference signals remain

constant and there are no disturbances -- one step at a time!

When the program is run, a new environmental structure is set up, and

iterations then proceed until a key is struck. On the screen at the left,

the reference signal, perceptual signal, and error signal from each control

system are displayed. On the right, all 75 environmental variables are

shown in three columns. And that's it -- that's all the program does right

now. There is a certain hypnotic fascination in watching the numbers change.

What I don't know about this model would fill a sizeable paper. I don't

know what the final input weights are, or how the outputs are connected to

the environmental variables. I don't know if the input functions have

achieved orthogonality at the end. What we have here is a monster that

needs further study. I welcome all efforts by others to find out what's

going on. All I know at the moment is that we have here a set of 15

individually-reorganizing control systems which are capable of making their

inputs match a particular reference value independently of what the others

are doing. They are reorganizing their perceptual input functions to do

this, by the E. coli method. The environment comes to some steady state,

the significance of which I do not know. But it seems to work.

The source code is appended. I will post the executable code on my ftp

site, and make a link to it on my Web page.

Best,

Bill P.

## ···

================================================================

program reorg2;

{

NSYS control systems act on an environment by putting out NSYS output

signals, o. The environment contains NVAR environmental variables, v,

each of which is affected by zero to NSYS of the NSYS output signals,

according to weights WOUT. The environmental variables interact

according to physical laws that define the properties of the environ-

ment.

Each of the control systems contains an input function that has 1 to NVAR

sensors, each sensor detecting the state of one of the environmental

variables. The weighting given to each sensor signal, InWt, is adjustable

by a reorganization system associated with each control system. The

weighted sum of the sensor signals produces a perceptual signal p, which

is compared with a reference signal r to produce an error signal e. The

error signal in each control system is multiplied by an output gain

factor g and integrated to produce the output signal.

Reorganization is driven by the error signal in each control system.

The rate of reorganization is determined by the rate of change of

the squared error, ErSqNew - ErSqOld. Each input weight has associated

with it a DeltaWt, which is multiplied by the squared error and added

to the input weight on every iteration. Thus the input weights

continually change. Every time a reorganization occurs, the size of

DeltaWt is selected randomly from the range between plus and minus

MAXDELTA.

}

uses dos, crt;

const NSYS = 15;

NVAR = 75;

NIN = 12;

SPEED = 1e-6;

MAXERR = 0.0;

type csystype = record

p,r,e,g,o: single;

Sensor: array[1..NIN] of single;

InWt: array[1..NIN] of single;

DeltaWt: array[1..NIN] of single;

ErSqNew, ErSqOld: single;

end;

csysptr = ^csystype;

envtype = record

envvar: array[1..NVAR] of single;

EnvMatrix: array[1..NVAR,1..NVAR] of single;

OutMatrix: array[1..NVAR] of integer;

SenseMatrix: array[1..NSYS,1..NIN,1..NVAR] of Boolean;

end;

var csys: array[1..NSYS] of csystype;

env: envtype;

envfile: file of envtype;

ch: char;

procedure initializesys;

var i,j,k: integer;

begin

for j := 1 to NSYS do

with csys[j] do

begin

for i := 1 to NIN do InWt[i] := 0.0;

o := 0.0;

r := 200.0*(random - 0.5);

g := 0.001;

end;

assign(envfile,'environ.mat');

end;

procedure setenvironment;

var i,j,k: integer;

begin

i := 1;

{$i-}

reset(envfile);

{$i+}

if (IOResult = 0) and (i = -1) then read(envfile,env)

else

begin

with env do

for i := 1 to NVAR do

begin

for j := 1 to NVAR do envmatrix[i,j] := 1.99*(random - 0.5);

OutMatrix[i] := random(NSYS-1) + 1;

end;

for i := 1 to NSYS do

for j := 1 to NIN do

for k := 1 to NVAR do

env.SenseMatrix[i,j,k] := random > 0.7;

rewrite(envfile);

write(envfile,env);

end;

close(envfile);

end;

procedure control;

var i,j,k: integer;

begin

for i := 1 to NSYS do

with csys[i] do

begin

p := 0.0;

for j := 1 to NIN do

begin

sensor[j] := 0;

for k := 1 to NVAR do

with env do

if SenseMatrix[i,j,k] then {if connection to env variable}

sensor[j] := sensor[j] + envvar[k];

p := p + InWt[j]*Sensor[j];

end;

e := r - p;

ErSqOld := ErSqNew;

ErSqNew := e*e;

o := o + g*e;

end;

end;

procedure environment;

var i,j: integer;

begin

with env do

begin

for i := 1 to NVAR do {zero env variables}

envvar[i] := 0.0;

for i := 1 to NVAR do {output effects}

envvar[i] := envvar[i] + csys[OutMatrix[i]].o;

for i := 1 to NVAR do

for j := 1 to NVAR do {internal effects}

envvar[i] := envvar[i] + EnvMatrix[i,j] * envvar[j];

end;

end;

procedure reorganize;

var i,j: integer;

begin

for j := 1 to NSYS do

with csys[j] do

begin

if (ErsqNew - ErSqOld) > MAXERR then

for i := 1 to NIN do DeltaWt[i] := 2.0*(random - 0.5);

for i := 1 to NIN do

InWt[i] := InWt[i] + DeltaWt[i]*abs(e)*SPEED;

end;

end;

procedure showresults;

var i,j,k: integer;

begin

window(1,1,40,NSYS+1);

gotoxy(1,1); clreol;

for i := 1 to NSYS do

with csys[i] do writeln('r=',r:8:3,' p=',p:8:3,' e=',e:8:3);

window(41,1,80,25);

clrscr;

for i := 0 to NVAR-1 do

begin

gotoxy((i div 25)*9 + 1,(i MOD 25) +1); with env do write(envvar[i+1]:6:0);

end;

end;

begin

clrscr;

randomize;

initializesys;

setenvironment;

while not keypressed do

begin

environment;

control;

reorganize;

showresults;

end;

ch := readkey;

end.