[From Bill Powers (920711.0800)]
Martin Taylor (920710.1530) --
What I like about your proposal is the idea that there's a random-
output reorganizing process operating in the (potential) CNS
hierarchy, gradually being replaced by (or paralleled by) systematic
control. This reorganizing process could monitor certain built-in
aspects of potential control systems in the CNS, such as the signals
being emitted by comparators (which are so simple that we can assume
them to be part of the initial complement of parts). Also, the brain
is physically organized so that sensory computers tend to be lumped
together, as do motor output computers. There may be aspects of these
more or less localized networks that permit monitoring for invariance,
negative feedback relations between motor and sensory signals, and so
on -- without an implication that the reorganizing process knows in
advance what perceptions or actions will be appropriate for the
current environment.
One bit of confusion was caused by my using "sensing" in an ambiguous
way. I agree, of course, that all control requires sensing. When I
said that intrinsic variables were outside the sensory interface, I
meant that they are outside the CNS sensory interface. I don't think
that chemical sensors and comparators can be counted as part of the
CNS hierarchy. If some biochemical states of the organism become parts
of experience (as in emotions), they must affect neural sensors
connected to the CNS. Under my concept of the reorganizing process, if
biochemical states are to become a basis for reorganizing, they would
be sensed in some other way, probably by chemical or autonomic
sensors. To the CNS, sensed internal states are no different from
sensed external states; they are simple data about the current state
of affairs in the world outside the CNS. They have no value or any
built-in reference states as far as the CNS is concerned.
Let's say that the photon-based reaction affected the concentration
of CO2 in the organism. Then if the wiggle-motor became more
specifically sensitive to deviations in the CO2 level, both above and
below optimum, then the organism would begin to control for light
level.
You're making the photon reaction itself produce the CO2. What I'm
interested in is the case in which the photons arise from some object,
the presence of which implies some OTHER effect on CO2 in the body.
The photons might come from a warning light that indicates a leak in a
CO2 container. Now the organism has to learn to get out of there when
the warning light (whistle, vibration) is sensed, even though the
light itself is not the cause of the elevated CO2 tension in the
bloodstream, and position in space, not CO2 tension, is the variable
that becomes controlled. So the organism must learn to control for a
specific level of one perception as a way of controlling another
variable inside itself that has no necessary connection to the
controlled perception, save for happenstance properties of the
external world.
The pigeon has to be able to learn to walk in a figure eight (itself a
control process) as a means of making food appear. Walking has no
effect on improving nutritional state, save for the mad scientist in
the environment. Something has to make the learning of one control
process depend on an indirect and arbitrary effect of that control
process on the environment, and by that indirect route on the internal
state of the organism. This is the kind of thing that my reorganizing
system is meant to accomplish. I don't think that any localized
reorganizing principle could do it.
ยทยทยท
------------------------------------------------------------------
I think that before we get any further into complicated arguments and
misunderstandings, we should do some work on simulating
reorganization. I'm going step by step on this, and am not ready to
show a whole indirect reorganizing process yet. At the moment, what I
have is a method of solving simultaneous equations by reorganization.
This is not meant to imitate any particular organismic process; that
comes a few steps further on. What it does is illustrate some
principles.
The basic setup is this:
There are 10 perceptual functions, each producting a linear function
of 10 environmental variables. The form of the linear functions is
generated by 10 weights for each of 10 perceptual functions, so that
p[i] = a[i,j]*v[j], where 0 <= i < 9, 0 <= j < 9
The weights a[i,j] are generated at random in the range between -1 and
1 and are fixed. The initial values of v[i] are zero.
There are also 10 fixed reference signals r[i], generated at random in
the range -50 to 50. Thus we can compute 10 error signals e[i] = r[i]
- p[i].
The reorganizing system computes the square root of the sum of squares
of all error signals, which is the distance in 10-dimensional
hyperspace between the point p[i] and the point r[i]. On each
iteration, the distance is compared with the distance on the previous
iteration, to provide a measure of velocity toward or away from the
point r[i].
If the velocity is positive on any iteration, a reorganization takes
place. For all negative velocities, the direction of movement
resulting from the last reorgnization is applied over and over.
Reorganization is a two-step process.
First, a 10-element vector delta[j] is filled with random numbers
between -1 and 1. It is normalized by dividing each entry by the
square root of the sum of the squares of all entries; this makes the
entries into direction cosines in a 10-dimensional space.
Reorganization thus changes the direction of this vector randomly in
10-space. This normalization is not essential, but seems to make
convergence faster.
Second, the 10 elements of delta[j] are added to the 10 current values
of the environmental variables v[j] after multiplication by
"stepsize." This causes the complex environmental variable to move a
distance "stepsize" in hyperspace.
The loop is now closed, because that movement in hyperspace results in
a change in perceptions p[i] and therefore in the errors e[i].
The variable stepsize can be a small constant, or (for faster
convergence) can be proportional to the remaining hyperspace distance.
I have used the latter method.
Here is part of a run arbitrarily cut off at 5000 iterations:
Iteration Error left, # consecutive # consecutive
fraction of max steps reorgs
n= 1 e/emax= 0.996 steps= 1 reorgs= 1
n= 14 e/emax= 0.978 steps= 12 reorgs= 1
n= 17 e/emax= 0.977 steps= 2 reorgs= 1
n= 24 e/emax= 0.975 steps= 5 reorgs= 2
n= 29 e/emax= 0.974 steps= 3 reorgs= 2
...
n= 4968 e/emax= 0.011 steps= 8 reorgs= 1
n= 4974 e/emax= 0.011 steps= 3 reorgs= 3
n= 4984 e/emax= 0.011 steps= 8 reorgs= 2
n= 4990 e/emax= 0.011 steps= 1 reorgs= 5
n= 4994 e/emax= 0.011 steps= 3 reorgs= 1
-----------------------------------------------------------------
DATA SUMMARY after run:
Environmental variables (target of reorganization):
49.84 13.35 -24.48 -3.29 25.79 14.76 -0.48 -9.68 67.15 -0.92
Perceptual coefficients (picked at random, normalized to 1.0):
-0.890 -0.460 -0.750 0.370 -0.850 0.840 0.610 0.630 -0.360 0.440
-0.850 0.060 -0.950 -0.950 0.530 -0.910 -0.150 0.370 0.400 0.200
-0.690 0.870 0.790 0.140 -0.570 0.100 -0.740 -0.390 -0.430 0.540
-0.390 -0.760 -0.400 0.720 -0.450 -0.760 0.340 -0.930 -0.560 0.130
-0.460 0.700 0.170 0.190 -0.360 0.550 0.990 -0.090 0.390 -0.800
0.030 -0.810 -0.040 0.220 -0.440 -0.490 0.110 -0.280 0.560 -0.280
0.640 -0.650 0.310 0.670 -0.970 0.050 -0.220 0.310 0.730 0.930
0.650 -0.050 -0.240 -0.460 0.300 0.820 -0.750 -0.570 -0.430 -0.820
0.030 -0.140 0.840 -0.910 0.200 0.140 0.310 0.200 0.790 -0.910
0.750 0.870 -0.780 -0.360 0.470 -0.940 0.640 0.230 0.620 0.650
Perceptual signals:
-20.93 -30.02 930 10.08 33.01 -20.97 -0.87 15.00 36.93 -1.97
Reference signals (picked at random):
-21.00 -30.00 10.00 10.00 33.00 -21.00 -1.00 15.00 37.00 -2.00
Error signals:
-0.07 0.02 0.70 -0.08 -0.01 -0.03 -0.13 -0.00 0.07 -0.03
Count = 5001, RMS Err/max = 0.0106
---------------------------------------------------------------------
What's interesting about this control system is that it is
independently controlling 10 perceptual variables with respect to 10
arbitrarily-selected reference signals, by means of randomly altering
10 environmental variables on which all the perceptual variables
depend in different ways. The control employs only the total error,
not the error in each channel by itself.
As I mentioned in a previous post, I have also made this work by
reorganizing the coefficient matrix a[i,j], with the environmental
variables fixed at random settings. So we have proof of principle for
two major ways of reorganizing: reorganizing the input function, and
reorganizing the feedback link (the present case). There's no reason
why both modes of reorganization can't be going on at the same time.
All that matters is whether those two points in hyperspace are getting
closer together or farther apart.
The basic random process occurs between the total error signal and the
set of environmental variables. This set of variables could represent
whole control systems, with the random adjustments being made on the
forms of their input functions, the signs of output connections, and
the various factors influencing loop gain. This arrangement would look
like the one you're suggesting, where the CNS is reorganizing itself.
Here's the C function that does the actual reorganizing. Most of the
variables are globals. The init() routine and the routine that
calculates error signals are also included:
/*below is called once per iteration */
void reorg(float *e, /* pointer to list of error signals */
int nerr, /* number of error signals */
float *delta, /* pointer to motion array */
float *var, /* pointer to controlled variable array */
int nvar) /* number of entries, delta & var array */
{
float temp;
static int numsteps = 0;
static int numreorg = 0;
static int newstate = 0; /* state=0 means not reorganizing */
static int oldstate = 0;
lastesq = esq;
esq = 0.0;
for(m=0;m<nerr;++m)
esq += e[m]*e[m];
distance = sqrt(esq);
rate = distance - lastdistance;
lastdistance = distance;
stepsize = 0.7 * distance/maxdistance;
if(rate > 0.0)
{ /* reorganize delta array */
newstate = -1;
++numreorg;
for(m=0;m<nvar;++m) /* pick all new deltas at random */
delta[m] = (rand() - 0.5*RAND_MAX)/(0.5*RAND_MAX);
temp = 0.0;
for(m=0;m<nvar;++m) /* normalize to 1.0 */
temp += delta[m]*delta[m];
temp = sqrt(temp);
for(m=0;m<nvar;++m)
delta[m] /= temp;
/* delta now is a set of direction cosines */
}
else {++numsteps; newstate = 0;}
if(newstate != oldstate) /* print consecutive # steps, # reorgs */
{
if(newstate == 0)
{
printf("\x0d\x0an= %6d e/emax= %7.3f steps= %3d reorgs= %2d",
count,distance/maxdistance,numsteps,numreorg);
numsteps = 0; numreorg = 0;
}
oldstate = newstate;
}
for(m=0;m<nvar;++m) /* move contr var in hyperspace */
var[m] += stepsize*delta[m];
}
void calcerr() /* compute all error signals */
{
for(m=0;m<numvars;++m)
{
p[m] = 0;
for(n=0;n<numvars;++n)
p[m] += v[m]*a[m][n]; /* compute perceptual variables */
e[m] = r[m] - p[m]; /* compute error signals */
}
}
void init()
{
for(m=0;m<numvars;++m)
{
r[m] = random(100) - 50;
delta[m] = 0;
for(n=0;n<numvars;++n)
a[m][n] = 0.01*(random(200) - 100);
}
count = 0;
maxdistance = 0.0;
for(m=0;m<numvars;++m)
maxdistance += r[m]*r[m];
maxdistance = sqrt(maxdistance);
}
Feel free to use, modify, etc.
---------------------------------------------------------------------
Best
Bill P.