[Martin Taylor 921006 11:30]
(Bill Powers 921005 13:30)
One place where I, and I suspect Greg, have not come to agreement with you
is in the limitation of the initiation of reorganization to error in intrinsic
variables--at least this is how I interpret your term "critical error." You
have, once or twice, allowed that a form of critical error is some kind of
integration of error over all or part of the hierarchy, but for the most part
I read you as identifying critical error with errors in variables I would
characterize as body-integrity variables. My view of the mechanism of
reorganization is (I hope) the same as yours, but the triggering, for me, is a
sustained and particularly a growing error in any ECS. What is reorganized
is the output links from that ECS, either by sign changes or by delinking
or forming new links. That (local) reorganization may well trigger further
reorganization in either lower or higher ECSs by affecting their ability
to control, but apart from this avalanche effect, I don't subscribe to a
concept of global reorganization.
In the Little Baby project, our first experiment will be with local
reorganization, with a Poisson rate that is a monotonic function of the
instantaneous value of (e*e + k*e*de)/G where e is the current error, de
the difference in error from the previous sampling moment (a surrogate for
the derivative), k an arbitrary parameter, and G the gain. Chris has
implemented this, but we haven't tried it yet. The reason we put G in there
was that you reported bad effects if you switched the sign of a link in a
high-gain ECS.
(Incidentally, have you progressed any further in your experiments on
reorganization mechanisms?)
···
=========================
My main difficulty with Hebbian learning is that it seems unmotivated;
that is, it's so local that I don't see how it could be related to
critical variables.
Let's take that in two parts. The motivation for Hebbian learning is that
it allows for gradient learning. Typically, the network doing the learning
is intended for classification of some more or less complex input data space,
such as identifying handwritten characters according to their letters of the
alphabet. Less often, but importantly, it is intended to reduce the apparent
degrees of freedom of incoming data by performing what amounts to a non-linear
principal components analysis. In both cases, the network develops its
weights in response to the statistics of its input, where input may be only
the incoming data or may include "training" (teacher's identification of
"this" pattern as an instance of Q, for example). As an example, we might
have a multilayer perceptron with an input data space of, say, 20 by 30 pixels
each of which may have a vlaue between zero and unity, representing the
amount of "ink" in the pixel in an image of a handwritten character. There
may be few or many input units, all of which see these 600 inputs. The input
units connect to few or many hidden units, and the hidden units connect to 26
output units. The teacher decides beforehand which letter is to be output by
each of the 26 output units, and the idea is that if Q is presented, the Q
output unit should have a unity output and the others should have a zero output.
The network is provided with many examples of handwritten characters imaged
in this 20 x 30 space, and for each, the teacher specifies what the outputs
of the 26 output units should be. A training algorithm (of which there are
many) alters the weights of all the units according to the errors of what
the output units should have perceived and what they did perceive. With luck,
the next time that character is presented, the set of 26 outputs will be
nearer to the desired pattern of one unity value and 25 zeros.
In a teacherless system, the changes in weights are such as to exaggerate
differences among input patters, leading units within a level to decorrelate
their outputs, or to converge to a common output, depending on their initial
sets of weights, the statistics of the input patterns, and the local
interconnections among units. Each additional level increases the range of
the complexity of discriminations that the net can make. One layer can
discriminate only linearly discriminable patterns (I'm talking about units
that simply produce a non-linearly compressed weighted sum of their inputs,
not more complex units such as radial basis functions). Two layers can
discriminate arbitrary connected regions, and three can discriminate
arbitrary finite sets of regions (if I remember correctly). This is true
whether the network is trained by a teacher or not.
My idea about the network of ECSs is that the connected perceptual input
functions form precisely a multilayer perceptron if each is of this simplest
(non-linear summation) type. Unlike a normal multilayer perceptron, the
ECS has a local criterion for whether it is contributing to the network as
a whole--its error. In a normal neural network that is only an S-R one-way
system, responsibility for any output of the net is distributed over a large
number of internal units. In a control net, each ECS knows whether it is
controlling its percept effectively, and responsibility can therefore be
localized. If this is so, then the error can be a criterion for Hebbian
alteration of the perceptual input weights, and, we might hope, for robust
and rapid learning when it is compared with the performance of a normal
perceptron learning algorithm.
When an ECS fails to control, it may be because its output are ineffectively
connected, because there is conflict with another ECS, or because the CEV it
is perceiving is inherently uncontrollable given the effectors that link the
hierarchy to the world (like the rising of the sun). The first case
demands reorganization (and possibly Hebbian changes in the output weights),
the third demands Hebbian changes in the perceptual input function (and
possibly reorganization), and the second case seems balanced between the two.
But it is not clear whether anything internal to the ECS can be used to
discriminate among the three cases.
In the Little Baby project, the second (probably) experiment will be to start
with a random set of output connections, and use Hebbian learning on the
perceptual input functions to see whether the baby can learn to perceive the
world in a way compatible with its (fixed randomly assigned) outputs. The
third, and most interesting experiment, if the first two succeed, is to see
whether reorganization and Hebbian learning can be used together, actions
adapting to perceptual input functions that change toward more controllable
forms. That's a little way off yet, but I could imagine that at least some
results from one or two experiments might provide a Christmas present for the
group. I hope so.
=============
I still don't understand the mechanism by which competing behaviors
prevent each other from happening outside the organism. Just saying
that a behavior has a "low impedance" doesn't explain much. You still
leave some of the systems with unsatisfied errors inside the organism,
meaning very large outputs. Why aren't those large outputs producing
large efforts?
When one of the mutually competing behaviours acts so as to provide a percept
that satisfies the higher-level ECS's reference signal (taking the car gets
me closer to my destination), the reference signals for the other would-be
competitors are reduced. As we approach the destination, the reference
signal for perceiving myself to be taking the bike is reduced, as is the
error in that I perceive myself not to be taking the bike. I suspect that the
imagination loop is active in such cases, too, in that when I take the car,
I can imagine myself thereby arriving at the destination, immediately
eliminating the error signal in the "bike-taking" ECS. But suppose the
car will not start. Then the higher-level reference is not being satisfied,
and as you say, the output of the "taking bike" system increases, to the
point where it overtakes the "taking car" system and inhibits it. I get out
of the malfunctioning car, kick it for luck, and get the bike out. (That's if
I don't also have a high-gain ECS with a reference for perceiving the car to
work properly, regardless of whether I am using it at the moment. If I do,
I may call the repair truck, which inhibits the higher system that requires
me to reach my destination.)
If the hierarchy has been effectively reorganized, that
output will reduce the error more probably than not (whether it
actually does is dependent on the momentary state and disturbances
of the world and on other control systems that affect components of
its feedback circuit, but the ECS knows nothing of that).I don't see what "probably" has to do with it.
The world is unstable, with unpredictable disturbances. Sometimes pushing
on something that always worked one way now makes it move the other way.
There are disturbances and conflicts that can cause perverse effects even
if the world actually is working the way you expect.
This was more a defensive paragraph, against those who would correctly
point out that no control system always finds its percepts moving in the
way that they usually do, given a particular output signal. If they did,
you might as well have a simple pre-planning system of the type you call
"cognitive". Control systems exists as a defence against the unpredictability
of the world.
The only way for an
error to be small is for the output to be in the state that brings the
perceptual signal nearly into a match with the reference signal and
keeps it there.
I disagree. There are lots of situations in which error is small by chance.
I return to the degrees of freedom argument. Almost all controllable percepts
are not, at any moment, being controlled. But when their error exceeds
tolerable limits, control may be shifted to them. This implies that most
of the time, most percepts have tolerable errors as a consequence of other
behaviour or the luck of the world.
If there's any uncertainty about whether the error has
been reduced, you're going to have a very low-gain control system,
incapable of opposing disturbances to any interesting degree.
Again, I disagree, and this is the basis of the statistics argument that I
am sure will flare up many tims before we come to agreement on it. You tend
to take the long-term view of what happens as a control system comes to a
stable state with (as I perceive an inderlying assumption) an invariant
reference signal. I tend to concentrate on the transient behaviour of the
feedback loop, as I feel that the world is perpetually disturbing percepts
at all levels, thus changing reference signals at all levels, sometimes
abruptly. I don't think that many ECSs get much opportunity to come to a
steady state, though they may be close a lot of the time. You can have
high-gain systems that sometimes misjudge the data on which they base their
error signals, so long as the misjudgment doesn't last too long or happen
too often.
Anyway, as I said, "probably" was for accuracy rather than as an assertion
that the probability was far from unity.
Martin