Plausibility of random reorganization

[William_T_Powers2]

[From Bill Powers (920702.1600)]

Martin Taylor (920702.0800) --

I said that I didn't think that what you proposed was plausible for
reasons that I hoped were clear.

...

Taking the simple case in which link() can be 1, 0 or -1, a random
reorganization has a probability 0.33... of doing the right thing if
there is only one dimension, 0.111... in two dimensions, and (1/3)^n in >n

dimensions. If there are three ECSs in each of two layers, that is >roughly
a one in twenty-thousand chance. All other sets of connections >lead to
some conflict, and even if we grant the probability that some >other sets
provide stability, the odds are not good that global >reorganization by
non-targeted random alteration of link sign will >reach an optimum quickly.
The problem is the same as that of molecular >evolution as seen by the
creationists. You can't do it that way. You >have to grow stably.

I now remember this point that you brought up some time ago -- just didn't
make the connection. I don't have a definitive answer, but I think that
your analysis is making some assumptions that have alternatives. I'll not
dispute that "targeted" reorganization might be necessary (although when I
used the term "target" the other day, I was referring to the whole
hierarchy). I have proposed a version of targeting based on the
phenomenological idea that awareness directs reorganization to problem
areas. But having no model of awareness or attention, I haven't pushed that
very hard. Nor am I convinced that random reorganization won't do the
trick.

One alternative to targeting that handles SOME of the statistical problem
is the idea of critical phases in maturation. This is consistent with the
idea that the growth of the hierarchy is almost entirely bottom-up. Under
this concept, when it's time to learn hand-eye coordination, in the crib,
that's the only level of organization susceptible to reorganization, and so
on up the levels. This is not to say that reorganization occurs exclusively
at the top level at a given time; only that there is a top level, that it
gets progressively higher with time, and that reorganization has no effect
above this level. But I'm not sure that even this idea is necessary.

I'm made a little suspicious by your way of framing the 3-D learning
problem. To speak of "the chance of doing the right thing" makes it seem
that the outcome of the random act is either right or wrong, and also that
you have only one stab at it. If E. coli had to gamble everything on one
tumble, it would be in bad shape. In fact, after any tumble in either the
1-D, the 2-D, or the 3-D case, the chances of heading in a direction more
favorable than unfavorable, after a single tumble, are about 50 percent.
You're treating each dimension as an independent case, which would imply
that the probability of all three cases being in the favorable half-region
is only 1 in 8 (you calculate 1 in 27). If you think of a tumble as
selecting a direction in space, however, the probability of this direction
being in one hemisphere rather than in another is 1 in 2. It isn't
necessary for any tumble to aim directly up the gradient; all that's
required is a component in that direction. The probability is 50 percent
that the component will be between 0 and 100 percent of the swimming
velocity -- it would be interesting to know the actual average velocity but
it's not zero.

Note that even in a hypersphere of n dimensions, the chances of n
simultaneous reorganizations creating a change toward rather than away from
a given point in the hyperspace is still 50 percent. Of course the average
velocity toward the target point decreases with the number of dimensions --
but any bias will get you there eventually. This is one of the basic
principles of methods of descent (I think -- I'm no expert). There are, of
course, methods of STEEP descent, but I don't see right off how they would
be implemented by a reorganizing system.

Putting the problem in terms of right versus wrong choices makes the
probability of organizing even one level of control seem incredibly small.
But I think this is the wrong way to set up the problem. EVERY form of a
perceptual function will yield a perceptual signal that is a regular
function of external events. There is not just exactly one combination of
inputs that will yield the "right" perception, with all others being
"wrong." There are many possible ways of perceiving a given environment
that will allow control, and many ways of exerting control that will have
at least some beneficial effect on intrinsic state. On the scale of
individual perceptual signals at the lowest levels, the number of equally
good alternatives must get astronomical. I think you're misstating the
combinatorial problem.

One aspect of control, the sign of the effect of error on action, is binary
in nature and has only a 50 percent probability of being chosen right by a
random process. When a given control system such as a spinal reflex is
being organized, however, the mostly likely feedback effect will be none at
all, because there are dozens or even hundreds of parallel systems all
hooked up more or less the same way. This makes a 50-50 chance of getting
it right into a continuous distribution with the most likely one being
neutral. All that's required to get SOME control is that there be more
loops in the negative feedback mode than in the positive feedback mode. A
biased random walk will work quite well to optimize the amount of negative
feedback.

Another factor that has to be kept in mind is that an infant left to
reorganize itself into a child will surely die. The infant is supported
from outside while it gets its behavioral control systems into order. It
can spend a long time making mistakes. It can go through millions and
millions of reorganizing trials both overtly and in the imagination mode,
24 hours a day. Your point about reorganization being called upon to make
rapid correct decisions simply doesn't hold up: that's not necessary. If we
leave it up to children to make immediately correct reorganizations of
their systems for avoiding oncoming cars, there won't be many children
left. In organisms that are not born with rather extensive complete control
systems that control the most important variables, there is no alternative
but to protect the developing young from the need to solve control problems
by the slow process of reorganization.

Finally, only the simplest control systems involve a huge number of degrees
of freedom (something you should consider in line with your DoF paper).
Each successive level, up to a point, drastically reduces the number of
degrees of freedom. The first new system at a given level allows for only
1! Even passing from heat intensity receptors to the sensation of warmth
involves an immense convergence: heat detected anywhere on the skin is
warmth. So the most difficult reorganizing tasks are those at the lowest
levels -- where there is the greatest amount of preorganization of neural
pathways and the highest rate of convergence.

I think that in considering degrees of freedom, you are doing your mental
calculations as if all control systems are present and active from the
beginning. My view is that in human beings at least, there are very few
low-level behavioral control systems available in the beginning, and no
higher-level systems at all.

All these considerations must considerably alter calculations of the
chances of random reorganization being successful. But I will still not
rule out some sort of targeting.

···

------------------------------------------------------------------
Best,

Bill P.

[_Martin_Taylor]

[Martin Taylor 920703 18:00]
(Bill Powers 920702.1600)

Once again, we are getting close to agreement. But (luckily) we are not quite
there yet, I think.

One alternative to targeting that handles SOME of the statistical problem
is the idea of critical phases in maturation. This is consistent with the
idea that the growth of the hierarchy is almost entirely bottom-up. Under
this concept, when it's time to learn hand-eye coordination, in the crib,
that's the only level of organization susceptible to reorganization, and so
on up the levels. This is not to say that reorganization occurs exclusively
at the top level at a given time; only that there is a top level, that it
gets progressively higher with time, and that reorganization has no effect
above this level. But I'm not sure that even this idea is necessary.

Yes, I was trying to push this. But rather than saying that a new top level
is being developed, I still like the idea that one is inserting levels, thus
redefining what was there before. The difference is one of viewpoint, I
think. See later, about the baby.

I'm made a little suspicious by your way of framing the 3-D learning
problem. To speak of "the chance of doing the right thing" makes it seem
that the outcome of the random act is either right or wrong, and also that
you have only one stab at it.

Right or wrong--yes. One stab, no. I never intended that implication.

You're treating each dimension as an independent case, which would imply
that the probability of all three cases being in the favorable half-region
is only 1 in 8 (you calculate 1 in 27). If you think of a tumble as
selecting a direction in space, however, the probability of this direction
being in one hemisphere rather than in another is 1 in 2. It isn't
necessary for any tumble to aim directly up the gradient; all that's
required is a component in that direction.

That's true when there is only one degree of freedom for the controlled
percept: "satisfactoriness of the environment" and three degrees of freedom
for action. But the more common case is when the action degrees of freedom
are fewer than the perceptual degrees of freedom. Then you have to worry
about conflict, and components in those other directions do matter.

One in 27 is correct, because a link can have any of three values, not two.
We are talking about reorganization that permits making and breaking links,
as well as changing signs, are we not?

Even when we are ignoring the conflicts induced by components in directions
orthogonal to the one causing the reorganization, my argument was not about
whether the control vector would eventually point in the right direction, but
about how fast it would do so. I should not think that changes induced by
reorganization should ever occur faster than the Nyquist rate for the feedback
loop in question, so I was interested in the probability of changes that
were significant improvements. I grant that 1 in 2 will be improvements on
no control, but most of those will be trivial improvements if the space has
high dimensionality. When I first brought up this topic, I considered anything
within 60 degrees of the optimum direction to be significant improvement.

EVERY form of a
perceptual function will yield a perceptual signal that is a regular
function of external events. There is not just exactly one combination of
inputs that will yield the "right" perception, with all others being
"wrong." There are many possible ways of perceiving a given environment
that will allow control, and many ways of exerting control that will have
at least some beneficial effect on intrinsic state. On the scale of
individual perceptual signals at the lowest levels, the number of equally
good alternatives must get astronomical. I think you're misstating the
combinatorial problem.

Yes, you are quite right about that. My error is to require a particular ECS
to control a particular perceptual degree of freedom. This will be valid if
the perceptual input function of each ECS is prescribed in advance, but of
course it cannot be. Each ECS must learn what it is perceiving as well as
to control that percept. That thought carries much implication, which I
haven't considered enough to pursue here. But it does carry the implication
that there is an enormous amount of symmetry in the hierarchy that a random
reorganization system can initially exploit. But it can't do so once the
symmetry has been broken by some ECSs having learned to control their
percepts. Then, with random reorganization, I think the combinatorial
problem is as I stated, the more so the more ECSs have acquired control.

Another factor that has to be kept in mind is that an infant left to
reorganize itself into a child will surely die. The infant is supported
from outside while it gets its behavioral control systems into order. It
can spend a long time making mistakes. It can go through millions and
millions of reorganizing trials both overtly and in the imagination mode,
24 hours a day.

Yes, that was my point about it being no accident that infants of all species
being born either unable to act (very low loop gain) or with built-in control
(but unable to learn new controls at the level that is inborm) like a deer
or a chicken. Deer can run at birth, chicken can peck at seed (but cannot
learn to adapt to prism displacements of their vision). But I think "millions
and millions" may be saganesquely excessive. It's possible, especially at
low levels, I grant. At higher levels, things move more slowly.

Your point about reorganization being called upon to make
rapid correct decisions simply doesn't hold up: that's not necessary.

It does, if the system being reorganized is actively controlling, with
reasonably high gain.

In organisms that are not born with rather extensive complete control
systems that control the most important variables, there is no alternative
but to protect the developing young from the need to solve control problems
by the slow process of reorganization.

Yes, that's what I meant.

Finally, only the simplest control systems involve a huge number of degrees
of freedom (something you should consider in line with your DoF paper).
Each successive level, up to a point, drastically reduces the number of
degrees of freedom. The first new system at a given level allows for only
1! Even passing from heat intensity receptors to the sensation of warmth
involves an immense convergence: heat detected anywhere on the skin is
warmth. So the most difficult reorganizing tasks are those at the lowest
levels -- where there is the greatest amount of preorganization of neural
pathways and the highest rate of convergence.

A really speculative point! Apart from the statistical convergence that has
nothing to do with control, due to the natural redundancy of the real world,
I have envisioned the possibly controllable percepts (not degrees of freedom
for perception) as growing in number as we go up the levels, before reducing
at the highest levels. Sort of barrel-shaped, rather than conical.

I think that in considering degrees of freedom, you are doing your mental
calculations as if all control systems are present and active from the
beginning. My view is that in human beings at least, there are very few
low-level behavioral control systems available in the beginning, and no
higher-level systems at all.

No, there's a misunderstanding here. I tried, as I remember, to put two
alternatives into play. One was indeed the matured system, which presents
a problem for random reorganization because of the likelihood that untargetted
random reorganization will disrupt areas that are working very well. The other
was a developing system, and for it, I suggested that we might consider the
top level as representing optimum values for the intrinsic variables, even
if initially there were no other levels. All other levels are inserted
inbetween the top and the world-interface.

Since we are (by agreement) working from the lowest common denominator of no
prior construction of ECSs, we have to include evolutionary development here.
No matter what the evolutionary level, the prime concern is to maintain those
intrinsic variables near optimum long enough to pass on a structure description
to the next generation (whether it be by cloning, seed-spreading, or whatever).
The primary control system has this function.

I think that the place where we have a disagreement is how this primary
control system effects its control in an organism that can control other
percepts in its environment. (Can trees?) My preference is for a simgle
hierarchy, in which the primary control system has been elaborated to effect
its control through the provision of reference signals to other ECSs. Yours
is for the primary control system to be separate from another hierarchy, and
to effect its control by blind modification of that second hierarchy.

I don't think there are ground other than aesthetic (Occam's razor) for
choosing between these organizations, unless it can be shown that either
would not work. If you allow that your primary control system can act
by targetting local areas of the sensory-motor hierarchy, then I see little
possibility for distinguishing them on grounds of plausibility. They come
almost to mean the same thing in different words. Probably some differences
do remain: I think that maintenance of error, or more particularly the
uncontrolled growth of error, in an ECS seems a plausible reason for
reorganizing something about that ECS, whether it be the signs of some or
all of its outputs, the nature oits perceptual function, or even shutting down
or inverting its gain (are not the most intense missionaries the recently
converted?).

Have you modelled to reorganization of a moderately complex hierarchy? That
would be a lovely demo, if you have.

Finally:

I have proposed a version of targeting based on the
phenomenological idea that awareness directs reorganization to problem
areas. But having no model of awareness or attention, I haven't pushed that
very hard.

This would be the effect of "teaching" as opposed to learning, wouldn't it?
Wasn't this where we came in?

Martin

[_Martin_Taylor]

[Martin Taylor 920704 13:00]
(Bill Powers 920702.1600)

This is a slight reprise in the combinatoric problem. Perhaps I should have
titled the posting "combinatorics and conflict" but I like to keep the thread
going with a constant title, for later reference.

Putting the problem in terms of right versus wrong choices makes the
probability of organizing even one level of control seem incredibly small.
But I think this is the wrong way to set up the problem. EVERY form of a
perceptual function will yield a perceptual signal that is a regular
function of external events. There is not just exactly one combination of
inputs that will yield the "right" perception, with all others being
"wrong." There are many possible ways of perceiving a given environment
that will allow control, and many ways of exerting control that will have
at least some beneficial effect on intrinsic state. On the scale of
individual perceptual signals at the lowest levels, the number of equally
good alternatives must get astronomical. I think you're misstating the
combinatorial problem.

Yesterday I mentioned a symmetry argument that made Bill's point. I had
intended to add a note about conflict that makes it more forcefully, but
forgot. Here it is.

If the number of sensory degrees of freedom equal the number of action
degrees of freedom, then it is possible to organize control so that different
ECSs control independent percepts, and that all percepts can simultaneously
be maintained at their reference levels. Under these conditions, there
is a symmetry group of "right" perceptions and output links (a link, remember,
in this argument has a value -1, 0, or 1). Any member of the symmetry group
is an optimal control system. This reduces the combinatoric problem, but
still leaves the number of optimal control systems very small in the universe
of randomly connected control systems. I can't do the maths. Maybe someone
else can. But that's not the end of the story. It's where I left the story
yesterday.

The continuation of the story is based on there being far more sensory DoF
than action DoF (degrees of freedom). Under these conditions, it is not
possible for all percepts to be brought simultaneously to their reference
levels. There is intrinsic conflict (I use the word advisedly, because I
link it conceptually to the physico-chemical intrinsic variables that determine
survival). There should be some kind of metric for the amount of conflict,
that depends on the long-term average error over the control network.

If there is intrinsic conflict, there is some minimum possible level of
average error greater than zero. When that minimum is exactly zero, only
the symmetry group described earlier will represent an optimum organization.
But if some error is intrinsic, then a much larger group of organizations
will be optimum or very close to optimum. This corresponds to Bill's words:

There are many possible ways of perceiving a given environment
that will allow control, and many ways of exerting control that will have
at least some beneficial effect on intrinsic state. On the scale of
individual perceptual signals at the lowest levels, the number of equally
good alternatives must get astronomical.

I'm not sure how astronomical the number gets in comparison to the number that
are possible and bad, but certainly this argument justifies Bill's final
sentence (of the quoted paragraph:

I think you're misstating the combinatorial problem.

I was.

The problem then becomes a practical one. To what degree was I misstating, and
as a practical matter does the error affect the main thrust of the argument,
that random reorganization, untargetted within a control hierarchy, is
unlikely to achieve good results in a control system that is effectively
interacting with the real world?

The question can, in principle, be addressed by computational experiments,
but I think it would be hard in practice. To do the experiment, one would
have to design a model world with controllably many degrees of freedom, in
which the model hierarchy could be reorganized. There would have to be some
effect of its behaviour on some simulated intrinsic variables, and so forth.

I'll leave for another day the complications that arise when the behaviour of
the world is discontinuous.

Martin