Reorganization and Trial-and-Error - WHOA!

[From Bill Powers (2009.08.04.2204 MDT)]

Ted Cloak (2009.08.04.1318 MST) –

TC: Whoa! I just flashed on the
big mistake, PCT-wise, in Pinker’s metaphor; which I think beautifully
illustrates the limitation of conventional cognitive neuroscience as a
behavior-explainer.

BP: Check out demo 8-3 in LCS3. It’s the Little Man with better graphics.
You can move a target around in 3-space and the arm and hand will track
it. The arm has 14 degrees of freedom, of which three are used by the
level that does the tracking. There are no computations of joint angle,
either forward or backward.
It’s interesting to put your finger on the tip of your nose, then reach
out and touch a corner of the computer screen or any other handy target.
Do this a few times, then close your eyes, move your finger from nose to
the target, and open your eyes.
I’m off about an inch at arm’s length. I’m also off the tip of my nose by
some amount, too, when I go back and forth several times with my eyes
shut.
Actually, an inch of error at arm’s length isn’t bad. But pay attention
to how you do it. In my case, I don’t do it by adjusting joint angles; I
do it by imagining where the tip of the finger is. I have a sort of image
of my whole arm and the space it’s in, and I control that image. With my
eyes shut, the only way I can estimate where my finger is is from
sensing the current joint angles. That’s not very accurate, but
it’s more accurate than I expected. This shows about how much can be
expected from model-based control. Ordinary negative feedback control of
perception is far more accurate.

There seems to be a spatial map in the head in which visual images and
kinesthetic sensations are adjusted to give about the same perceptions.
In experiments with vision, various means have been used to distort this
map, like the famous one with prisms. A person can, with much practice,
learn to ride a bicycle again while wearing the prism glasses, but it
takes a lot of practice. One of the funny bits about the experiment was
that when it was over, the bicycle rider gave the glasses back to the
experimenter and fell off the bicycle. It took about as long to
recalibrate the map while going back to normal.

If a disturbance is applied to the arm when you reach out, you’ll be off
by a lot more when your eyes are closed. This was especially true in the
experiments done by Isaac Kurtzer at Brandeis, in which the subject sat
in a room that could be rotated. In reaching radially outward, the arm
was subject to a sideways Coriolis force that depended on the speed of
rotation. No skin pressure was sensed, of course; the arm just didn’t go
the way the subject wanted. Initially, the disturbing force simply caused
the arm to deviate. With eyes-open observation of the result of each
reach, subjects eventually learned to reach straight out toward the
target with their eyes closed. Of course after that, when the room
rotation was stopped, they deviated the other way!

TC: Can Pinker’s metaphor of
equation-solving be reconciled with the idea of reorganization?

The fetus gaining control of his limbs starts out wildly, just thrashing
in all directions. But there are built-in preexisting selectors at
each level which tell each next lower level control system whether its
reference standards are getting better or worse, and it remembers. So
eventually, when a higher level center is controlling the perception Salt
Cellar in Hand, say, the lower level centers solve their equations with a
minimum of trial and error. The hand smoothly, “effortlessly” reaches out
and picks up the salt cellar. But there’s always some trial and error
involved, every time. We just stop noticing it.

Look at Demo 8-1, ArmControlReorg. Here we have an arm with 14 degrees of
freedom which starts out with 14 incomplete control systems. Each control
system’s output is connected to ALL of the joint angles through a total
of 196 weights. There are 14 disturbances acting on the joint angles, and
there are 14 reference signals, five of which can be set to vary in
several different patterns. The weightings all start out at zero, and the
reorganizing system, using only the information in the error signals,
gradually does an E. coli reorganization on all 14 systems that ends up
with all joints under good control. You can pause the reorganization at
any time and see how good the control is, then restart it and see the
control improve some more.

An interesting undocumented feature of this demo is that you can turn off
the reference signal variations and let the system learn control simply
by learning to oppose the disturbances. After learning has neared an
asymptote, you can turn off the disturbances and turn on a smooth pattern
of reference signals, and control will be just as good, with no further
reorganization. This works the other way, too: train with varying
reference signals only, and observe that the arm will resist disturbances
just as well as if trained with the disturbances acting.

This is a direct demonstration of learning control systems, not
behaviors.

Pinker, despite his facility with language, is nowhere near understanding
any of this.

Best,

Bill P.