Model-real switching; learning/reorg; explained data

[From Bill Powers (960302.0730 MST)]

Martin Taylor 960301 1515 --

     In Bill Powers' approach to reorganization, the top level
     perceptual references are all zero.

There are several possibilities for any given highest-level reference
signal: it is zero; it is the average value (over some time) of the
perceptual signal; it is a value set at random through reorganization;
it is genetically fixed. I've never settled on one explanation. During
maturation, the level that is "highest" keeps changing.

     Learning has nothing to do with reference levels, anyway. It has to
     do with the functions and the linkages that produce negative loop
     gains (that is to say: control)

Got to be careful about the way people will interpret "learning": this
is true of "reorganization" but not necessarily of the broader and
looser term, "learning." One of the first things I do when I meet a new
person is to learn his or her name. But that's not reorganization.

I agree generally with your comments to Shannon.

···

------------------------------------------------------------------------
Martin Taylor 960301 17:00 --

     I have the same problem with Bill P's switch between real-time and
     model-based control. Why either-or?

Well, just figure out what happens if you have both. You have two
perceptual signals, one from the model and one from the real world. How
do you combine them to yield "the" perceptual signal that is compared
with the reference signal? Do you add them together? Average them? And
what happens when the real perceptual signal is lost? If you're
averaging them, then the net perceptual signal suddenly is halved!

I think that what really happens is that you operate with the real
perceptual signal as long as it's available, because that gives you the
best control. When that signal disappears, your behavior immediately
starts to run away, but this is caught by a higher system which switches
(when possible) to a model that allows you to carry on for a bit longer.
If there is no good model, then you switch to controlling some other
perception that is related to the control task, hoping it is good enough
to keep things under control until the real perception comes back. When
you drive from sunlight into an unfamiliar dark tunnel, you switch to
keeping the steering wheel in a fixed position and hope that there's no
curve in the tunnel that will get you before your eyes adapt. Model-
based control is really useful only when the environment is familiar and
you can count on there being no surprises.
-----------------------------------------------------------------------
Shannon Williams (960301.1815) --

     The diagram that I sent earlier is just a PCT diagram with some
     neural nets thrown in. Does that constitute a model? If not, then
     tell me what I need to do to make that a model.

Pick some real task that can be implemented by a human being interacting
with a computer. Write a program organized as your diagram is organized
to carry out the task. Compare the behavior of your program with the
behavior of a real person doing the same task. Modify the program until
it behaves like the person. Then you have a usable model.

If you aren't at the stage of experimentation with real people, then you
can _propose_ a model. To propose a model, you write a program that is
organized like your diagram (as you understand it), run it with a
simulated task, and see what the program actually does (as opposed,
possibly, to what you thought it would do). This at least demonstrates
that what you say about your diagram is true. You have a working model.
It may not behave like a real person yet, but at least it behaves.

If you don't know how to write a program that is organized like your
diagram, then you may be working toward a model, but you don't have one
yet. Many parts of HPCT are still at that stage. Some have gone to the
stage of working models. A few have gone to the stage of usable models,
where the model produces behavior very close to that of a real person.

When I say "write a program" I mean "Demonstrate in some way that a
system organized as your diagram is organized would really do the things
you say it would do." Programming a computer is a very easy way to test
a model, but any real test that shows how the model would actually
behave would do as well. It's very easy to draw a diagram and make
claims about what a system organized like that would do. But showing
that it actually would is a horse of a different feather.
--------------------------
     Tell me what data has been explained by HPCT, and I will tell you
     how it is explained by a PCT diagram with neural nets.

A person holds a control handle that moves a cursor on a computer
display screen. Also on the screen is a target that can move. Inside the
computer, two disturbance generators produce smoothed random patterns of
disturbing variables, obtained by filtering the output of a pseudo-
random number generator. One disturbance moves the target, the other is
added to the effect of the control handle on the cursor.

A control-system model recieves information about target and cursor
position, and outputs a number that affects the cursor, with the same
disturbance patterns acting. The parameters of this model are adjusted
until the simulated handle movements match (as well as possible) the
handle movements of a real person doing the same task.

Then the model is run again using a new randomly-generated pair of
disturbances, using the parameters obtained from the first experiment,
to create a predicted pattern of handle movements. The new disturbance
patterns correlate less than 0.2 with the old ones. After this is done,
the person does the same task with the same (new) pattern of
disturbances. The handle movements of the person are compared with the
stimulated handle movements of the model.

The predicted handle movements deviate by 5 to 10 percent RMS from the
real ones (comparing them point by point), and the correlation between
model and real handle movements is somewhere between 0.99 and 0.999.
-----------------------------------------------------------------------
Best to all,

Bill P.

[Martin Taylor 960303 01:12]

Bill Powers (960302.0730 MST)

Martin Taylor 960301 17:00 --

    I have the same problem with Bill P's switch between real-time and
    model-based control. Why either-or?

Well, just figure out what happens if you have both. You have two
perceptual signals, one from the model and one from the real world. How
do you combine them to yield "the" perceptual signal that is compared
with the reference signal? Do you add them together? Average them? And
what happens when the real perceptual signal is lost? If you're
averaging them, then the net perceptual signal suddenly is halved!

I hadn't conceived of either summation or averaging, but of a merging
of information to provide a result that might be more accurate than either.
But I'll grant that such a connection isn't the kind of simple thing that
might be done in one neuron.

The genesis of my "same problem" is the transient that occurs at the
switching moment. The transient is inevitable if the real-time signal
is suddenly lost, but that is often not the case. In dealing with the
world, one may well be using sight, sound, and touch on the same object.
One of them may be telling you more about the state of a perception
you are interested than another is, and at the same time you may be
using model-based information. I am thinking of an example of examining
food at a reputable or disreputable store. In the reputable store, you
will examine it less carefully before deciding it matches your reference
for "good fruit" than you will at the disreputable store. You take more
data from the model "this store sells only good stuff" than from your
seeing the right colours, feeling the right texture, and hearing the
right crackle (or whatever). You don't do without the real-time, but
neither do you ignore the model.

Perhaps this kind of thing happens only at higher levels. But I'm always
a bit wary of structures that inherently introduce transients into the
system operation.

···

----------
Shannon Williams

There are neural networks that require teachers and neural networks that
don't. And I am getting more and more mystified with your repeated
statements that HPCT doesn't deal with learning. Your recent messages
leave me with the distinct impression that you really don't yet know
what people are talking about in dealing with HPCT, and now I have a
similar impression with respect to your understanding of neural networks.

I'm sorry to be snarky, but to misquote Barry Goldwater, arrogance in
support of ignorance is no virtue. I'm sure you have lots of good ideas
in what you would like to model, and that some of these ideas are
likely to prove useful tests of or additions to HPCT, but you are much
better at presenting those than you are at commenting on HPCT or neural
networks.

Bill Powers has reminded me that not all learning in HPCT is through
reorganization; there is also memory for specific items and events. But
even within the concept of reorganization there are at least 12 different
kinds of learning possible, some of them identical to standard neural
network learning approaches (without teachers). What has not been done
within HPCT is to assess these 12 different possibilities to see which,
if any, are used in what parts of the hierarchy under what circumstances.
What has been done is a modest amount of modelling to see what the
possibilities are (Allan Randall did also start one project to try to
use the most trivial kind of reorganization algorithm to get a small
network to learn a simple syntax, but the money ran out before we got
very far).

Like Bill, I find your language bewildering, when on one line you say
you don't think of behaviour as being emitted, and shortly thereafter
use language like:

When you talk about two people solving their conflicts by "going up
a level", you are not talking about changing one's focus from
emitting behavior to determining what behavior to emit.

In PCT (nevermind HPCT) you deal with what perceptions to achieve, since
you cannot determine what behaviour to emit unless you have a perception
for the behaviour itself. So your statement is literally correct, since
when we talk about "going up a level" we aren't talking about emitting
anything.

When we talk about "going up a level" we mean that we cease dealing with
the set of perceptions about which we were talking, and try to deal
with the perception(s) whose control needs the control of the lower
level ones we were talking about. Instead of fighting about who gets
the apple, the kids deal with why they want the apple--is it to test
dominance, is it to eat something sweet, is it to see the other unhappy?
Wanting the apple is one level, wanting to feel the taste and texture
of the apple is another, as is wanting to show one's dominance over the
other. By going up a level, you can sometimes resolve a conflict by
noting that there are other perceptions whose control would lead to the
desired result.

Martin