Martin-Tom: #1

From Tom Bourbon (930623.0936)

Yesterday, I lost access to CSG-L for several hours. During that time,
Martin Taylor and I pursued a direct discussion about information, plans and
control. The discussion began on CSG-L asnd we thought we were still on
the net, but were not. We agree that parts of our private discussion might
be of interest to others following the threads on those topics;
consequently, we will post the discussion. It will come in two
installments -- Martin-Tom #1 and #2.

The discussion began on the net when I read a remark by Martin to Hans Bloom:

ยทยทยท

================================
Subject: Re: Power gain, power loss.

[Martin Taylor 930621 11:10]
(Hans Blom 930619)

..

On models, I tend to side with Hans. It is part of the whole information
argument. The more information is avaialble within the control system,
the less is to be acquired from the CEV through the perceptual apparatus,
and the better control can be.

==================

From Tom Bourbon (930621.1323)

[Martin Taylor 930621 11:10]
(Hans Blom 930619)

Misunderstandings sometimes are better resolved by a non-combatant.

Most of Martin's post was about power gain in a control system. But at one
point Martin said:

On models, I tend to side with Hans. It is part of the whole information
argument. The more information is avaialble within the control system,
the less is to be acquired from the CEV through the perceptual apparatus,
and the better control can be.

Martin, to paraphrase a line from the movie, "Field of Dreams," all I can
say is, "If you build it, we will come." Take one example of
control, as it is recreated or predicted by PCT models, and show me, in the
results of simulations, how making more "information" available "within the
control system" improves the recreations and predictions from the model.
Then show me that those results generalize, with no further tinkering with
the model, to new conditions, with unpredictably different disturbances and
targets. That is not much to ask. Just improve on the performance of a
single-level, single-loop PCT model.

And please delineate how your ideas in the remark to Hans differ from, say,
a plan driven system that relies on information in the form of
programs for action, thereby freeing itself from a need to rely on
information about the CEV obtained through the pereptual apparatus. As you
stated it, I see no difference.

This is not a put down. It is the only way to do business, if you rely on
models to test your assumptions. It is my often repeated plea that you
present the evidence, in the form of improved performance of the PCT model.
Nothing else will impress us or win us over. You already know that. But
I assure you that, if you build it, we will come.

=====================================

Inadvertently, the next exchanges were private, starting from Martin:

====================================

Subject: Re: Power gain, power loss.

[Martin Taylor 930621 18:00]
(Tom Bourbon 930621.1323)

And please delineate how your ideas in the remark to Hans differ from, say,
a plan driven system that relies on information in the form of
programs for action, thereby freeing itself from a need to rely on
information about the CEV obtained through the pereptual apparatus. As you
stated it, I see no difference.

The difference is in those words "relies on" and "freeing itself from." I
have no concept of either. Change them to "uses" and "reduces its
need for" respectively, and I have less of a problem.

Then show me that those results generalize, with no further tinkering with
the model, to new conditions, with unpredictably different disturbances and
targets.

What I assume would be in the model doesn't have much to do with the
disturbances and targets, but with what Bill has labelled f(e)--the
effect of a particular output change on the CEV. Reorganization is
one way of building a f(e) that conforms to a predetermined model, which
has the characteristic of being monotonic, as steep as can be constructed,
and leads to negative feedback. That "model" needs no explicit form.
It works with little information about the environment (which, Bill,
incorporates all the lower-level ECSs, not just the part of the world
outside the skin envelope) other than that the sign of the feedback is
constant and the environmental gain stays adequately high. Bats, on
the other hand, seem to adjust their perceptual input filters according
to the expected time and frequency of the (doppler-shifted) echo. They
need the model to distinguish the very low-power but precisely determined
echo from whatever else is going on in their acoustic world.

How could such a model work? In the neural-net world, one rather
powerful form of node is called a sigma-pi node. It does summation
and multiplication, and can be used as a variable filter. It would
be quite reasonable, I think, for a perceptual input function to contain
the pi part of the sigma-pi, in addition to the sigma that is generally
acknowledged to be there. The input to the pi could come from the
output signal, changing the relative sensitivity of different elements
of the PIF, and thereby changing its prior uncertainty about the expected
signal. That's just one way it could work.

I'm not committed (yet) to internal models in general. I can see their
potential usefulness, but they add a complexity to the ECS with which I am
not happy. In the syntax predictor that Allan is developing for me, we do
not include (yet) any internal model. We hope we will not need to include
one to achieve good prediction. We are starting by relying on perceptual
input functions that include differentiation. Nevertheless, when we
get to noisy, smoothly changing representations of the syntax, I am
at least open to the idea that we will have to incorporate models.

As I said, it's a question of the required information rate from
perceptual signals. If you are among those who consider it an
uninteresting quantity, you will not be interested in the possible
value of an internal model as a component of an ECS.

==================

Here's a counter-challenge to the skilled modellers. I think it is fair,
because we have not yet developed our own model, so we can see whether
anyone, ourselves included, can solve the problem.

Define a formal grammar (say a BNF grammar) with 3 levels between the
root and the leaves. Assert for each leaf symbol a description consisting
of a location in an arbitrary 3-space (by analogy, think of phonetic
feature values for phonemes). Let a control system "see" the succession
of locations defined by the successive symbols output by executing the
grammar with predefined probabilities of taking the different branches.
The ouput of the control system is a location in 3-space. The three
"intrinsic variables" that the control system must maintain are the
difference between the locations of the output symbols and its own
three dimensional output. The control system may be designed or it
may learn (ours will learn).

Obviously, if the grammar output moves very slowly, any 3-D control
system will work. Our problem is to get the control system to move
to the right place as early as possible, preferably in synchrony with
the motion of the grammar output point, which is moving quickly.

So far, we have not defined a challenge grammar or specified its rate
of output, but we assume that the output point will have to stay stable
for at least two compute cycles for the control system to have any chance
of learning. We think that our control system will learn to have about
as many levels as there are in the grammar, but that remains to be seen
(it will grow by inserting ECSs between the "intrinsic variable" control
ECSs and the top perceptual layer, as discussed last week).

=====================================

Subject: Re: Power gain, power loss.

From Tom Bourbon (930622.1231)

[Martin Taylor 930621 18:00]
(Tom Bourbon 930621.1323)

And please delineate how your ideas in the remark to Hans differ from, say,
a plan driven system that relies on information in the form of
programs for action, thereby freeing itself from a need to rely on
information about the CEV obtained through the pereptual apparatus. As you
stated it, I see no difference.

The difference is in those words "relies on" and "freeing itself from." I
have no concept of either. Change them to "uses" and "reduces its
need for" respectively, and I have less of a problem.

Fine. Change the words. Now, please because I still do not understand, tell
me how the model implied in your remarks to Hans differ from, say, a plan
driven system that "uses" information in the form of programs for action,
thereby "reducing its need for" information about the CEV obtained through
the perceptual apparatus. As you stated it, I see no difference.

Then show me that those results generalize, with no further tinkering with
the model, to new conditions, with unpredictably different disturbances and
targets.

What I assume would be in the model doesn't have much to do with the
disturbances and targets, but with what Bill has labelled f(e)--the
effect of a particular output change on the CEV. Reorganization is
one way of building a f(e) that conforms to a predetermined model, which
has the characteristic of being monotonic, as steep as can be constructed,
and leads to negative feedback. That "model" needs no explicit form.
It works with little information about the environment (which, Bill,
incorporates all the lower-level ECSs, not just the part of the world
outside the skin envelope) other than that the sign of the feedback is
constant and the environmental gain stays adequately high. Bats, on
the other hand, seem to adjust their perceptual input filters according
to the expected time and frequency of the (doppler-shifted) echo. They
need the model to distinguish the very low-power but precisely determined
echo from whatever else is going on in their acoustic world.

How could such a model work? In the neural-net world, one rather
powerful form of node is called a sigma-pi node. It does summation
and multiplication, and can be used as a variable filter. It would
be quite reasonable, I think, for a perceptual input function to contain
the pi part of the sigma-pi, in addition to the sigma that is generally
acknowledged to be there. The input to the pi could come from the
output signal, changing the relative sensitivity of different elements
of the PIF, and thereby changing its prior uncertainty about the expected
signal. That's just one way it could work.

I'm not committed (yet) to internal models in general. I can see their
potential usefulness, but they add a complexity to the ECS with which I am
not happy. In the syntax predictor that Allan is developing for me, we do
not include (yet) any internal model. We hope we will not need to include
one to achieve good prediction. We are starting by relying on perceptual
input functions that include differentiation. Nevertheless, when we
get to noisy, smoothly changing representations of the syntax, I am
at least open to the idea that we will have to incorporate models.

As I said, it's a question of the required information rate from
perceptual signals. If you are among those who consider it an
uninteresting quantity, you will not be interested in the possible
value of an internal model as a component of an ECS.

Please, all I asked was:

Then show me that those results generalize, with no further tinkering with
the model, to new conditions, with unpredictably different disturbances and
targets.

Of course, in the original I asked to see a generalization of the
results of simulations by the model you suggested. That is all I need to
see, for you to convince me that what you say about information theory
*does* translate into imnprovements in the performance of the PCT model.
In the demonstration, you are free (encouraged) to assume the model in its
fully developed and informed state. You need not simulate evolution,
conception, birth, maturation, learning, social control proceses, or
enlightenment. Simply take an extant PCT model, add to it the features or
measures you believe must be there for it to be an information theoretic PCT (IT
PCT)
model, and let it run. I described my criteria for improvement in other
posts long ago, and in one addressed to Hans Bloom a few minutes ago. A
demonstration like that would clear the air of gigabytes of "I said," "You
said," "We said," and the like. And it would focus the discussion on the
real issue -- does the PCT model work and, if so, can it be improved?

Here's a counter-challenge to the skilled modellers. I think it is fair,
because we have not yet developed our own model, so we can see whether
anyone, ourselves included, can solve the problem.

This is another kind of "challenge" entirely. In fact, my offer is not a
challenge. I am merely saying that we know the PCT model works for certain
instances of control by humans. We know the model can be and should be
improved. We are eager to enlist the support of anyone who wishes to join
in that endeavour. The criteria for demonstrating improvement in the model
are simple and direct. Have at it. We have even published and posted the
PCT model (all two lines of it, if you include the environment) many times,
so you can avoid the need to develop your own model. Please, use ours as a
testbed for your ideas. (I am completely serious -- no attempt by me to be
cute, clever or condescending.)

This is not a contest in which we try to prove prowess and skill -- not for
me it isn't -- I have neither of those "attributes." My skills are limited.
I would like to see people with skills and resources superior to my own
devote some of their time and creativity to working on our project.

Define a formal grammar (say a BNF grammar) with 3 levels between the
root and the leaves. Assert for each leaf symbol a description consisting
of a location in an arbitrary 3-space (by analogy, think of phonetic
feature values for phonemes). Let a control system "see" the succession
of locations defined by the successive symbols output by executing the
grammar with predefined probabilities of taking the different branches.
The ouput of the control system is a location in 3-space. The three
"intrinsic variables" that the control system must maintain are the
difference between the locations of the output symbols and its own
three dimensional output. The control system may be designed or it
may learn (ours will learn).

Obviously, if the grammar output moves very slowly, any 3-D control
system will work. Our problem is to get the control system to move
to the right place as early as possible, preferably in synchrony with
the motion of the grammar output point, which is moving quickly.

So far, we have not defined a challenge grammar or specified its rate
of output, but we assume that the output point will have to stay stable
for at least two compute cycles for the control system to have any chance
of learning. We think that our control system will learn to have about
as many levels as there are in the grammar, but that remains to be seen
(it will grow by inserting ECSs between the "intrinsic variable" control
ECSs and the top perceptual layer, as discussed last week).

You got me there, Martin. Congratulations. I sure can't do that, but then
I never claimed to be a skilled modeler. Now, can I interest you in
helping us figure out how to improve the PCT model for something as mundane
and trivial as stick wiggling?

=====================================

From Martin Taylor, 22 June 1993

Subject: Re: Power gain, power loss.

Tom,

You responded to me personally, so I do the same. Did I mail to you,
rather than posting to CSG-L? I don't remember.

As I understand it, the problem with outflow plans is that they cannot
work because the world both is disturbed by influences unknown to the
pseudo-control system and because the impact of the output on the thing
to be "controlled" is not always the same. If the impact were defined
with high probability, and if the thing to be controlled were effectively
isolated from disturbance, outflow planning would work.

Do we agree so far?
------------------------
[Tom -- present time -- 23 June]

Yes.
------------------------
[Martin]

Now:

[Martin]

The difference is in those words "relies on" and "freeing itself from." I
have no concept of either. Change them to "uses" and "reduces its
need for" respectively, and I have less of a problem.

[Tom]

Fine. Change the words. Now, please because I still do not understand, tell
me how the model implied in your remarks to Hans differ from, say, a plan
driven system that "uses" information in the form of programs for action,
thereby "reducing its need for" information about the CEV obtained through
the perceptual apparatus. As you stated it, I see no difference.

[Martin]
If you use "relies on" and "freeing itself from", you are talking about
a plan-driven outflow system. If you say "uses" and "reduces its need for"
you are talking about a normal control system that probabilistically
anticipates outputs that might affect the world usefully, but remains
based on the present difference between current perception and current
reference.

----------------
[Tom - now]

I am not sure how a normal control system "probabilistically anticipates
outputs that might affect the world usefully," unless you mean something
like evaluating programs in imagination mode, then using one that seemed to
work sufficiently well in imagination. If that is the case, the program is
for perceptions, not actions, otherwise there is no difference between the
wordings you compared: there is not much difference, if any, between
"plan-driven outflow" and "outputs that might affect the world usefully."

----------------------

[Tom, previously]

Of course, in the original I asked to see a generalization of the
results of simulations by the model you suggested. That is all I need to
see, for you to convince me that what you say about information theory
*does* translate into imnprovements in the performance of the PCT model.

[Martin]
You use the example of a sawtooth tracking task quite often in your
demonstrations. What do people do if after many cycles of the sawtooth
you stop the target at the mid-point and leave it there. Doesn't the
tracker overshoot before coming back to the target? But does the tracker
overshoot when the target reverses direction at the peaks of the sawtooth?
I think not, at least not after the first few cycles.

I haven't tried this, but here is a place where I make a prediction that
you have plenty of data to test. I am guessing that a simple ECS model
of one level tuned to a best fit to the human data will fail in two
specific places: (1) There will be a reversal in the sense of the
predictive miss at the peak between (a) the first one or two peaks, and
(b) peaks late in the sequence; (2) if you stop the target motion at a peak
after many tracking cycles, the human will reverse and come back, whereas
the model will not. The sawtooth should be fast enough that the human
tracks well, but measurably imperfectly.

Is this right?

----------------------
[Tom -- now]

In examples I post on the net, I often use sawtooth targets, but only
because I can draw them in ASCII. I haven't figured out how to draw random
target paths. But since you mentioned them, events will occur as you
described them in the hypothetical demonstration -- if you stop the target,
the person will overshoot, then come back. (And will also reveal the
presence of other levels, by looking at you, or at the computer, and saying
things like, "it stopped working," or "what happened?")

However, the "overshoot" probably is not a result of "predictive movements."
The situation looks very much like the one described by Rick and Bill in
their chapter, "Levels of Intention in Behavior," in Wayne Hershberger's
book, *Volitional Action: Conation and Control*, (1989, North-Holland, for
those who do not know the book). What they studied, and what you describe,
is the result of differences in time constants for different levels in a
hierarchical control system. The "overshoot" you describe in the
demonstration, like that they describe in their chapter, occurs when for a
brief time a lower level system continues operating on the "old" reference
signal from above. It takes time for the higher level to detect error
which produces the new reference signal for the lower level. During the
time between onset of changes in the rules in the enviro\nment, and
alteration of the error signal qua reference signal from the higher level,
the lower level keeps on with the originally "correct," but now "incorrect,"
reference signal -- it keeps working perfectly.

-------------------------

[Martin]
I'm actually rather surprised to think that the "world-model" idea
might work at this low level. I had thought of it as being more
useful above the category level, and particularly at the program
level and above. But the theoretical background makes no such
distinction of levels, and if it actually does work as I presume,
the result should abort a lot of future fruitless discussion. The
question will (I hope) turn to how the world model information actually
is implemented in an ECS. Is it in the output function, the perceptual
input function, or (as I presume) in the imagination loop?

----------------
[Tom -- now]
A world model (in imagination mode?) must affect the performance of the
system at *every* level, once the system begins to act on the model.
----------------

[Martin]
As for the word "challenge," I agree with your comments. It is an ill-chosen
word. However, much of what has been going on has the flavour of challenge,
and the word came quite naturally. I would be much happier if it didn't.
And I'm not the modeller. That's not my skill, either. I'm more of a
theoretician (as you doubtless have observed). Allan is doing the work.

=======================

[Tom -- now]

Fine! I like the idea of cooperation or collaboration, perhaps with a
bit of heat to liven things up, better than challenges. (Care for a round
of stick wiggling?)

That has us almost caught up. I will send a few of today's posts a little
later.

Until later,
  Tom Bourbon