# model-based control

[Martin Taylor 970225 14:45]

Rick Marken (970224.2240 PST)]

Bill Powers (970224.1135 MST) --

> Rick, I think you'd better explain your model of the sine-wave
> stuff again. I was under the impression that you were using
> rate-plus-proportional perceptual feedback in the second level
> of control, which of course would put a phase advance into the
> reference signal of the lower-level system.

The sine wave stuff I am thinking of was done at least four years
ago -- maybe five.

Actually, three, if you were doing it at the time you discussed it on
CSGnet.

I don't even have the programs anymore; I think
I was writing this stuff as HyperCard stacks. All I remember about
it was that I was trying to think (not very effectively, as it
turned out) of a way to show two levels of control (in a behaving
subject) happening at the same time. I was writing the two level models
to see if there was some way to tease out two levels of performance in
the model; I would then test the subject with the method that worked on
the model.

That's all I remember; I think my ideas never got less vague than that.
I think I played around with sine wave disturbances of
different frequencies applied to perceptions controlled by
different levels of the model.

Here's a memory refresher of what we both said:

···

------------------------------------
+Rick Marken (940202.1330)

+The basic "levels revealing" experiment is pretty simple. The subject
+does the tracking task as usual (trying to keep c = t). In one
+condition of the experiment, dt is a smoothed random series (the
+usual PCT kind); in the other condition , dt is a sine wave (of
+about the same frequency as the center frequency of the random
+series-- about .3Hz). In both conditions, the disturbance to the
+cursor, dc, is exactly the same.
+...
+Here are some representative results with yours truly as subject:
+
+ Type of Target Movement (dt)
+ Random Sine Wave
+
+Pursuit 6.88 15.8
+
+Comp 3.7 2.1
+
+
+The numbers are stability factors (S) indicating the ability
+to control the pursuit (t-c) and compensatory (c) variables
+when target movement was random vs predictable (sine).
+Very nearly the same S values were obtained every time I
+did the experiment. The interesting result is that, while pursuit
+control (the ability to control t-c) improves substantially
+(it more than doubles, going from 6.88 to 15.8) when the
+target is predictable (a well known fact), compensatory control
+(control of the position of the cursor relative to a changing
+reference level) remains the same (or even declines). In fact,
+the apparent decline in compensatory control is an artifact
+of the difference in the variance of the reference level in the
+random and sine wave target movement conditions. I found this
+out by running a single level control model (with parameters
+adjusted for best fit) in both conditions.
+...
+Stronger evidence comes from my preliminary attempts to develop
+a two level model of this task. The lowest level is almost
+identical to the single level of the first model but the reference
+signal is now the predicted position of the target, t', rather than
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+the target itseld (as it was, functionally, in the first model). I
+haven't really modelled the second level system yet (the one that
+produces the predicted target positions -- which are reference inputs
+to the lower level system). I got the predicted target positions, t',
+for the model by sampling ahead in the sine distrubance table.
+The model that generated stability factors closest to the one's
+observed sampled the equivalent of [ ] ahead. [ Martin: If you are
+predict the since disturbance in order to match the subject's
+performance?]. Here are the results for this "two level" model:
+
+ Type of Target Movement (dt)
+ Random Sine Wave
+
+Pursuit 6.67 15.85
+
+Comp 3.78 2.4
+
+(The one level model was used again to get the data in the
+random condition).

(To which I replied [Martin Taylor 940204 18:30]:
-Very roughly, 180 msec according to my calculations. My intuition would
-have said about double that. I don't know which to trust. But I'll have
-to go with 180 msec unless I can find a mistake in the calculation. You
-obviously know the right answer, so we'll see.
-
-> But note that the model's
->ability to control the compensatory variable is almost exactly
->the same as the subject's IN BOTH CONDITIONS. This is strong
->evidence that we are looking at two levels of control when the
->SUBJECT is controlling with the predictable target.
-
-I don't see the logic here. To me it is strong evidence that the subject
-is using the prediction information, but that could be manifest in any
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^
-manner, including multi-level control. You have shown that you can make
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-a 2-level model that produces a good match to the human data, but you
-haven't shown what the second level perceives or is controlling. All
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-you have done is to provide a prediction as a reference signal. You as
-experimenter know where this prediction comes from. Where does the
-hypothesized second-level control system get it from?

and later [Martin Taylor 940205 12:30]:

-Yes, I think I found a mistake. ...340 msec, which is
-my corrected estimate of the lead time for Rick's predictor reference
-signal, in place of the 180 msec I gave in my previous response.
-
-Maybe this is all wrong, but I'll go with it, at least for now.

To which Rick answered [Rick Marken (940205.1930)]:
+>Very roughly, 180 msec according to my calculations.
+
+That's RIGHT! And based on pretty reasonable guesses. But did
+post where you revised your estimate to the wrong value) had nothing
+to do with IT?

Later again [Martin Taylor 940209 14:00]

-based on intuition and a bit of calculation (180 msec) that I realized
-the next morning had an algebraic error. Fixing the error led to a
-calculated result of 340 msec. Rick let me know that 180 was right,
-so I went back to see whether there was any further obvious error in the
-information-theory part of the argument. In doing so, I came up with
-3 problems. One is very simple. I had made a second algebraic error
-(something CSG-L readers know I am prone to do). I had assumed that
-Rick's "stability factor" represented the ratio between the squares
-of the actual disturbance and the (actual-prediction) equivalent
-disturbance, rather than what he had written that it was. When this
-error is corrected, the prediction lead comes out to about 230 msec. Still
-wrong, but wrong on a more legitimate ground.

------------------That's enough history----------

I think the foregoing probably represents what happened better than
Rick's vague memory does, or than mine did. Rick's model using the
predictor of the sine wave (his words, at the time, though he now
disclaims the idea) fit his own (human) data better than did a model
without the predictor. And it was possible to compute the magnitude
of the predictive lead time, at least to within the right order of
magnitude.

And I don't think that the problem was (as Rick suggested a couple of
days ago) that the 0.3 Hz sine-wave disturbance was too fast for effective
control.

I don't think I ever got to the
point where I successfully derived the varying reference input to
the lower level system from the error in the higher level system.

No you didn't. You used your designed-in exact predictive simulation
(to use the word Bill P. prefers to "model") of the disturbance, instead.

Interesting how words based on "predict" are prominent in your 1994
messages in explaining what you did, but are wrong in describing it when
you talk about it in 1997.

Martin

[From Bill Powers (970226.0555 MST)]

Martin Taylor 970225 14:45 --

+Here are some representative results with yours truly as subject:
+
+ Type of Target Movement (dt)
+ Random Sine Wave
+
+Pursuit 6.88 15.8
+
+Comp 3.7 2.1
+
+
+The numbers are stability factors (S) indicating the ability
+to control the pursuit (t-c) and compensatory (c) variables
+when target movement was random vs predictable (sine).
+Very nearly the same S values were obtained every time I
+did the experiment. The interesting result is that, while pursuit
+control (the ability to control t-c) improves substantially
+(it more than doubles, going from 6.88 to 15.8) when the
+target is predictable (a well known fact), compensatory control
+(control of the position of the cursor relative to a changing
+reference level) remains the same (or even declines). In fact,
+the apparent decline in compensatory control is an artifact
+of the difference in the variance of the reference level in the
+random and sine wave target movement conditions. I found this
+out by running a single level control model (with parameters
+adjusted for best fit) in both conditions.

The results you report are not necessarily evidence about the difference
between predictable and unpredictable control. If you know only that the
sine-wave disturbance is at the CENTER frequency of the bandwidth of the
random disturbance, you still don't know where the center frequency is
relative to the bandwidth of control. If, for example, the corner frequency
of the bandwidth of control was 0.3 Hz (loop gain equal to 0.7 of the
low-frequency gain), half of the random disturbance spectrum would be
outside the control bandwidth. This by itself could explain the differences
in stability factor between the left the right columns.

The only way to compare predictive and nonpredictive control by this method
would be to run the tracking error through a narrow-band filter at the
sine-wave disturbance frequency, so you're looking at the same band of
frequencies with both types of disturbance, and _then_ compute a measure of
the control error. This assumes a linear system, so the measure wouldn't be
very accurate, but you're looking at some extreme differences here. Each
whole number in the stability factor (if you're computing it as defined in
the _Spadework_ paper) represents approximately one standard deviation, so
for the pursuit tracking results, you're talking about almost 9 standard
deviations of difference in the variance. Compared with the sine-wave
situation, the random situation shows almost no control at all.

Control data are very sensitive to losses of control, when you get the
errors down to 10% RMS of the disturbance amplitude or so. Even a momentary
lapse of control lasting only a few seconds can reduce the correlation
between modeled and real behavior from 0.99 to 0.9. We know that a random
disturbance with a bandwidth from 0 to 0.6 Hz makes control VERY much more
difficult than a disturbance with a bandwidth from 0 to 0.3 Hz. So when
frequency components above 0.3 Hz are included, they would contribute a very
large amount to the uncorrected error.

The best measure of a difference between random and sine-wave disturbance
effects is the delay in the control loop, which we determine by simulating
the control system and matching parameters. If the sine-wave disturbance
really involves prediction, then the delay should be substantially smaller
than for the random disturbance. In fact, it should average zero, with as
many errors due to premature error correction as due to late error
correction. I don't recall seeing any effect on delay due to differences in
disturbance bandwidth, although one would expect low-bandwidth disturbances
to be proportionally more predictable than high-bandwidth disturbances.
I have never compared the delays for sine-wave and random disturbances. This
would be worth doing.

Best,

Bill P.

[From Rick Marken (970226.0830)]

Martin Taylor (970225 14:45) --

Here's a memory refresher of what we both said:

Thanks. Sometimes I forget how smart I am;-)

I see that I was trying to develop a "levels revealing experiment"
and that I was approaching it in terms of predictable (sinusoidal) and
unpredictable (random) disturbances when these disturbances are visible
(pursuit tracking) and not visible (compensatory tracking). The basic
results were reported in terms of stability. The problem here was to
equate the disturbances in terms of control difficulty. That's what the
compensatory task was used for. By running the subject and a single
level model in a compensatory task with both disturbances I found that
the predictable disturbance was actually harder to control than the
unpredictable one (this is how I delt with the problem that Bill
Powers (970226.0555 MST) mentions in his post, mistakenly [and
astonishingly;-)] assuming that the research reported was done by you
rather than me).

The single level model matched the results of the compensatory task,
where it was found that control of the cursor was actually _worse_
with the predictable disturbance. Since control was much _better_
with the same predictable disturbance in the pursuit condition it
was clear that two levels of control are involved in the poursuit
tracking situation. I was starting to develop a model of the behavior
in the pursuit situation. In a sense I built only half of the model;
the higher level system was simply a signal source; and this signal
was derived by sampling from the sine table that provided the
predictable disturbance. Obviously the real system could not be
setting its reference that way. I was starting to work on a real,
two level model of the situation but got distracted. Maybe I'll
start on it again. Thanks for reminding me, Martin!

Anyway, Martin goes on to say:

I think the foregoing probably represents what happened better than
Rick's vague memory does, or than mine did. Rick's model using the
predictor of the sine wave (his words, at the time, though he now
disclaims the idea) fit his own (human) data better than did a model
without the predictor. And it was possible to compute the magnitude
of the predictive lead time, at least to within the right order of
magnitude.

Yes. But this was just a pseudo-model. The "predictor" reference signal
was (as I noted) derived by sampling ahead in the sine wave disturbance
table. I just wanted to see what would happen if a higher level system
_could_ provide a reference to the lower level system that was an
estimate of the future position of the target. But the model that
produced that "predictive" reference was not built! I never got to that
point. The model would have to have a higher order system that controls
a temporal pattern of cursor movment (a fixed reference to this system
would specify a sine wave perception of a particular phase, frequency
and amplitude, for example) by varying the reference to the lower level
system that controls cursor position. This is the model that was never
buiilt.

Interesting how words based on "predict" are prominent in your 1994
messages in explaining what you did, but are wrong in describing it
when you talk about it in 1997.

The position of a sinusoidally moving target is predicatable; the
position of a randomly moving target is not. A model that tracks a
sine wave target could use prediction (calculate the expected
position of the target at time t+dt based on it's position at
time t) but the PCT model I was going to build would involve no
prediction. It would control a "transition" or "temporal
configuration" perception: -- the perception of sinusoidal motion
of the cursor -- by varying the reference to a lower level system that
is controlling cursor position.

Best

Rick

[Martin Taylor 970226 12:10]

Bill Powers (970226.0555 MST)]

I'm a bit puzzled by this message:

Martin Taylor 970225 14:45 --
>+Here are some representative results with yours truly as subject:
>+
>+ Type of Target Movement (dt)
>+ Random Sine Wave
>+
>+Pursuit 6.88 15.8
>+
>+Comp 3.7 2.1
>+
>+
>+The numbers are stability factors (S) indicating the ability
>+to control the pursuit (t-c) and compensatory (c) variables
>+when target movement was random vs predictable (sine).
>+Very nearly the same S values were obtained every time I
>+did the experiment. The interesting result is that, while pursuit
>+control (the ability to control t-c) improves substantially
>+(it more than doubles, going from 6.88 to 15.8) when the
>+target is predictable (a well known fact), compensatory control
>+(control of the position of the cursor relative to a changing
>+reference level) remains the same (or even declines). In fact,
>+the apparent decline in compensatory control is an artifact
>+of the difference in the variance of the reference level in the
>+random and sine wave target movement conditions. I found this
>+out by running a single level control model (with parameters
>+adjusted for best fit) in both conditions.

The results you report are not necessarily evidence about the difference
between predictable and unpredictable control.

The reasons for puzzlement are twofold: (1) I'm not reporting any results.
I'm quoting Rick's message of the cited date. (2) I'm not citing the
results as evidence about the difference between predictable and
unpredictable control. Rick cited them in the process of showing that
a model with predictive control fitted the human data better than did
a model without predictive control. The time-advance of the prediction
that fitted best was about 200 msec. (I use "model" here to mean a
computed simulation of a control system).

So I don't know what the rest of your message is about. (I'm leaving the
quote in, so perhaps you will be able to point me to the bit or bits
that might be relevant.

If you know only that the
sine-wave disturbance is at the CENTER frequency of the bandwidth of the
random disturbance, you still don't know where the center frequency is
relative to the bandwidth of control. If, for example, the corner frequency
of the bandwidth of control was 0.3 Hz (loop gain equal to 0.7 of the
low-frequency gain), half of the random disturbance spectrum would be
outside the control bandwidth. This by itself could explain the differences
in stability factor between the left the right columns.

The only way to compare predictive and nonpredictive control by this method
would be to run the tracking error through a narrow-band filter at the
sine-wave disturbance frequency, so you're looking at the same band of
frequencies with both types of disturbance, and _then_ compute a measure of
the control error. This assumes a linear system, so the measure wouldn't be
very accurate, but you're looking at some extreme differences here. Each
whole number in the stability factor (if you're computing it as defined in
the _Spadework_ paper) represents approximately one standard deviation, so
for the pursuit tracking results, you're talking about almost 9 standard
deviations of difference in the variance. Compared with the sine-wave
situation, the random situation shows almost no control at all.

Control data are very sensitive to losses of control, when you get the
errors down to 10% RMS of the disturbance amplitude or so. Even a momentary
lapse of control lasting only a few seconds can reduce the correlation
between modeled and real behavior from 0.99 to 0.9. We know that a random
disturbance with a bandwidth from 0 to 0.6 Hz makes control VERY much more
difficult than a disturbance with a bandwidth from 0 to 0.3 Hz. So when
frequency components above 0.3 Hz are included, they would contribute a very
large amount to the uncorrected error.

The best measure of a difference between random and sine-wave disturbance
effects is the delay in the control loop, which we determine by simulating
the control system and matching parameters. If the sine-wave disturbance
really involves prediction, then the delay should be substantially smaller
than for the random disturbance. In fact, it should average zero, with as
many errors due to premature error correction as due to late error
correction. I don't recall seeing any effect on delay due to differences in
disturbance bandwidth, although one would expect low-bandwidth disturbances
to be proportionally more predictable than high-bandwidth disturbances.
I have never compared the delays for sine-wave and random disturbances. This
would be worth doing.

Martin

[From Bill Powers (970226.2005 MST)]

Martin Taylor 970226 12:10 --

I'm a bit puzzled by this message:

Martin Taylor 970225 14:45 --
>+Here are some representative results with yours truly as subject:
>+
>+ Type of Target Movement (dt)
>+ Random Sine Wave
>+
>+Pursuit 6.88 15.8
>+
>+Comp 3.7 2.1
>+

results would have come from Rick. I don't see anything to take back,
however. I should have caught this at the time. Nothing I said, however,
relates to the difference between pursuit and compensatory tracking. I find
that fairly mysterious, since my own tracking doesn't seem to be much
different in the two situations (although I haven't really given it a hard
look).

I can well understand how projects like this get abandoned. When you start
getting into the details -- the hard part -- you want to talk them over with
someone, and you realize that there's nobody else there. So what's the
difference if you finish the project or not?

Best,

Bill P.

[Martin Taylor 970303 11:50]

Rick Marken (970226.0830)]

Martin Taylor (970225 14:45) --

> Here's a memory refresher of what we both said:

Thanks. Sometimes I forget how smart I am;-)

I see that I was trying to develop a "levels revealing experiment"
and that I was approaching it in terms of predictable (sinusoidal) and
unpredictable (random) disturbances when these disturbances are visible
(pursuit tracking) and not visible (compensatory tracking).

Yes, you were, and you said so. It was (and is) a good idea. And I accept
that you are smart, if forgetful.

Since control was much _better_
with the same predictable disturbance in the pursuit condition it
was clear that two levels of control are involved in the poursuit
tracking situation. I was starting to develop a model of the behavior
in the pursuit situation. In a sense I built only half of the model;
the higher level system was simply a signal source; and this signal
was derived by sampling from the sine table that provided the
predictable disturbance. Obviously the real system could not be
setting its reference that way. I was starting to work on a real,
two level model of the situation but got distracted. Maybe I'll
start on it again. Thanks for reminding me, Martin!

Excellent!

I agree with everything you say. Especially that the provision of the
predictive reference signal in a real organism CANNOT come from an
externally provided model, as it did in your demonstration. So far,
you have shown that an exact predictor allows the simulation to fit the
human data better than does a simulation that omits the predictor. The
next stage is to show that a two (or more?) level simulation can adapt so
as to provide a predictive reference signal that works as well as does
the externally provided exact one.

Anyway, Martin goes on to say:

> I think the foregoing probably represents what happened better than
> Rick's vague memory does, or than mine did. Rick's model using the
> predictor of the sine wave (his words, at the time, though he now
> disclaims the idea) fit his own (human) data better than did a model
> without the predictor. And it was possible to compute the magnitude
> of the predictive lead time, at least to within the right order of
> magnitude.

Yes. But this was just a pseudo-model.

It wasn't a model that a simulated real system could have generated
of how the predictor waveform might have been produced, but it was
a model that _exactly_ replicated how the predictor was _in fact_
generated. I don't think that's very "pseudo." But this is just a
verbal niggle.

The "predictor" reference signal
was (as I noted) derived by sampling ahead in the sine wave disturbance
table. I just wanted to see what would happen if a higher level system
_could_ provide a reference to the lower level system that was an
estimate of the future position of the target. But the model that
produced that "predictive" reference was not built! I never got to that
point.

More's the pity. It would be wonderful to be able to show that a two-level
system without an explicit model (simulation) of the disturbance process
could do as well as the exact external model in fitting the human data.

The model would have to have a higher order system that controls
a temporal pattern of cursor movment (a fixed reference to this system
would specify a sine wave perception of a particular phase, frequency
and amplitude, for example) by varying the reference to the lower level
system that controls cursor position. This is the model that was never
buiilt.

One possibility is what you have proposed before--that the derivative of
the perceptual signal of the first level be used as the perceptual signal
of the second level. In the case of the sine-wave disturbance, this
derivative will be also a sine wave, with some phase shift. The amount
of phase shift depends, of course, on the phase lag with which the
output is added to the disturbance (the output of a linear system
being itself a sine-wave if the disturbance is one).

In fact, I used your suggestion (and credited it to you in print) in
modelling (simulating) the sleep study data, because using the predictor
gave a better fit to the human data than I got with any of the single
level model variants I tried. And I found that the placebo group seemed
to rely more on the predictor as the sleep loss progressed, whereas the
two drug groups didn't (or if they did, it was only very slightly).

> Interesting how words based on "predict" are prominent in your 1994
> messages in explaining what you did, but are wrong in describing it
> when you talk about it in 1997.

The position of a sinusoidally moving target is predicatable; the
position of a randomly moving target is not.

"Randomly" is a word you have to be very careful with. A "randomly"
varying disturbance with a bandwidth of 1 Hz is almost exactly predictable
one microsecond after it has been measured, but is no more predictable
0.5 seconds after a measurement than if no measurement had been made at all.
The predictability declines over time since the measurement, but is not
uniformly zero. The situation is not "either-or."

A model that tracks a
sine wave target could use prediction (calculate the expected
position of the target at time t+dt based on it's position at
time t) but the PCT model I was going to build would involve no
prediction. It would control a "transition" or "temporal
configuration" perception: -- the perception of sinusoidal motion
of the cursor -- by varying the reference to a lower level system that
is controlling cursor position.

Yep, I would think so. But using the simple derivative (a transition,
but not a temporal configuration, if I understand correctly) works
quite well. More complex predictors (higher-level perceptual input
functions) might work better, especially for disturbance variations
of specific patterns. Experiment would tell.

Martin

[From Bill Powers (960304.1000 MST)]

Martin Taylor 960304 10:45 --

It depends on the situation. In some cases it ["merging"] is the
process that goes on when one gains precision by observing
soemthing a little longer, in some cases it is the process that
allows one to use:

> ... information from
>several different independent inputs, each giving a different view
of the external situation.

Which is _not_ "a different problem," as you claim it to be.;,

I thought the problem we were talking about was whether switching
between real-time and model-based control of the same variable was
necessary. My argument is that if real-time perceptions are available,
control will be better for using them instead of a model. So I proposed
a small modification of Hans' model in which the basic updating of the
model works exactly as in his system, but where the model loop is a PCT
loop instead of an open-loop calculation. With this arrangement, the
perceptual input to the comparator can come from either the real-time
perception or the model's output perceptual signal. The model then
serves as backup for the normal mode of control, but is not used unless
the perceptual signal is unavailable. This is the sort of thing we do
when we type in some preliminary commands when the computer screen
hasn't warmed up yet so we can't see what we're typing, or when the
mainframe response is so slow that we can type in two or three commands
before there's any response to the first one. If we've made a mistake,
the real-time perception immediately takes precedence when it becomes
available.

As to "merging," this is a sufficiently elastic term that it would fit a
number of contradictory models. Safe, but not very useful.

Anyway, the model's output is _not_ an independent measure of the same
external situation. The only information in either signal comes from
the real perceptual signal.

That is true, but the real perceptual signal in question may have
occured minutes, hours, days, or years in the past.

I don't see the advantage in using a model output that is minutes to
years out of date rather than the perceptual signal that is available
now. Nor do I see how you could "merge" the two so as to use both at the
same time for comparison with the reference signal.

The model's output is not a measure of the current external
situation at all. It doesn't pretend to be. It's a measure of what
the current external situation may be if the external world behaves
the way it used to do, given what the observed external situation
was a short while ago. It's only useful insofar as current
observations are an imprecise measure of the current external
situation, or when the effects of current output will only be
reflected in perception after some time delay in the external
world.

Remember, we were talking about whether using the present-time
perception or the modeled perception is an either-or choice. You
proposed that they could be "merged" so no choice was needed. I don't
know of any "merging" operation that would actually work; some time back
(actually, many years ago if you count my initial efforts to work out a
way to use the imagination connection) I tried to work out a way that
would allow continuous use of real and imagined information at the same
time, and failed. The concept of switching the signals wasn't my first
try; it was a last resort. If you read Hans closely, you will see that
he, too, gave up on using both sources of perceptual signals at the same
time; he proposes that model-based control is the permanent mode of
operation. Hans and I are working under the constraint that we want at
least to have a working model in mind, if not an actual program. You are
working, apparently, under looser constraints.

... it is hard, if not impossible, to tell the difference between
the behaviour of a complex model-based system and a hierarchic one.
My personal bias is toward the simple hierarchy at the lower levels
and toward models at higher, especially symbolic/logical levels.
But I've yet to hear of any evidence that it is possible to tell
the difference without actually going in and dissecting the control
system's physical/physiological structure. Indeed, you recently
told someone (Shannon, was it?) that there was no way to tell the
difference.

I was talking about hierarchies, not the difference between model-based
and PCT control. It's easy to tell the difference between the latter
two: you just cut off the real-time perceptual input and watch what
happens (but see below). I agree that model-based control is more likely
to enter at higher levels.

The only serious problem (model-wise) arises when the real perception
disappears without warning. There is no system that can handle this
problem, unless it's a model-based system that always works off the
model instead of the real perception, as Hans Blom proposes.

Well, if this is true, it contradicts what I just said. But I don't
think it is true. I have two problems with it. Firstly, I'm always
sceptical when someone says "There is no X that Y except this one."

I said that the only system that can handle a sudden loss of input
signal without a glitch is one that is always in the imagination mode.
That means that I can imagine no other system that can do this.

All perceiving systems everywhere combine information from various
times and places, and I see no reason why one of those sources
should not be a model of the perception to be expected in some
situation.

I don't either, but that's the problem we're talking about. A signal
that is a model of the perception to be expected is precisely the output
of a model. A real-time perception is a function of the external world,
and is not spontaneously generated. The problem we are talking about is
what happens when the system is controlling the real-time perception
and, without warning, the perception is lost. The only choices are

(a) control is lost, with outputs behaving as if a large disturbance had
occurred as the perceptual signal drops to zero.

(b) outputs continue roughly in their former pattern, at least for some
short time, because the perceptual signal is being internally generated.

So far, nobody has suggested a way in which the control process could
rely BOTH on the real-time perception (to gain the advantages of this
mode of control) AND on a modeled perception of the same variable (to
gain the advantage of avoiding a large transient in the behavior). To
say that the two sources are "merged" is only to assert that some way
must be possible, without suggesting any way that would actually work.
If you suggest that there is a way, it's incumbent on you to demonstrate
that there is.

I think that in a simple hierarchy the reference signals from
higher levels even after a "loss of input accident" will still have
the appropriate lead times that have all along been reducing the
"ad-hoc" error corrections of the lower levels--think of Rick
Marken's demonstration of two-level fit with simple sine-wave
tracking.

If anything, the lead in Rick's model will make the loss of control
quicker, because it is based on the first derivative of the input
signal.

···

--------------------------------
There is one way in which you could see a perceptual signal as a model
without actually having any model in the imagination connection. That is
where the perception is derived as a time-function of several input
variables. A simple example: the perceptual signal is the integral of a
summed set of lower-level perceptual signals. If the lower-level
perceptual signals are all suddenly lost, the value of the integral will
not drop to zero; it will simply cease to change. If the perception was
nearly matching the reference signal before the loss, it will continue
to nearly match the reference signal after the loss, and the output
based on it will continue at its former level, since the error signal
will remain the same.

It would be hard to distinguish this phenomenon from model-based
control, even though the mechanism is entirely different. In general,
the higher-level perceptions are of a kind that can neither appear nor
disappear instantly. The perception itself may filtered in a complex way
so it changes smoothly even though the events on which it is based are
intermittent or vary more rapidly. You may notice that a bartender
frequently gives you the wrong change, but it may take many experiences
of this before your perception of his honesty begins to take a downward
trend. If the bartender suddenly reforms and starts giving the right
change, you will continue to behave as if he is dishonest for some time,
until the added input information begins to raise your perception of his
honesty again. This does not require any internal model in the
imagination position; it's just the way a sluggish perceptual function
works. Also, notice that it's hard to distinguish a slow-acting
perceptual function from an adaptive perceptual function.

The main problem in this picture is the problem that the output of
a simple scalar perceptual function cannot distinguish between a
loss of input and a zero-level signal.

This is true, but it depends on an unmentioned assumption. When we've
spoken of a sudden loss of input, we've been tacitly assuming a low-
level perceptual function in which the perceptual signal follows from
the input information with only a negligible delay. But suppose that
there is an edge-enhanced intensity perception followed by a level that
does a leaky perceptual integration to produce the final sensation
signal. Now a sudden loss of input will result in a slow change in the
final signal, as the leaky integrator's output decays. The overall
effect, if the integration just compensates for the edge-enhancement, is
a perceptual signal that follows changes in the input quite rapidly, but
which essentially holds its last value after the input information is
lost. Again, this could give the appearance of a low-order model-based
control system, even though no such arrangement is present.

The disadvantage of model-based control is that it commits you to
assuming a world with characteristics that change slowly.

Isn't that a good assumption much of the time? Isn't it also true
of the behaviour of a hierarchy?

In "characteristics" I was including disturbances, which one can't
assume will remain the same much of the time.

The "characteristics" in question represent the environmental
feedback function. However, if the disturbance influence has had a
coherent temporal pattern over time, that pattern cannot be
separated from the temporal characteristics of the environmental
feedback function unless it can be separately observed.

I get the feeling that this argument, in a subtle way, moves the
determinant of control behavior outside the organism. The character of
the environment that is being controlled is _defined_ by the perceptual
function. We do not begin with a variable to be controlled, and then
develop a control system to control it. That's what engineers do, but
not what organisms do.

The main adaptation that has to occur is in the details of the output
function. The perceptual system determines what it is about the
environment that is to be controlled. Once that is determined, the
output function must adapt so that error signals always produce outputs
that tend to reduce the error signal, converging toward zero error.
Since, in a feedback system, there is a wide range of output functions
that will create successful control, all that is needed is an output
function that will work, not an output function that is the exact
inverse of the environmental feedback function. In a system with a fixed
output function, control can continue even though there are both
additive and parametric disturbances in the feedback function. So it is
not necessary to distinguish what kind of disturbance is acting, except
when control is so poor as to disturb other parts of the whole system.
And even then, no explicit distinction is necessary; the parameters of
the output function simply change until control is restored.

The effects of either will show up in a model, and Artificial
Cerebellum, or an effectively phase-advancing higher-level control
system.

The A.C. and the phase-advancing higher-level control system are set up
so they require no explicit knowledge of the environmental feedback
function. The A.C. particularly needs no information about the
environment at all: its adaptation is based strictly on the error signal
entering it. It will adapt whether the cause of variations in the error
signal is a pattern of changes in the reference signal, or external
disturbances. In fact, if a regular pattern of disturbances is used, the
output function will adapt automatically to include a periodic term in
its transfer function! The artificial cerebellum is actually a very
powerful and general adaptive control method, but since it hasn't been
considered by Modern Control Theorists, it will get little attention.
Not Invented Here!

A while ago, I presented a counter-example to that from personal
experience. Playing table-tennis by the light of the full moon, I
found it necessary to hit the ball when visually the ball had only
reached the net. The delay in the visual system at low light is
that long. I had to use deliberate phase-advance (or model) to hit
the ball. Since my opponents took longer to realize what was
happening, I won a lot of games.

I was speaking, of course, of control under normal conditions. When
there is a lag in the perceptual system, as at very low light-levels,
you do need some phase advance to reduce the loss of control, and your
solution was ingenious. But you will notice that since you won a lot of
games, your human opponents seem not to have come across this conscious
strategy. We are talking about a learned strategy here, not a built-in
property of human systems. If this sort of adaptation were automatic and
universal, you would not have won any more games than usual.

If you had played in sunlight wearing dense sunglasses, and your
opponents had played without sunglasses, you would not have won a lot of
games. Compensations for deteriorated perceptual functions do not
restore control to normal. They simply prevent it from being as bad as
it might otherwise be.

If I seem to disagree with you, it's not on the question of
preferred interpretation. It's on the necessity of one
interpretation rather than another.

And if I seem to disagree with you, I'm only reminding you that more
possibilities than one exist, so it would be premature to settle on any
one of them as a final answer or a general principle.
-----------------------------------------------------------------------
Best,

Bill P.

[Martin Taylor 960304 15:15]

Bill Powers (960304.1000 MST)

And if I seem to disagree with you, I'm only reminding you that more
possibilities than one exist, so it would be premature to settle on any
one of them as a final answer or a general principle.

Funny. I thought that was _my_ line. Glad you agree!

But we are beginning to talk at cross-purposes again. You say, for example:

And even then, no explicit distinction is necessary; the parameters of
the output function simply change until control is restored.

The effects of either will show up in a model, and Artificial
Cerebellum, or an effectively phase-advancing higher-level control
system.

The A.C. and the phase-advancing higher-level control system are set up
so they require no explicit knowledge of the environmental feedback
function.

The word "explicit" is a construct that you have added to my concept. It was
"explicitly" omitted from what I was talking about. I'm not clear why you
are so insistent that a model need "explicit" knowledge of what it is
modelling. Some do, some don't.

The A.C. particularly needs no information about the
environment at all: its adaptation is based strictly on the error signal
entering it. It will adapt whether the cause of variations in the error
signal is a pattern of changes in the reference signal, or external
disturbances. In fact, if a regular pattern of disturbances is used, the
output function will adapt automatically to include a periodic term in
its transfer function!

And that was a point I was making. Unless the _whatever-it-is_ can observe
the disturbance separately, there's no way that regularities in the
environmental feedback function can be distinguished from regularities
in the disturbance influence (I initially mistyped "disturbiance" there,
which sounds to me like a sueful word, which I will refrain from using:-)

Now back to the main point, which tends to get lost quite easily. Perhaps
that's because (a) it shifts during the discussion, and (b) it's different
for each player. To me, the main point is that always perceptual signals
are based on the interplay among many different inputs--sources of data, and
that obviates the need for transient-generating switching among input
sources.

Anyway, the model's output is _not_ an independent measure of the same
external situation. The only information in either signal comes from
the real perceptual signal.

That is true, but the real perceptual signal in question may have
occured minutes, hours, days, or years in the past.

I don't see the advantage in using a model output that is minutes to
years out of date rather than the perceptual signal that is available
now. Nor do I see how you could "merge" the two so as to use both at the
same time for comparison with the reference signal.

There is no difference in principle if one of those sources of data is
the output of a model that has been built up over the years and is working
on the data received over the last few milliseconds. You seem to confound
the time-scale over which the model is built with the time-scale of the
data on which it is working.

I thought the problem we were talking about was whether switching
between real-time and model-based control of the same variable was
necessary. My argument is that if real-time perceptions are available,
control will be better for using them instead of a model.

And mine was that a transient will be induced by the switch, regardless
of whether there was a transient in the data source. If a suitable method
for combining the model data with the real-time data could be found, the
transient would be reduced, especially when the real-time data loss was
slow (as in the examples you give of higher-level processes). Combining
real-time with model data will normally give more precise results than
either, if the two sources could plausibly represent the same thing. (Not
that they _are_ the same thing). If they couldn't, then one has to be
discarded--and at a high level, we see a lot of discarding of the real-time
data. One has only to look at the belief people have in economic models
as opposed to what happens in the economy when governments try to balance
budgets by cutting spending:-(

As to "merging," this is a sufficiently elastic term that it would fit a
number of contradictory models. Safe, but not very useful.

True. If it happens at all, I would not presume that it happens in the same
way at each level. But it is not the case that I have no model for any such
combinations.

Remember, we were talking about whether using the present-time
perception or the modeled perception is an either-or choice. You
proposed that they could be "merged" so no choice was needed. I don't
know of any "merging" operation that would actually work;

Back in 1962 I proposed a generic model for figural after-effects that used
just this kind of combination of data sources. In a figural after-effect
experiment, a person is asked to view some environmental pattern that does
not change over some longish period of time, and thereafter to view a
pattern that is the same except for some specific deviation from the original
"anchor" pattern. The typical result is that the "test" pattern is observed
to deviate further from the "anchor" pattern than its measured deviation
would suggest.

I say "view" but the same thing happens in most, if not all, sensory
domains. In the paper in which I proposed the model (Canadian J Psychol, 16,
1962/4, 247-277), I used the model to fit data from a variety of experiments
reported by other people and myself: location in space of a point sound source,
location in space of a single marked point, the tilt of a near-vertical line
after viewing a tilted line, the tilt of a line after observation of
a grid, and deviation of the width of a block felt between the fingers.

The model used a critical assumption: It is assumed that the observer will
use information from all available sources about an event, giving
information from each source a weight inversely proportional to the
variability of the source.

Using two other assumptions supported by data from quite different studies,
it is possible to generate a 2-parameter equation to model the data as a
function of measured deviation between anchor and test patterns. In all
the studies except one, the two parameters had the same value, and the
fits were better, the more precise the data. In the other study (the
kinaesthetic width of a block), one of the two parameters had to be
increased by a factor of nearly two, but the other stayed the same.
That was quite encouraging.

After doing this, I came across another study in which the parameter was
not the deviation between the anchor and test; the anchor-test deviation
was held constant but the contrast of the visual display was varied. Data
were available in handbooks, from which the change in the variability of
observation as a function of contrast could be determined. I fitted the
FAE data as closely as the data quality permitted _using no fitting
parameters at all_. All the parameters of the model for that study were
obtained from studies by other people in other situations--variability
as a function of contrast, and the constant values of the two fitting
parameters for visual and auditory figural after-effects of different
kinds (Psych Review, 1963, 70, 357-360).

For a control system to use "variability" explicitly requires that there be
a perceptual system in which "variability" level is the output. But the
figural after-effect model seems to suggest that the perceiving system
in some way uses the variability itself. I can speculate as to how that
might happen, but it would only be speculation.

(And the speculation is:
there is a fast stage and an integrative stage in the perceiving function,
the first feeding into the second; the output of each stage is separately
available and the absolute (rectified, squared?) difference between them
integrated to provide a second "reliability" output of the perceptual
function. Pure speculation, and it might not work if modelled. I mention
it only to argue that "merging" is not a totally vacuous concept in at
least one situation. It's at least well enough specified that it could be
modelled as a mechanism. The FAE model suggests that the perceiving system
acts at least as if something like this is operating.)

some time back
(actually, many years ago if you count my initial efforts to work out a
way to use the imagination connection) I tried to work out a way that
would allow continuous use of real and imagined information at the same
time, and failed. The concept of switching the signals wasn't my first
try; it was a last resort. If you read Hans closely, you will see that
he, too, gave up on using both sources of perceptual signals at the same
time; he proposes that model-based control is the permanent mode of
operation.

You have the advantage of me, in that the kind of interaction implied by
my figural after-effect model has not been tried in a control system. But
your following comment is not fair:

Hans and I are working under the constraint that we want at
least to have a working model in mind, if not an actual program. You are
working, apparently, under looser constraints.

I'd love to have a working model. Failure to find one is not evidence that
one cannot be found, though the longer the search, the less likely it is
that the model, when found, will be plausibly simple.

Indeed, you recently
told someone (Shannon, was it?) that there was no way to tell the
difference.

I was talking about hierarchies, not the difference between model-based
and PCT control.

Wasn't whoever it was talking about a non-hierarchic (i.e. complex, model-
based) system? If it was Shannon, she was talking about one that uses a
neural network learning system as a component of at least the perceptual
function. That's a particular case of model-based perception.

It's an interesting special case of model-based perception, because the
neural network that she proposes for the perceptual function could very
well be exactly the normal HPCT perceptual input function connections of
all the layers below the one being considered. When one notes that identity,
it becomes clear that when you look at only one control loop in the standard
HPCT hierarchy, at least above the third level, it _does_ have a neural
network input function and a neural net output function, and can be seen
as being a model-based system. But this is true only of each ECU seen in
isolation.

It's an interesting observation, nevertheless.

···

------------------

I get the feeling that this argument, in a subtle way, moves the
determinant of control behavior outside the organism. The character of
the environment that is being controlled is _defined_ by the perceptual
function. We do not begin with a variable to be controlled, and then
develop a control system to control it. That's what engineers do, but
not what organisms do.

Time-scales again!

An organism can _define_ any CEV whatever so long as the relevant sensory
inputs are available. But if there is no easily discovered way for the output
to affect that CEV in a consistent way, the perceptual function is likely
to get reorganized so that it comes to define a different CEV, for which
some useful output function can be found. So in that sense the environment
provides a set of possible perceptual functions that admit of good control,
without determining what perceptions will be controlled.

The environment doesn't determine what perceptions _will be_ controlled,
but it limits the range of what perceptions _can be_ controlled. Over
evolutionary time, some part of that range is restricted further, more
so in some organisms than in others. And over a life-time the range is
restricted to the perceptions that we _do_ control. However, during
reorganization, perceptions can be in a transient state, _defining_ any
CEV that is possible to the sensor systems. But CEVs that impose impossible
or difficult requirements on output functions will not long survive as
the definitions of existing perceptual functions; the functions will change.

The main adaptation that has to occur is in the details of the output
function. The perceptual system determines what it is about the
environment that is to be controlled. Once that is determined,...

How, other than by the discovery that such control is both possible and
advantageous for controlling the intrinsic variables?

...the
output function must adapt so that error signals always produce outputs
that tend to reduce the error signal, converging toward zero error.

Fine, but what if they don't (and because of the particular environment,
if they cannot)?

Since, in a feedback system, there is a wide range of output functions
that will create successful control,

For many possible perceptions, that's so. For some, there may be only one
way within the constraints of available muscle and bone, and for some there
may be no output function that is easily found by e-coli or gradient-search
methods, or even none at all.

all that is needed is an output
function that will work, not an output function that is the exact
inverse of the environmental feedback function.

That is true, and one should add that the environmental feedback function
may not have an inverse in the mathematical sense. All that reorganization
will give you is a system that produces a tolerably good control of the
intrinsic variables through control of a variety of perceptions.

------------------

I'm not clear why we are arguing, and the more it goes on, the less clear
I am. So much of it seems to hinge on what you ended with and I both start
and end with--you think that I commit to something I suggest as an alternative
possibility, and I guess I see the same thing in your writings. You look
more committed to what you write than you actually are. So...

And if I seem to disagree with you, I'm only reminding you that more
possibilities than one exist, so it would be premature to settle on any
one of them as a final answer or a general principle.

Something on which we both agree:-)

Martin

[Hans Blom, 960305]
(Bill Powers (960228.0600 MST))

I've been trying to think about model-based control in the context
of PCT, and the difference between symbolic and analog computation.
I'm quite sure that model-based control does happen, and that in
general outline it happens more or less as Hans Blom and the "modern
control theorists" propose. I doubt rather seriously, however, that
it is implemented in a brain in the same way Hans implements it in a
computer.

and I agree. My basic argument was that a full-fledged Extended
Kalman Filter requires full connectivity from every memory element to
every other one, whereas we don't find this in our nervous system.

We need to know how long a person or animal can go without sensory
feedback before losing control.

The problem of no perception at all arises only in a one-dimensional
model / control system. More generally, only some _part_ of the high
dimensionality of the perceptions will be missing, and as you know
from neural networks, classification (or modelling, or control) will
work quite nicely if some dimensions are missing. This is one example
of where a one-dimensional analysis is very misleading.

... Hans' model was superior in one regard to the PCT model in that
it adapted itself to the situation while the PCT model was simply
given fixed parameters. This adaptation was somewhat limited, in
that when feedback was interrupted, Han's model lost control within
a fraction of a second, essentially just as fast as the PCT model
did.

Due to a very bad signal to noise ratio, as you remember. With very
little noise/disturbance, control may remain quite adequate for long
periods of time.

But I would like to put that consideration aside for now, because
there are other simpler methods of adaptation that would also work,
particularly if the overall model has to imitate only the imperfect
control that a real person shows rather than reaching some ideal
degree of control.

Quite true. The engineering literature contains several different
schemes that do away with the full computations of conditional
probabilities. Their performance is always less than when probability
calculations are done, but sometimes not much less.

In particular, I am struck by the practical difficulties involved in
having both a forward function and an inverse function, with the
parameters from one being transferred for use in the other. Much has
been made of the perfectness of this method, since in principle the
controlled variable can be made to match the reference exactly,
whereas the PCT model must always contain some error to make it act.
But now we see that this perfection depends critically on being able
to compute one function that is the exact inverse of another.

This is not correct. If the system to be controlled has dynamics,
even in a one-dimensional system an exact inverse won't be possible
when te reference level changes abruptly. No exactness is possible if
the output is somehow limited. In multi-dimensional systems it
becomes clearer that not exactness, but a best projection upon the
dimensions that can be controlled is what is required. More simply:
take a step of the optimal step size in the optimal direction. And in
a control context, not too large deviations from true optimality
hardly matter; the next instant will provide a correction.

I'm not trying to shoot down the whole concept of adaptive control
here; only the unrealistic method used to implement it, and the
spurious claim that this method is inherently perfectable. The most
serious defect of this implementation is the duplication of
functions: the necessity of doing the computations of the model
twice, once forward and once in reverse. This duplication is
unnecessary.

There is a duplication because two things go on at the same time:
model adaptation (keeping our knowledge of the world up to date,
whether we use that knowledge at that time or not), and control. This
is a feature, not a bug. Isn't it true at at many times we're just
passively storing our impressions of the world without being able to
use them _at that time_? But that doesn't make those impressions
worthless; they can be used for control at a later time, when they
_are_ useful.

Greetings,

Hans

···

================================================================
Eindhoven University of Technology Eindhoven, the Netherlands
Dept. of Electrical Engineering Medical Engineering Group
email: j.a.blom@ele.tue.nl

Great man achieves harmony by maintaining differences; small man
achieves harmony by maintaining the commonality. Confucius

------------------- ASCII.ASC follows --------------------
[Hans Blom, 950621]

(Bill Powers (950613.1115 MDT))

Bill, thanks for your long and thoughtful reply. I am, as always,
to which you might be able to put it. Yet some problems remain.

You're perfectly right in saying that " the outputs of your
comparators do something useful even when everything has become
stabilized." In a negative feedback control system there is
_always_ error unless for the moment all disturbances happen to
be exactly zero, and no output is required to hold the
controlled variable at its reference level.

But I am more interested in the analysis of the stable state. As
a thought experiment, set all disturbances to zero and check to
what state the system as a whole settles down to. If even when
everything has settled down, non-zero "errors" are required to
keep the system in its stable state, it is difficult for me to
think of these "errors" as errors -- they can also be interpreted
as control variables that keep other variables at their correct
consider to be error.

This is the fact about negative feedback that W. Ross Ashby got
hung up on. He saw that if the disturbance could be sensed, then
in principle his regulator could compensate _exactly_ for the
effect of the disturbance on the critical variable. Because a
negative feedback control system always had to have some error
to produce a compensating output, it could never even in
principle control exactly. Ashby decided that the only "real"
control system was the compensator.

Isn't he right IN PRINCIPLE? In your scheme of things, a negative
feedback control system always has some error (1 in 100 in your
example) to produce a compensating output, even if the error is
small. If a compensator can avoid even this small error, wouldn't
it IN PRINCIPLE be better, particularly in those cases where we
want to have a control quality that exhausts all information in
the feedback signal down to the noise level?

... The object now [in modelling a human arm] is not to achieve
perfect control and zero error, but to design a system that
shows the same amount of error as the real system [the human
arm]. This means adjusting parameters such as output amplifi-
cation until the model's behavior is as close to the real arm's
behavior as possible -- including imperfections of control.

I agree with the former sentence, but not with the latter. The
problem is that we do not yet have the correct model for how the
human arm operates -- although some think they do -- and there-
fore we might have to play around with different control para-
digms to find out how they might compare. Your PID-like control-
ler with its "output amplification" might give the best match,
but so might a compensator-like controller, in which case trying
to tune an "output amplification" is the wrong approach. This
problem is far from settled, I think (and as the current litera-
ture on arm models shows), so we won't know until we try out all
the possible competing approaches.

The same holds for the models we use in tracking experiments.
The model can be adjusted to track far better than the person
does.

In some circumstances, but not in others. A long standing example
of this is the approximately zero response times that subjects
soon start to demonstrate when tracking a repeating (e.g. square
wave) pattern. I have yet to see a PCT-model that can do this
well enough, compared to a human subject. To me, this example
indicates that we ought to find out how this type of prediction

We adjust the model's parameters to make its simulated handle
movements match those of the real person as closely as possible;
then we use the resulting parameter values as measures of the
person's control properties.

That is exactly the problem. If you presuppose a PID-like organi-
zation in the human arm when the real organization is different,
you can adjust parameters until doomsday without finding a good
enough match. This problem is the same as the problem of "unmod-
elled dynamics" or system noise in Kalman Filter based control-
lers, where you DO recognize the problem. Later in your mail you
do recognize the problem, though:

If you decide that you're going to build a compensator, the
constraints are very different from what they are if you decide
you're going to build a negative feedback control system. Even
the parameters of the system will be different and require

An aside:

The amplifier in the higher system [in your diagram] assures
that only a small velocity error is sufficient to change the
reference force over its entire possible range. So the velocity
error is never allowed to get very large.

A small error in your diagram equals a small "nerve current". Due
to the discrete nature of "nerve currents" (quantized as action
potentials), one would expect that small currents result in
rather noisy control signals when amplified. Given your example
of an error of 1 in 100, the error signal would be around 3
action potentials per second, assuming an upper rate of 300 a.p.
per second. This most likely would give Parkinson-like oscillat-
ions. Does this correspond to the reality of a human arm?

This [reasonable independence of elementary control systems] can
be accomplished if the perceptual signal in each system is
computed from the set of lower-level input signals through a
function orthogonal, or reasonably orthogonal, to the input
functions of all other control systems at the same level which
are active at the same time.

This might also be a solution to the size problem of the covari-
ance matrix of the Extended Kalman Filter: it could be split up
in (almost) independent sections, resulting in far fewer varia-
bles to be computed per iteration.

How do you think the orthogonalization comes about? Is it gene-
tically given or learned? Could it be related to the early death
of so many brain cells in the months immediately before and after
birth?

The result is that the perceived world is partitioned into
approximately orthogonal dimensions, with independent control in
each dimension.

The Extended Kalman Filter approach could do more or less the
same, but not necessarily with independent control in EACH dimen-
sion; dimensions that cannot be orthogonalized could remain
together in a vector-type control unit.

The Test for the controlled variable, as you suspect, can be
done in a simple way only when the reference level is constant.

Yes, in essence it is the scientific method: keep as many of the
independent variables (other reference levels AND other observat-
ions) as you can constant, and observe the remaining relation(s)
between independent and dependent variables. Have there been
discussions about the importance of keeping perceptions limited?
I doubt that a subject who is happily tracking away and hears the
exclamation "fire!" coming from somewhere will control his joy-
stick well from then on. Even "coffee!" would do it for me ;-).

If there are unpredictable variations in the reference level, no
variable will prove to be perfectly stabilized by the person's
actions. However, if we could roughly identify a number of such
controlled variables, we might be able to deduce the nature of a
higher-level system that is using them to control some higher-
level variable.

Watch out. Who sets the reference level? The subject, don't we
agree on that? Doesn't that imply that there are no unpredictable
variations in the reference level?

Moreover, it is difficult to learn if you are also controlling
well. In fact, if you are controlling perfectly, you cannot learn
at all (says theory) -- there is also no reason to learn (says
practice). The larger the mismatch between world and world-model,
the more rapidly learning takes place -- if learning CAN take
place, i.e. presupposed a model with a sufficient number of
degrees of freedom.

If you want to learn (say about higher-level systems), you might
want to consider doing this is a non-control context. Maybe a
stimulus-response type of context?

In the hierarchical model, I think the whole system comes to a
state where error is more or less evenly distributed among all
the active systems. This seems to be about what you are saying.

Depends. According to the optimal control point of view, the
system stabilizes in a state where the optimality criterium
(weighted sum of squares of errors) is minimized. Differences
with your assumptions are are: The errors are weighted; some low-
level errors contribute more to overall-error than others. When
there would be no noise in the system, all errors would be zero
if there are no conflicts. If there are conflicts, conflicting
goals are realized equally well (measured in terms of the error
criterium).

I think it's rather futile to speak of "optimal" or "best"
control -- the whole system simply controls all the variables of
concern to it as well as it can.

"As well as it can" equals "optimal", isn't it? The problem is,
however, that you cannot measure "good", "better", "best" if you
do not have a yardstick. Hence the ubiquitous use of an optimali-
ty criterium in modern control theory. According to a control
theory professor whom I recently met, he just discovered the
second control problem that could not be reformulated as an opti-
two exceptions (I'm very curious!), but this demonstrates that in
practically every case a control problem can be viewed as an
optimalization problem. In effect, doesn't The Test try to dis-
cover what is optimal in a given controller?

... If the brain were born knowing everything it might be called
upon to control, perhaps it could do somewhat better.

Could do WHAT better? Which goals would be realized better, and
at the expense of which other goals? There are many more bodily
limitations, besides our brains: due to a lack of actuators, it
is generally only possible to fully realize one (conscious) goal
at the same time. More than one goal at a time, and you're bound
to find yourself in conflict (another Zen lesson: realize your
goals in succession, not simultaneously). This suggests that at
the highest level there can only be one goal.

Well, when I want to turn on the light over the sink and turn on
the disposal instead, I think of that as an "error" (blunder)
and not as optimal behavior in any sense. I just hit the wrong
switch. Of course I tell Mary that if she knew the whole story
she would see that this is really optimal behavior, not old
age....

Glad to see you understand me

Part of the environment is the other people in it. When _they_
are reorganizing, it becomes pretty difficult to predict how
that part of the environment should be modeled.

Unless you have a good model of HOW people reorganize, based on
extensive observation and experience. Some therapists seem to be
able to do just that. So it isn't impossible, just pretty diffi-
cult -- until you've built up an accurate model. Unless of course
you think that people reorganize in a random way. But that does
not fit in with my experience.

Note that the term "unmodelled dynamics" implies that there are
things "out there" that we cannot, will not, or do not model and
that we therefore assume to be "random". Not that part of what
happens in the world IS random (it may or may not be, we just
don't know), but that our knowledge is limited. A more extensive
model may (partially) extract knowledge (parameter values) from
that "random". In science, in particular, we should not assume
that the world is random, but that where we perceive randomness
we run into our own wall of ignorance.

... If we really had a deterministic algorithm that could
define the optimal control system -- including methods of
control we haven't even thought of yet -- the whole game would
be over, wouldn't it?

Yes and no. The problem is not that we cannot design algorithmic
methods to search for the optimal controller; that problem is
akin to the mechanical methods that exist to generate all theo-
rems of first order logic or all possible successions in a chess
game. The problem is that WE have to specify what is optimal:
what is the goal (or what are the goals and what are their rela-
tive importances); what are the constraints. It is all too easy
to forget some goal or constraint. Given a specification, a map-
ping to an optimal system can in principle be found (by clever
heuristics or dumb exhaustive search), and in practice if we
specify which (countable number of) methods and how many para-
meters may be used in the construction of such a system.

... the world-model is no longer changing x, although it may
still be changing its internal parameters a,b, and c, and thus
still changing the parameters a, b, and c involved in generating
u. Is that what you mean? The parameters are still changing and
influencing the outer control loop, I guess.

Yes, that is what I mean. Even if perception is perfect, it still
takes some time to build up the model after a change of the para-
meters of the world. Think of it in this way. In a completely
noise-free system where the model structure matches the structure
of the world, the model has 4 free parameters (x, a, b, c). Now
it would take 4 observations in order to be able to solve 4 equa-
tions with 4 unknowns. With system noise <> 0, 4 observations
(i.e. 4 equations) are still required to get a first estimate of
the 4 free parameters. But subsequent observations, i.e. more
than 4 equations, can be used to generate a "best fit" and thus
average out the noise, thus improving the estimates, decreasing
their variances.

What I found was that with an external disturbance of xt, the
values of the world-model parameters became very different from
those of the external system: _a_ became about 2*at, and _b_

With an external disturbance of xt, the model converges to para-
meter values that accomodate the disturbance as well as possible.
So the estimated values of a, b and c will greatly depend upon
the amplitude and the type of the disturbance, but also on the
particular sequence in which the noise samples were generated.
You will find other values with other disturbances.

I use open-loop systems all the time, too. My "contempt" for
open-loop systems is specialized: it occurs when I see people
trying to explain behavior with an open-loop system under
conditions where that is a grossly wrong model.

I can understand that. A compensator with incorrect parameters
cannot control well, if at all. That is why I advocate ADAPTATION
of the parameters of the compensator. It does seem to be a real
problem in humans, I agree, that their models frequently reach
convergence before such is warranted. Or, stated differently,
that they are not willing to revise their models when they en-
counter "outliers". Freud called this "denial".

I think that as far as any ordinary organism is concerned,
transmission of genes to the next generation is strictly an
accident, not a goal. It's a side-effect of controlling other
variables that the organism CAN sense.

What for one person is the main thing, may be a side-effect for
another. If the two are perfectly correlated, neither The Test
nor any other method may be able to distinguish what is what.
Either we need more knowledge, or we need a religion, a set of
unprovable axioms. Ultimately the latter, I suppose.

not mine. The quality of a closed-loop negative feedback system
is measured by its ability to resist unpredictable disturbances,
not by the way it fails under the slightest change in its
environment.

I thought I was talking about control in general. The "ability to
resist unpredictable disturbances" can probably be expressed as a
minimization of a function such as

t = T

···

-
\ (x (t) - xopt (t))^2
/
-
t = 0

or some such (its square root, say). In that case, a rapid react-
ion of x after a change of xopt will contribute significantly to
minimization of such an error criterium. I do not think that we

The reason your world-model behaves so perfectly is that the way
you calculate u automatically takes into account any variations
in the parameters; in fact the effects of the parameters cancel
out. If the world-model were any more complex, so you couldn't
find the value of u just by solving an algebraic equation, you'd
have to have an automatic inverse-finder as well as the system
for adjusting the forward parameters in the model.

I have no idea why you keep coming back to an "inverse-finder"
time after time. My demo did not incorporate one, and neither
would a more complex model necessarily need one.

Let me explain to you when an "inverse-finder" WOULD be required.
That would be in the situation where xopt (t) remains unspecified
in the formula above, because an optimal TRAJECTORY is not re-
quired, but only an optimal END POINT. So the error criterium
would become something like

x (T) - xopt (T)

where xopt (T) might not only be a (vector) position but maybe
also a (vector) velocity and/or acceleration. Because xopt is not
specified for t < T, the situation is completely different: in
effect, an xopt (t) must ALSO be calculated for all t, 0 < t < T.
Because of the extra required calculation of the trajectory of
xopt (t), a solution for a "best" xopt (t) over the time from t=0
to t=T must in effect be computed. This is of course not so when
xopt (t) is PRESCRIBED from t = 0 to t = T. In that case, no
"inverse" needs to be computed.

I hope that from now on this confusion won't crop up anymore.

Where is dt, the length of physical time represented by one
iteration of the program? If the model truly had mass, then
changing the definition of dt would also require changing the
definition of at.

The time dt can be chosen at will, but must be small enough. As
usual. Taking dt twice its original value would change the
formula

x (t+1) = c + a * x (t) + b * u (t) + f (t+1)

using

x (t+2) = c + a * x (t+1) + b * u (t+1) + f (t+2)

into

x (t+2) = c + a * [c+a*x(t)+b*u(t)+f(t+1)] + b*u (t+1) + f(t+2)
= c + a * c +
a * a * x (t) +
a * b * u (t) + b * u (t+1) +
a * f (t+1) + f (t+2)

which, if we choose u (t+1) = u (t) and introduce a composite
noise g (t+2) = a * f (t+1) + f (t+2), equals

x (t+2) = [c+a*c] + [a*a] * x (t) + [a*b+b] * u (t) + g (t+2)

where you see that the original parameters a, b and c now have
different values. Does that settle your question?

When u suddenly changes, how long does it take xt to change
accordingly? One iteration, as you have designed your program.

After one iteration, xt would be changed slightly. It takes many
more iterations before xt settles to a new stable value after a
step change of u. Simplify the model equation by setting f(t)=0,
c=0 and b=1. Set a = 0.9:

x (t+1) = 0.9 * x (t) + u (t)

Assume that u has been zero for some time, and (therefore) x as
well. Now set u=1 henceforth and see how x changes:

x (t+1) = 0.9 * x (t) + 1

x (0) = 0
x (1) = 1
x (2) = 1.9
x (3) = 2.7
...
x (infinity) = 10

No physical system could respond perfectly in a single
iteration.

Certainly. But that is not what I modelled.

This surprising finding [world-model <> world] needs to be
explained, doesn't it? When there is a disturbance, the world-
model constants come to values different from those of the real
system (I have observe this, too).

Basically there are two causes: 1) incorrect initial assumptions
and 2) learning in the face of noise is a stochastic process
whose outcome is guaranteed to be correct only ON AVERAGE. If you
choose initial assumptions with very large variances (pxx, paa,
etc) you eliminate the first cause, but at the expense of a
longer time to convergence. If you run a large number of trials
with different noise sequences, you will find that the converged
model-parameters have some probability distribution around the
world-parameter value. The width of the distribution covaries
with the noise power, all other things being equal.

Noise is a real problem in daily life as well, in humans as well
as in machines. Let me tell you an anecdote from my initial
trials with an adaptive blood pressure control system, in which
the effect of a hypotensive drug (sodium nitroprusside, SNP) on a
particular patient was to be estimated by correlating the change
in the blood pressure with the infusion flow administered to the
patient, in a way similar to what happens in my demo. This esti-
mate was required because the patient's sensitivity to the drug
varies so much (a factor of 80), that a non-adaptive controller
cannot perform well enough.

Now sodium nitroprusside is a drug that lowers the blood pres-
sure. Yet, when the controller started to infuse SNP, the blood
pressure went rapidly and significantly UP. So the model decided
that the b-parameter was positive, i.e. that SNP _raises_ the
blood pressure rather than lowering it. So the controller de-
creased the SNP flow rate. Then the pressure went down. But too
far, below the setpoint. So more SNP was infused. The blood now
pressure went up again. By now, the model was pretty certain that
SNP is a drug that RAISES the blood pressure.

Unknown to the model was the fact, that at the time when the
infusion started, the surgeon made his first cut; and also, that
go up was the pain due to the surgeon's cut, which had a much
more powerful effect than the (still small) SNP flow. After the
cut was over, the blood pressure dropped, simultaneously with the
decrease in SNP flow rate.

Well, you get the picture. A sudden noise burst at the wrong
moment may be explained as significant. And when it repeats, as
very significant. This leads to superstition. There are solutions
to the problem of building up incorrect knowledge, of course, but
these require more sensors, a more complex world-model, and/or
built-in limits on how far the values of the parameters can vary.
We chose a combination of the latter two approaches.

Greetings,

Hans

<[Bill Leach 950622.22:25 U.S. Eastern Time Zone]

[Hans Blom, 950621]

Isn't he right IN PRINCIPLE? In your scheme of things, a negative
feedback control system always has some error (1 in 100 in your ...

Absolutely not! I am more than astounded by your "fixation" on "model
based control".

Using a model _is_ a good idea with a WELL DEFINED process that has
decidedly limited disturbances of an unpredictable variety AND has
unreliable perceptual input.

In a practical world these requirements are errr... ahhh... uhmmm...
"difficult" to achieve.

Then there is the minor problem with the question of what sort of system
compensates for disturbance to the CEV "down to the noise level"? Is it
not a closed loop negative feedback control system?

... problem is far from settled, I think (and as the current literature
on arm models shows), so we won't know until we try out all the
possible competing approaches.

Ok, you try the other approaches... we'll work on the PCT version.
Also, your "and as the current literature ..." sounds a lot like what
Bill P. was talking about in one posting to Bruce Abbott.

Stating that "all the authorities ..." is not in itself a valid argument.

In some circumstances, but not in others. A long standing example
of this is the approximately zero response times that subjects
soon start to demonstrate when tracking a repeating (e.g. square
wave) pattern. I have yet to see a PCT-model that can do this
well enough, compared to a human subject. To me, this example
indicates that we ought to find out how this type of prediction

Do you have experimental data in support of this one? As I understand
the situation, such has been "claimed" for many years but actual
experiments have demonstrated the claim to be false.

That is exactly the problem. If you presuppose a PID-like organi-
zation in the human arm when the real organization is different,
you can adjust parameters until doomsday without finding a good
enough match. This problem is the same as the problem of "unmod-
elled dynamics" or system noise in Kalman Filter based control-
lers, where you DO recognize the problem. Later in your mail you
do recognize the problem, though:

The problem with your argument is that "tuning the parameters" of a PCT
model DOES give a good match!

Bill P.

If there are unpredictable variations in the reference level, no
variable will prove to be perfectly stabilized by the person's
actions. However, if we could roughly identify a number of such
controlled variables, we might be able to deduce the nature of a
higher-level system that is using them to control some higher-
level variable.

Watch out. Who sets the reference level? The subject, don't we
agree on that? Doesn't that imply that there are no unpredictable
variations in the reference level?

No, it does not imply that there are no unpredictable variations in
reference level(s). In the first place, as far as the control loop
receiving the reference input... ALL changes are unpredictable.
Additionally, it is not too difficult to postulate a situation where
as a part of a complex control function, a particular perception is
compared to a reference and the output is a reference for a lower
level system. In this case, either a change in the higher level
system's reference or a change in its' perception will change the
reference for the lower level system.

Moreover, it is difficult to learn if you are also controlling
well. In fact, if you are controlling perfectly, you cannot learn
at all (says theory) -- there is also no reason to learn (says
practice). The larger the mismatch between world and world-model,
the more rapidly learning takes place -- if learning CAN take
place, i.e. presupposed a model with a sufficient number of
degrees of freedom.

This strikes me as a rather arbitraty claim. Without some specific
definition applied to the term learning the statement is absolutely
meaningless.

If you want to learn (say about higher-level systems), you might
want to consider doing this is a non-control context. Maybe a
stimulus-response type of context?

Then again, you might not want to... you might want to see if such
could be a control process before throwing in the towel.

Depends. According to the optimal control point of view, the
system stabilizes in a state where the optimality criterium
(weighted sum of squares of errors) is minimized. Differences
with your assumptions are are: The errors are weighted; some low-
level errors contribute more to overall-error than others. When
there would be no noise in the system, all errors would be zero
if there are no conflicts. If there are conflicts, conflicting
goals are realized equally well (measured in terms of the error
criterium).

Your claim here seems to be inconsistent with at least my understanding
of HPCT. Not all errors are equal in HPCT.

In effect, doesn't The Test try to discover what is optimal in a given
controller?

I would suggest that the answer to that question is NO! It is pretty
well considered to be an accomplishment to just identify a CEV much
less determine what the organism considers optimal control to require.

Bill P.

... If the brain were born knowing everything it might be called
upon to control, perhaps it could do somewhat better.

Could do WHAT better? Which goals would be realized better, and
at the expense of which other goals?

There are many more bodily limitations, besides our brains: due to a
lack of actuators, it is generally only possible to fully realize one
(conscious) goal at the same time.

I don't know that I accept that this is a significant statement. How
"generally"? When might multiple goals be achieved at the same time?
Is this even significant if true?

More than one goal at a time, and you're bound to find yourself in
conflict (another Zen lesson: realize your goals in succession, not
simultaneously). This suggests that at the highest level there can only
be one goal.

Suggests it to you maybe, I don't know that I "buy it". I agree that in
general (there is that word again), are "actuators" are satifying a
single goal at a time (if you look in the "right" place at the "right"
level within the hieararchy). It is also probably necessary to be quite
specific about what you consider to constitute a goal to get everything
down to just "one at a time".

Again, I think that such a statement as "at the highest level there can
only be one goal." is mostly "smoke".

Bill P.

Part of the environment is the other people in it. When _they_
are reorganizing, it becomes pretty difficult to predict how
that part of the environment should be modeled.

Unless you have a good model of HOW people reorganize, based on
extensive observation and experience. Some therapists seem to be
able to do just that. So it isn't impossible, just pretty diffi-
cult -- until you've built up an accurate model. Unless of course
you think that people reorganize in a random way. But that does
not fit in with my experience.

And maybe since this is an exchange about one of my own earlier remarks
on this whole matter, it is this particular set of comments that prompted
my response to this posting.

An essential belief in PCT is that reorganization not only is a random
function but it MUST be such!

We run "smack up against" the problem that the early AI researchers
recognized and had originally hoped that AI would solve: How do you
design a system (control system) that can deal with unexpected problems
of a completely unanticipated nature?

The only answer that anyone has yet been able to come up with is that you
design a system that will try AT RANDOM techniques, possibly completely
unknown processes, in an attempt to solve the problem. The early AI
people (at least some of them) realized that random solution attempts
might not work and certainly would not be considered "effecient" or
"elegant" but for totally unknown and unexpected problems such at least
might work.

"Experience" might "guide" the process but when all known methods and
even all attempts at combining "known methods" in previously unknown
configurations fail, then the only chance for success is random attempts.

BTW, your "experience" with reorganization is not consistent with that of
a great many others.

that "random". In science, in particular, we should not assume
that the world is random, but that where we perceive randomness
we run into our own wall of ignorance.

While much "to do" has been made concerning someone's recent assertion
that radioactive decay was not a random function, I have not heard of any
electron sub-shell decays (when not forced by maser or laser
synchronization).

Yes and no. The problem is not that we cannot design algorithmic
methods to search for the optimal controller; that problem is
akin to the mechanical methods that exist to generate all theo- ...

And I believe that you here take a significant departure from the beliefs
of most (if not all) of the rest of on this listserver. There are some
very convincing arguments about why "determinism" is not absolute. While
I still "like" Einstein's "God doesn't place dice with the Universe", the
evidence is still again him.

-bill

[From Bill Powers (950906.1515 MDT)]

Martin Taylor (950906.1330) --

My question was, of course, intended to be rhetorical, since, if I
remember correctly, Rick HAS done this and reported the result--but
not, of course turning off all the perceptual inputs.

I was suggesting in my heavy-handed way that you actually run Rick's
model and turn off the inputs and see what happens.

> You are asserting here that the model must function without
input data.
>
>I have read and re-read my paragraph just above and I can't see
where I made any such assertion.

Well, it looks pretty explicit to me. I guess it's another case of
how often this happens. In this specific case, why would the model
need to extrapolate the disturbance into the future if it could use
real input data?

But I am explictly assuming that there is no indication to the control
system of the state of the disturbing variable. All that is sensed is
the current state of the CEV, which is the sum of one function of the
output and (in general) another function of the disturbing variable. the
forms of these functions are not deducible from the perceptual signal.
The only information available to the control system is whatever is in
the perceptual signal representing the CEV.

You (and others) have claimed for a long time that arbitrary

independent

disturbances are actually predictable,

That's a nonsensical statement that I have never made. What I have
said at various times relates to equivalent bandwidths and the
decay of precision over time since observations have been made. An
arbitrary disturbance has infinite bandwidth and hence zero
prediction time. A real disturbance has finite bandwidth and
becomes less predictable over time. A structured disturbance has
finite bandwidth and is likely to deviate increasingly from a
projected trace over measureable time. There's a huge difference
between these facts and the nonsense that you attribute to me.

I am talking about the facts as you presented them. You claim, probably
correctly, that a real disturbance has a finite bandwidth and is thus
predictable to some degree over some period of time. But you appear to
have claimed that this prediction can, over that period of time, be used
to predict the disturbance and improve control. I am simply challenging
this claim. I do not believe there is any method of predicting the
disturbance (the disturbing variable) that will produce better control
than we already get or even an equal degree of control, specifically in
the tracking model. Stick to the point. We're talking about Hans'
proposed model-based control, where we have an actual model to work
with. I claim that there is no way to deduce future states of an
arbitrary independent disturbance (of finite bandwidth if you like) that
will allow the model-based control system to equal the performance of
the ordinary feedback control system -- as long as the perceptual input
is intact. All you have to do to disprove my claim is to come up with a
single working model that works better than the PCT model, or even as
well.

I'm not talking about proving a generalization with one example. I'm
offering a generalization which can be DISPROVEN with a single example.
It should be easy to show that I'm wrong. You have an entire universe of
possible control methods to work from, except for the straight negative
feedback control system, which is mine.

These are the conditions I would assume in the theorem I would like
to see. I would impose the same conditions on the experimenter, as
well. Only the disturbances and the control system outputs (or the
state of the CEV) would be observable to the experimenter.

The experimenter is out of the loop; we're talking about a working model
that has to work by itself based on information it can get from its own
inputs. So whatever model is offered, it only gets to perceive the state
of its own CEV. The experimenter can know everything prior to turning
the model on. But given that the disturbance is defined only as a
smoothing function of a random-number generator which will never repeat
itself during the setup and experimentation, I don't see what good it
will do the experimenter to know what the disturbances are. No
particular pattern is ever going to appear again. The disturbance might
be a constant for three days, then a sine wave for six minutes, a square
wave for two seconds, and a random variable for 11 hours. Or it might
switch among these forms under control of a second random variable, at
times under control of a third. What the experimenter knows is not going
to help the control system cope with this disturbance. But a properly
designed negative feedback control system can do it easily -- that
pattern, or any other, right from the beginning.

Heck, I'll even let you hand-adapt the world-model-based control system
before we start the experiment, making it as perfect as possible. The
only thing you don't get to know is the form of the disturbances that
are going to occur -- if any occur.

If you don't mean that the model-based system must have no ongoing
input from the state of the CEV, what DO you mean here?

I mean that the model-based system gets no information about the CAUSE
of the disturbance, or the function relating that cause to the CEV.
Nobody gets that information; it is generated anew by a routine based on
one or more random number generators, so nobody, experimenter or model,
can know what the disturbance is going to look like or even when it will
start or what form it will take. The only guarantees are that its
amplitude will not overtax the output of the control system and its
bandwidth will have a reasonable limit. This will cause no problems for
the designer of a PCT model. It will cause all kinds of problems for the
designer of a model-based system. Hans has made statements that sound to
me like saying that NO control system can work under these conditions
(totally unmodelled dynamics). I'm willing to meet that challenge, if it
was meant.

In the "information about the disturbance" discussion, it has
ALWAYS been a given that the control system can observe ONLY the
state of the CEV. All else comes from those observations. An
ANALYST may look at the disturbances, if necessary, but the control
system cannot.

You have claimed, or I have understood you to claim, that the analyst
could deduce how much information there is in the perceptual signal that
is about the cause of the disturbance, and could show how this
information is "used up" to produce the actions that counteract the
effects of the disturbance. In our joint attempts actually to do such
calculations, programmed by me under instruction by you, with the
program checked by you, we failed to come up with any calculation that
led to understandable results. None of the results conformed to your
predictions. It does not seem possible, at this time, that perception of
the CEV contains any usable Information about the cause of the
disturbance, knowable either by the control system or by the analyst.

As Rene Levesque said: "A la prochaine."

Well, wash his mouth out with soap. I don't like to hear kids talking
like that.

···

-----------------------------------------------------------------------
Best,

Bill P.

[Martin Taylor 950906 19:15]

Bill Powers (950906.1515 MDT)

Aha!!!

Your message gave me an "Aha" experience. I hope mine will do as much for
you, so I put it up front.

Stick to the point. We're talking about Hans'
proposed model-based control, where we have an actual model to work
with.

You have been talking about Hans' particular model. I wasn't. No wonder
we have been talking past one another. I'm sorry for the confusion, for
which I have no doubt been in large part responsible.

I have not been in the slightest interested in finding a model-based
system that controls better than a non-model based system. What I have
been interested in is the notion that models, as explicit representations
of the outer environment, might be necessary or even useful parts of
control systems. It's that "explicit representation" that concerns me--
the notion that there might be a separate module, independently observable,
called "the model." Hans' version has a particular configuration of model,
but it's only one of many possible ones.

I posed a conjecture, which I neither believe nore disbelieve, that for
every perceptual control system that works through a hierarchy without
explicit models, there is a behavourally equivalent model-based control
system, and vice-versa. In the back of my mind, though not a necessary
part of the conjecture, is the idea that the equivalent "BEMCS" (a.k.a.
bug-eyed monster control system) might well be a single-level, but not
scalar, one. In other words, that it might be the case that one could
of linkages among complex control systems.

If the conjecture turns out to be true, other possibilities follow. In
particular, it follows that if there exists in the hierarchy a complex
control system (and I have in mind the possibility that the "program
level" might be such), then learning from a teacher might lead to
a kind of "directed reorganization" in which the "learned" perceptual
control is re-embodied in a simple hierarchic substructure.

If the conjecture is false, then the enquiry leads in quite different
directions, because it becomes possible (in principle) to determine
whether a particular case of control is based on the operations of
explicit models or on a possibly complex hierarchy.

···

----------------------

Given the above, I don't know whether it is of any value to comment on the
rest of your posting, but perhaps I might note some points of agreement.

But I am explictly assuming that there is no indication to the control
system of the state of the disturbing variable. All that is sensed is
the current state of the CEV, which is the sum of one function of the
output and (in general) another function of the disturbing variable.

Yes.

the forms of these functions are not deducible from the perceptual signal.

That's a kind of assumption, but it's an extra one that I think to be
not required, and perhaps not true, given:

The only information available to the control system is whatever is in
the perceptual signal representing the CEV.

which is agreed, and basic.

I claim that there is no way to deduce future states of an
arbitrary independent disturbance (of finite bandwidth if you like) that
will allow the model-based control system to equal the performance of
the ordinary feedback control system -- as long as the perceptual input
is intact.

That's agreed, so long as the words "arbitrary independent" modify the
word "disturbance." If you change them to "statistically structured"
then the situation may be different.

I would impose the same conditions on the experimenter, as
well. Only the disturbances and the control system outputs (or the
state of the CEV) would be observable to the experimenter.

The experimenter is out of the loop; we're talking about a working model
that has to work by itself based on information it can get from its own
inputs.

Well, that comment shows a deep misunderstanding. My "experimenter" is
trying to determine whether the control system that is acting on its own
is model-based or not. The experimenter is certainly out of the loop of
the control system being examined. All I'm saying is that the experimenter
can't look inside the control system, but can look at what the control
system can't see--the disturbing waveform (or even the cause, if that's
of any help, which it won't be).

So whatever model is offered, it only gets to perceive the state of its
own CEV.

Yep.

But given that the disturbance is defined only as a
smoothing function of a random-number generator which will never repeat
itself during the setup and experimentation, I don't see what good it
will do the experimenter to know what the disturbances are.

My experimenter might even select particular disturbances, so it's not
guaranteed that they will never repeat. The experimenter might choose
to make it repeat 50 times and then change, to see whether the control
system had adapted in any way; that might be a clue as to whether it
contined an adapting model (but then again, it might only signal some
kind of reorganization in a hierarchy without explicit models).

What the experimenter knows is not going
to help the control system cope with this disturbance.

Agreed. A necessary aspect of the experiment.

If you don't mean that the model-based system must have no ongoing
input from the state of the CEV, what DO you mean here?

I mean that the model-based system gets no information about the CAUSE
of the disturbance, or the function relating that cause to the CEV.

We stipulate that.

Hans has made statements that sound to
me like saying that NO control system can work under these conditions
(totally unmodelled dynamics). I'm willing to meet that challenge, if it
was meant.

Well, I'm with you on this one. I haven't understood why Hans says this.
At first I didn't believe that's what he meant, but he's said it often
enough that I do believe that you interpret him correctly. I HOPE we are
both wrong.

----------------------

You have claimed, or I have understood you to claim, that the analyst
could deduce how much information there is in the perceptual signal that
is about the cause of the disturbance, and could show how this
information is "used up" to produce the actions that counteract the
effects of the disturbance.

No. Not about the cause. Never. About the waveform, what we agreed
in face-to-face conversation to call the "disturbance effect."

In our joint attempts actually to do such
calculations, programmed by me under instruction by you, with the
program checked by you, we failed to come up with any calculation that
led to understandable results. None of the results conformed to your
predictions.

You are a little premature in this statement, aren't you? Some time ago
you sent me a program to check for accuracy in computing the mutual
information between two signals. I looked at it briefly, and agreed
that the single-point individual signal information calculations seemed
to be correct, but said that I thought there was a problem with the mutual
information calculation. I'm ashamed to say that it then got put aside
and I haven't looked at it again. But even so, it had nothing in it that
would suggest a computation of prediction of future statistics from past
states (a distribution of x(t+tau) given values of x(t), x(t-1), x(t-2)...).
That would be a minimum requirement.

As far as I am aware, the only prediction I have made is that the better
the control, the lower the correlation (or mutual information) between
disturbance and perceptual signal. While seeking in my dark and musty
when some control loop was in the imagination mode, I came across the
following, right at the start of the "information about the disturbance"
debate (Martin Taylor 930313 17:40):

The essential point about any stable system is that the uncertainty
observed at any pointin it will be stable over time. The closed
loop ensures this by opposing uncertainties that might be introduced
by the disturbance.

Overall, what this means is that numerically there will be no information
supplied by the disturbance to the preceptual signal. This may seem odd,
or even magical, but it has to be so. ...
The uncertainties may
on average reflect the uncertainties in the disturbance, in that control
is imperfect, but that is a question of the dynamics of the system
as a whole.

So I started with a statement that sounds awfully like your present:

It does not seem possible, at this time, that perception of
the CEV contains any usable Information about the cause of the
disturbance, knowable either by the control system or by the analyst.

I really DON'T want to restart the "information in perception" discussion
again. That's not because I don't think it's valuable, but because there's
plenty of good stuff going on in CSG-L without getting into yet another
fruitless round of arguments. If I ever come up with anything from it that
I think will give YOU a different insight into PCT (as opposed to giving
ME a different insight) then I'll post it. For now, linear algebra is
just fine with me, as are demonstration simulations and real-life applications
of PCT. In no way is the information-theory approach expected to change
any of these ways of looking at or using PCT.

As Rene Levesque said: "A la prochaine."

Well, wash his mouth out with soap. I don't like to hear kids talking
like that.

Neither do I. But in fact, "la prochaine" turns out to be October 30,
for Quebec. Let us hope that les separatistes are no more successful than
the last time.

Gotta go home. Hope you also can experience "Aha!"

Martin