[From Bill Powers (950525.0530 MDT)]
Rick Marken (950524.2145) --
I think your assessment of Hans' model still has something wrong with
it. One problem is that when you set the reference signal to a constant
value, the Kalman filter has no basis for calculating partial
derivatives and can't adapt properly until it has has some experience
with the faster disturbance. The fourth screen shows the loss of control
quite clearly.
An easier way to see the difference between control and no control is to
set the reference signal variations to 0.01*sin(run/25). This makes them
very small without interfering (much) with adaptation. Now during the
blind part of the second screen, you see the disturbance suddenly appear
in the top plot -- it is clear that it is being reduced by at least a
factor of 10 by the control process when the real controlled variable is
being sensed.
···
-----------------------------------------------
The reason is not that the Kalman filter has suddenly developed a
mysterious and unplanned ability to detect the effects of a regular
disturbance and adapt to it. It is that the disturbance variations are
being passed through to the model's output in the manner I described
yesterday. In the "process_observation" routine, we find
t := (y - x) / (pxx + pvv);
x := x + pxx * t;
This reduces to
x := x + (y - x)*pxx/(pxx + pvv) or, after adaptation, about
x := x + 0.5*(y - x).
This isn't the whole story because the parameters are also being
affected in real time by the fluctuating value of (y - x), which, after
adaptation, is the same as the disturbance. Also, the computation of x
is affected by the fact that in another program step in the same
iteration, x is computed from
x = c + a*x + b*u + pax.
The net effect on x is not clear, since we have the same variable, x,
being computed in two different ways in the same iteration.
I believe the net result is that hidden inside this adaptive model with
a Kalman filter is an ordinary control system that works through the
external world in the usual way once adaptation is complete. The primary
difference from the simple control system is that if present-time inputs
are lost, the controller can continue to produce the same pattern of
outputs and thus continue to drive the external system in the
appropriate way, as long as there are no independent disturbances and
the external parameters remain the same. Under a loose definition of the
word "control", the system can be said to continue controlling without
present-time sensory feedback. It does not continue controlling under
the meaning of control that we use in PCT, where an essential part of
the definition is resistance to disturbances.
--------------------------------------
We can now see that the primary effect of Hans' model is to enable the
controlling system to continue the same pattern of output variations in
the absence of direct sensory feedback from the real controlled
variable. The same result can, I believe, be achieved by a hierarchical
control system.
However, before we put too much emphasis on this ability to operate
blind we should ask whether it has an important role in explaining human
behavior (which is, after all, the primary purpose of PCT). If this
ability to operate temporarily using an internal world model is an
essential part of every real human control process, then our models
should all be revised to include this ability. In hierarchical models,
every system at every level should include it. This would greatly
complicate our models and our experiments, but if model-based control is
required to explain what we observe in experiments, then we would have
no choice but to put a world-model into every explanatory model and to
adopt Hans' model in general.
There is a simple, if not permanent, way around this problem. After the
adaptation has taken place, the system with a world-model inside it
operates in a way that is exactly the same as the way a simple model
with no internal world-model would behave. Resistance to disturbances is
maintained as long as there is a sensory connection to the real
controlled variable, and it is lost when the sensory connection is lost.
Statically and dynamically, the adaptive model becomes experimentally
indistinguishable from the model without adaptation, except under
conditions where the sensory connection is lost.
The practical meaning of this fact is that we can continue to use the
simpler PCT model as it stands as long as we are investigating behaviors
in which there is a continuous sensory contact with the world. In
effect, this is what we have been doing. We have always recognized that
in order for a control system to come into being there must be some
prior process of reorganization (read adaptation). This is one reason we
always require experiments to continue until asymptotic behavior has
been reached; we have recognized that in a novel task, parameters of
control will change until learning is complete. We measure the
parameters only after they have stopped changing.
In cases where sensory contact is lost, we would not immediately turn to
a model-based control system representation. First, we would have to see
what happens to real human behavior when sensory contact is lost. If the
person simply stops, or begins to act in irregular ways that do not
maintain the previous pattern of outputs, then of course Hans' model
would be no more correct than the simple PCT model, and possibly it
would be less correct. If the person behaves as the simple model behaves
when the sensory input is cut off, then the PCT model would continue to
be preferred. But if the person behaves in a way that fits neither
model, we would have to go back to the drawing board and try to find a
model that would explain what we see.
We have a modicum of evidence that for simple control tasks, the PCT
model represents correctly what the real person does when there is a
sudden reversal of the external connection. The initial response to the
reversal is the same in the person and in the PCT model: a runway
condition in which all the variables begin to increase exponentially. We
have not tried simply cutting the connection, but that is easily done.
In our reversal experiments, we find that the runaway condition persists
for slightly less than 1/2 second, followed by an _internal_ reversal of
the control process, possibly in the output function, which immediately
restores control. We have modeled this as a higher-level system that is
(in one version) monitoring the relationship between handle movement and
target-cursor distance, and maintaining the relationship to satisfy the
requirements of negative feedback. This hypothetical higher system takes
about half a second to recognize the change in relationship and complete
the required adjustment in the lower-level system, a reversal of the
sign in its output function.
So clearly a hierarchical model can, in this case, accomplish the same
thing that Hans' model does, and in a considerably simpler way. Instead
of having to compute running probability matrices and statistical
predictions, the hierarchical model simply computes the sign of the
relationship between handle movements and tracking error and acts in a
simple manner to maintain it in one state.
Such a hierarchical system would still have to be created by some
reorganizing or adaptive process, but what needs to be created is far
simpler than it would be if the entire control process, including sign
reversal, had to be adaptively carried out in a single-level system.
-----------------------------------
It is one thing to design an adaptive control system; it is another to
show that it is a model of real human behavior. Engineers can design
control systems that operate faster and more precisely than human
systems can, and perhaps they can even design adaptive control systems
that can continue to operate under circumstances where real human
systems simply fail. What can be designed is not necessarily what exists
in the natural system.
There is a tendency in computer-science circles to design a system that
operates in some clever way, and then look around for behaviors that can
be interpreted (and often overinterpreted) as fitting it. This is the
wrong way around. We must start with observations of real human behavior
in circumstances where we can measure accurately what is going on, and
then search for a model, clever or simple, that will reproduce the
behavior under all reasonable variations in circumstances that we can
think of. If, suffering loss of sensory information, a human being does
not continue to produce the same pattern of outputs as before, then the
correct model must NOT continue to produce the same pattern of outputs
as before. The fact that a model exists which can continue the pattern
is completely irrelevant in such a case, even though such an ability
might be very useful to the organism. The fact is that the organism, in
that experiment, does not have that ability.
--------------------------------------
I hope we can keep in mind the main thrust of the PCT Project. The main
idea is to reinterpret behavior in terms of PCT, to see whether we
become better able to explain behavior. There are many side-alleys that
can be explored, many unusual behaviors occurring under rare or extreme
circumstances that we could pause to examine and puzzle over. But the
main question is and remains, and will remain for a long time, "What
difference does it make to see behavior as the control of perception?"
For example, what difference does it make to see the sensory signals
from a bee's footpads as inhibiting the mechanisms that cause the wings
to oscillate, and to see the same signals as standing for a controlled
variable? I think it makes a great deal of difference.
If we see the inhibitory footpad signals as controlled perceptions, we
can imagine a reference signal that is set to bring the footpad pressure
up to some value, and the means of doing so as changing the drive
signals to the flying mechanisms. If the footpad pressure is too low,
the resulting error signal reduces the flying efforts, and the flying
efforts remain smaller until the footpad pressure is brought to the
reference level, the inhibitory signals cancelling the reference signal
and correcting the error. So settling onto a surface is brought about by
specifying a non-zero value for a perception of footpad pressure. The
change in flying efforts is a means of increasing footpad pressure. And
of course by setting the reference-pressure to zero, the same system
will reduce the footpad pressure to zero by increasing the flying
efforts until the bee is airborne rather than footborne.
The difference is that in the first interpretation, causality is
assigned to the footpad pressure, an environmental stimulus, while in
the second case causality is assigned to whatever changes the reference
signal for footpad pressure. In the second case, footpad pressure
becomes a controlled effect rather than an initiating cause. And in the
second case, there can be other systems also contributing to the
operations of the wings, even when the error in the footpad-pressure
control system is zero. So the bee can happily stand on a surface and
buzz its wings to create a draft, without taking off. In that case,
incidentally, I presume that the phasing of pitch and roll angles of the
wings is changed so that no lift is generated.
Anyway, let's not get hung up on details here, and let's not forget that
models are only as good as the experiments which show they are needed.
-----------------------------------------------------------------------
Best,
Bill P.