[From Bill Powers (980109.0859 MST)]
Martin Taylor 980109 0:10--
To my knowledge, Hans never suggested the absence of feedback. What he
did say was that models could be used to predict (a) the effect of output
(as does Bill Powers' Artificial Cerebellum), and (b) the future state
of the CEV if the disturbance varied in a predictable manner. I never
understood how this got transmuted into a claim that Hans denied the
use of feedback. He did say that predictability could be useful in
improving control.
The artificial cerebellum does not predict what the effect of the output
will be. It sums past error signals according to a weighting function to
produce a _present_ output action. The effect of this output action on the
environment that is relevant to control is determined, as usual, by the
form of the perceptual input function. The criterion for adjusting the
output function is derived from the present and past values of the error
signal. There is no estimate of future values of the error signal or of
effects of the output.
Prediction and control are essentially antithetical ideas. A prediction
requires assessing past and present states of variables, and from them
computing what their future state will be. Control, on the other hand,
requires selecting a future state that is desired, and then acting
continuously to keep the present state converging toward the desired state.
When control is occurring the outcome is predetermined, not predicted.
While the predetermination is not absolute, in the sense of predestination,
it is far more certain than any prediction, because a wide range of
unforeseeable circumstances that would invalidate any prediction is
automatically counteracted by changes in the action that occur as the final
state is being approached.
Prediction is basically an open-loop idea. It is the basis for most of the
conventional concepts about how behavior works. From the properties of the
organism and environment, the command signals needed to produce a
particular predicted outcome are calculated -- this is where all the
inverse dynamics and kinematics come in -- and then they are emitted.
Predict, plan, and execute. This is exactly the model that _has_ to develop
when you try to explain behavior without the idea of negative feedback
control. It's the main current idea that stands in the way of acceptance of
PCT.
This is why I'm so strongly against forcing interpretations so they include
the idea of prediction. I don't like seeing people say that something is a
"kind of" prediction, or that "you can see it as" prediction, or that
there's "implicit prediction" or "effective prediction" going on. All this
does is give credence to the predict-plan-execute model, and make PCT all
the harder to grasp. People are already too willing to take any new idea
and try to force it to fit what they already believe -- why make that
easier to do?
Where literal prediction does occur, we can find a place for it in the
model. But we have to keep the main idea in mind: that a prediction is a
perception, and that the _main_ organizational principle is the control of
perception.
In real predictive control systems such as airplane-landing systems, what
is controlled is the outcome of the prediction, not the present state of
the world. The pilot controls the predicted touchdown point. As time to
touchdown approaches, the length of time ahead involved in the prediction
gets shorter and shorter, so inaccuracies make less and less difference.
From the pilot's standpoint, however, the control task is always the same:
keep the crosshairs showing the predicted touchdown point on a particular
spot near the start of the runway. The actual control task is concerned
strictly with present-time perceptions.
In Hans's model, the world-model is not actually a predictive model: its
output is not a _future_ state of the world, but a simulation of its
_present_ state. It's always a bit behind, because the Kalman Filter
process has a finite bandwidth, and the estimated parameters are always
those appropriate to some period of time just past. The simulation of the
disturbance is not a simulation of its future state, but of its present
state, and it always lags the actual disturbance by at least one iteration
of the digital approximation. What is calculated is the _present_ part of
the simulated perception that is due to the disturbance, and this is the
basis for the next alteration of the output. In fact, the whole treatment
of the disturbance in Hans's computer model boiled down to a simple
present-time negative feedback loop, with a gain parameter that was
adjusted by the Kalman Filter process (or by Hans). I think this is
necessarily the case when one tries to handle _unpredictable_ disturbances.
The only feedback in Hans's model was in the Kalman Filter process. The
perceived state of the plant's output was compared with the output of the
world-model's simulation of the plant. The result of this comparison was an
error signal that was converted to a slow alteration of the parameters of
the simulation. The speed of the alteration was determined strictly by how
long one wanted the system to work after the input was cut off: the longer
this period of no-feedback operation was to be, the more averaging time was
needed to calculate the parameters, to avoid confusing the effects of noise
and unpredicted disturbances with actual changes in the external plant's
parameters. And of course the slower the corrections, the less able the
system becomes to handle short-term disturbances. I was never able to get
Hans to admit that this tradeoff exists, although in some of the papers he
directed us to, others in this field considered it a major problem and
proposed elaborate methods for partially overcoming it. In fact, Hans
admitted that in our last comparison of models, the PCT model behaved
almost identically to the Kalman-filter model, the reason being that he had
shortened the parameter-adjustment time to a single iteration. As a result,
his model failed just as immediately as the PCT model did when the input
was cut off. _All_ the supposed advantages of the model-based control
system then disappeared.
You say that prediction can improve control. In the sense that adding some
first derivative to the perceptual signal can put damping into a system, I
agree. But in general, prediction involves computations that take into
account the state of the world for some considerable time before the
present, so its bandwidth is limited: predictive control is used in
circumstances where there is plenty of time to make adjustments. The more
data from the past are taken into account, the more accurate the prediction
becomes (up to the point where chaos sets in). But at the same time, the
ability to compensate for unforeseen changes declines, because predictions
necessarily involve averages over time. There is no way that a predictive
control system can compete in speed with systems that can use present-time
information and react to it immediately.
My view of predictive control as a model of human behvior is that it will
be found in the higher, slower, levels of control. And it will always
suffer from the disadvantages of assuming that the future is simply an
extension of the past.
Finally, I should point out that Hans' model is only one way of handling
adaptive control. In a hierarchical model, we can have higher systems
monitoring the performance of lower systems in dimensions that are "meta"
to those involved in the controlled perceptions. For example, a higher
system could look at lower error signals and see whether they are
oscillatory, increasing, or simply unacceptably large. And a higher system
could adjust the parameters of lower systems to control these
metaperceptions. A higher system could look at perceptions from a large
array of lower systems, and adjust the gain in these lower systems to
maintain some average level of perceptual signal.
The main thing is that such higher systems would be concerned with internal
properties of the control processes themselves, and could act to optimize
control without any knowledge of the environment. The artificial cerebellum
is this type of control system; it simply tries to keep the error signal as
small as possible, and never knows what properties the environmental
feedback function has. Tom Bourbon's application of the E. coli
reorganization method to optimizing control is another example. Another is
the second-level system in the "reversal" experiments Rick and I did. There
is a recognizeable relation between action and error, or even a specific
behavior of the error signal alone, that could be controlled by a higher
system. In this ase, the higher system would act by inverting the sign of
the lower-level control system's output function, to effect recovery when
the external sign of feedback reverses.
None of these methods of adaptation requires that the brain contain a
literal simulation of the environmental feedback function. None of them
requires prediction. I suspect that still other methods could work and
would be worth looking into -- if people could only shake free of the one
simple conception that is the basis for Hans's model.
Best,
Bill P.