[From Bill Powers (950507.1900 MDT)]
Bruce Abbott (950507.1925 EST) --
I think Hans' model should be evaluated on its OWN merits, not on
those YOU'VE imposed. The model was designed to function within an
environment in which the disturbance waveform is fairly regular
(i.e., a function that can be fairly well described by the system's
internal world model equation, given the right parameters)
Whatever Rick said, I did evaluate Hans' model in its own terms first,
pointing out what it is good at doing. But you are mistaken in saying
that Hans' model can "function within an environment in which the
disturbance waveform is fairly regular." It cannot handle any
disturbance waveform at all; as Hans explained, "unmodeled dynamics"
can't be handled with this kind of model, whether they are regular or
not. The disturbance I used was a sine wave of constant frequency. In
Hans' model this disturbance appears unchanged in the real system's
output, and never becomes any smaller no matter how long you let the
adaptive model run.
The reason is quite simple: a disturbance of the real system does not
disturb the model. Since the model handles disturbing effects only in
terms of their amplitude and variance, it has no way of deducing the
disturbance and computing a compensating reaction to it.
In a more sophisticated model, I suppose that the model-correcting
circuitry could be made to search for regular variations in the
perceived output of the real system, and compute some compensations for
them. This, however, would work only for regular disturbances, and would
not work for irregular or arbitrary ones.
There is, potentially, a way of combining a closed-loop model (which
opposes disturbances without needing to know anything about them) with a
model-driven system of the kind Hans has shown us. However, I am at an
impasse in trying to achieve the combination.
One thing we might want is a model that uses a perception of the real
controlled variable when such a perception is available, in a simple
closed-loop control system without any world- model. A world-model could
be continously updated while this control is going on, because the model
is modified on the basis of the difference between its computed output
and a perception of the output of the real system. But the output of the
world-model would not be used unless the real-time information were
lost.
When the real-world connection is broken, we then want the model to
switch to using the internal world-model as a pseudo-perceptual signal,
exactly as in Hans' model. We would then have the best of both worlds:
the ability to resist arbitrary disturbances in a simple way when
perceptions of the controlled variable are available, and the ability to
continue controlling (in the absence of disturbances) when perceptual
contact with the external world is lost. I think we can observe BOTH
phenomena in behavior, so a complete model should be able to handle them
both. The standard PCT model can't handle control when there is a loss
of real-time perceptual information; the adaptive control model of Hans
can't handle control in the presence of arbitrary disturbances.
However, this "switch" is awkward -- it requires a higher-level system
to detect the loss of the real-time information, and switch the lower
system from using the real-time perception to using the output of the
world-model as the perception. It would be lovely if we could get the
basic control system to do its own switching without any auxiliary
system to help it.
I have considered the following possibility. Call the output of the
model xm, and the output of the true system xt. Forget the perceptual
noise for now, as it would affect both modes of operation.
Suppose that the perceptual signal is composed of the model's output,
xm, plus the difference between the model's output and the system's true
output, xt - xm. The value of (xt - xm) is already computed in Hans'
model and is the basis for modifying the world-model. Hans has explained
that the Kalman Filter approach guarantees that (xt - xm) will approach
zero, if the model is structurally correct and there are no "unmodeled
dynamics."
This means that the perceptual signal is
p = xm + (xt - xm).
Suppose that the model is adapted to work in the absence of any
disturbances, and that its output exactly matches the output of the real
system. In that case xt - xm = 0, and the perceptual signal is simply
the model's output, xm.
If a disturbance occurs, it causes a deviation of xt from the model's
output; call this change in the real system's output delta. Delta
represents the difference between the real system's output with and
without a disturbance acting. As a result, the perceptual signal is
p = xm + (xt + delta - xm), which is
p = xm + delta,
just as though the disturbance had caused a change in the output of the
world-model! The adaptive model would now be able to oppose the effects
of any arbitrary disturbing waveform acting on the world-model's output,
using a high-gain control loop acting through the world-model. Since the
same output acts on the real system, the effect of the arbitrary
disturbance on the real system would also be opposed.
This is fine for the case when the model has reached a match with the
physical properties of the real system. Now what happens if the model is
initially set to zero, so it produces no output xm at all?
With xm = 0 we have
p = 0 + (xt + delta - 0), or
p = xt + delta.
Again, just what we wanted! Now the control system uses the _real_
system's output as the perceived and controlled variable, and using the
same high-gain control system, acts to oppose the effects of a
disturbance on the real system. So we now have two conditions we wanted:
the model-based control condition which nevertheless can correct the
real effects of arbitrary disturbances, and the model-less condition in
which control is exerted in the normal PCT way. There is no switching of
any kind, and no need for a detector to tell which condition exists.
What about the remaining case, where the model exists and has been
matched to the behavior of the real system, but the perception of the
real system's output is lost (xt = 0). If xt is zero, there can be no
effects of delta on the perceptual signal, so ...
ALAS, this gives us
p = xm + (0 - xm) or
p = 0.
If there is no perception of the real xt, there is no perceptual signal,
and if there is no perceptual signal, the reference signal variations
are passed through to the output directly without any influence from
either the world-model or the actual state of the real system's output.
The loss of all feedback information, real or imagined, means that
control is drastically changed, and actually lost.
This is the closest I have been able to come to a "beautiful" merger of
the world-model based control system and a normal PCT control system. It
is truly beautiful up to a point, and then it collapses. We would have
to introduce an ad-hoc higher level system to reduce the gain of the
output function when real-time perception is lost, even to approximate
the required behavior. It looks as if the initial idea, of having a
higher system detect the loss of real-time perception and switch the
perceptual input to receive the world-model's output, is still the
simplest one.
But I really hate to give up on the beautiful model.
···
----------------------------------------------------------------------
Best,
Bill P.