[From Bill Powers (960228.0600 MST)]
I've been trying to think about model-based control in the context of
PCT, and the difference between symbolic and analog computation. I'm
quite sure that model-based control does happen, and that in general
outline it happens more or less as Hans Blom and the "modern control
theorists" propose. I doubt rather seriously, however, that it is
implemented in a brain in the same way Hans implements it in a computer.
I also doubt that it exists at the lower levels of control.
As a method of designing artificial computer-driven control systems to
operate "plants" of known nature, model-based control seems to be a
powerful method (although I don't know how well it works in practice --
is this how most contemporary control engineers now design their control
systems?). The question is whether it is also a general model of control
processes in living systems (from the bacterium to the human being).
We need to know how long a person or animal can go without sensory
feedback before losing control. If this turns out to be only a very
short time for a specific control task, then there are much simpler
arrangements than the one Hans proposes that could give the appearance
of continued control for a brief period after loss of sensory
information. If loss of sensory feedback in a particular task leads to
an immediate runaway effect, then clearly there is no model-based
control associated with that task. The basic factual question is, "Do
people act as if they are using model-based control, and if so, under
what circumstances?" The answer can be provided only by experimentation;
there is no way to answer the question on hypothetical grounds, or
through qualitative and essentially data-free anecdotes.
In a comparison of a very simple PCT model and Hans' very elaborate
model-based control model of a simple tracking experiment, Hans' model
controlled slightly better than the PCT model did. But both models
controlled two orders of magnitude better (in terms of remaining error)
than real people do in the same task: a tenth of a percent as opposed to
10 percent RMS error (at the level of difficulty used). Of course Han's
model was superior in one regard to the PCT model in that it adapted
itself to the situation while the PCT model was simply given fixed
parameters. This adaptation was somewhat limited, in that when feedback
was interrupted, Han's model lost control within a fraction of a second,
essentially just as fast as the PCT model did. But I would like to put
that consideration aside for now, because there are other simpler
methods of adaptation that would also work, particularly if the overall
model has to imitate only the imperfect control that a real person shows
rather than reaching some ideal degree of control.
The basic principle of control that Hans proposes is a method of
adjusting a model and of producing an output that affects both the real
world and the internal model. The internal model receives the output
signal, and responds to it by producing a perceptual signal. For
adaptation, this synthetic perceptual signal is compared with one
derived by sensing the real world, and the difference is used to modify
the model.
With adaptation complete, the model is used by the system in two ways.
First, it produces a synthetic perceptual signal for comparison against
a real one. This involves running the model _forward_. Second, the same
model is also used in its inverse form: its inverse is used to convert
the reference signal, open-loop, into an output signal. This is
something I have only recently realized. When I originally drew a
diagram of Hans' system, I drew an internal feedback loop, as if the
synthetic perceptual signal were being compared with the reference
signal, and the error was producing the output. Hans drew his diagram in
a similar way. But in fact, there is no internal feedback loop of that
kind. The synthetic perceptual signal is not compared with the reference
signal.
What is done instead is to let the adaptation procedure adjust the
parameters of the model; then the model's inverse, with those
parameters, is used open-loop to convert the reference signal into an
output signal. The internal "loop" is closed only through transferring
the parameters from the forward model to the inverse of the model.
I think this is a correct diagram of the organization of Hans' model,
where previous diagrams have been, at best, misleading. Notice that the
synthetic perceptual signal is NOT part of an internal loop that
includes the output path. It is used only in adjusting the parameters of
the model.
In a computer program, it is no problem to compute the forward form of
the model and also its inverse. The designer of the program knows the
general form of the real-world system (with parameters of the model
being left for the adaptation to adjust), and therefore can also compute
the inverse form (assuming that one exists). The parameters can be
stored in memory or a register, then recalled for use when evaluating
the output of the inverse function.
In one sense, a nervous system can do the same thing. In Hans' computer
program, we have the evidence that a nervous system (Hans') can set up
symbolic computations, work the math with pencil and paper to compute
the forward and inverse models, and write a computer program to
implement the computations. However, one is permitted to wonder whether
the nervous system has any _other_ way to do the same thing, without
going through the step of conversion to symbols and the execution of
procedures that one has to go to school to learn.
In particular, I am struck by the practical difficulties involved in
having both a forward function and an inverse function, with the
parameters from one being transferred for use in the other. Much has
been made of the perfectness of this method, since in principle the
controlled variable can be made to match the reference exactly, whereas
the PCT model must always contain some error to make it act. But now we
see that this perfection depends critically on being able to compute one
function that is the exact inverse of another. In symbolic mathematics
this can often be done, but if we require that these functions be
represented by neural networks that operate without using discrete
symbols, then not only must the inverse be perfectly computed, but it
must be computed by a different network from the network computing the
forward function. Further, the changing parameters of the forward
function (represented, no doubt, by synaptic weightings) must be
continually and perfectly transferred (somehow) from the forward to the
inverse neural network.
I'm not trying to shoot down the whole concept of adaptive control here;
only the unrealistic method used to implement it, and the spurious claim
that this method is inherently perfectable. The most serious defect of
this implementation is the duplication of functions: the necessity of
doing the computations of the model twice, once forward and once in
reverse. This duplication is unnecessary.
There is, in fact, a way to achieve the same result within any
_practical_ limits of perfection without this duplication of functions.
All that is required is to set up an internal loop that works as a PCT-
style control system. The adaptive part of the model can be left exactly
the same.
ref signal
>
v
--------------->---- COMPARATOR
> >
> > error signal
> v
> OUTPUT FUNCTION
> >
> > output u
synthetic --<--- FORWARD MODEL<-----
perception | x' ^ |
> param | adjust |
\ | |
-> adaptive process |
/ | system
real | |
perception | y |
- - - - - - - - - - - - - - - - - - - - - - - - - - -
> > environment
x <----- REAL WORLD <------ u
f(u)
Now there is an output function of the same kind that would be used in
any PCT model. The loop inside the controlling system is a PCT loop. Its
dynamic characteristics are adjusted for stability, and its gain is made
as high as possible or as high as needed (an integrator would produce
effectively infinite gain at low frequencies). The synthetic perception
is directly compared against the reference signal, so only the error
signal enters the output function. This means that small errors in the
functions affect the difference between the synthetic perception and the
reference signal, and are strongly corrected. Also, the same output
function can work over a wide range of parameters of the forward model,
now the only form of the model used by the system. A simple adaptive
function associated with the output function (not shown) and using only
the error signal as input can maintain stability of the system. This
adaptation does not have to be explicitly coordinated with the main
adaptive process.
Notice that if the synthetic perception x' is maintained near the value
of the reference signal, then the output u is effectively the inverse
model function of the reference signal. The difference between this and
the original diagram is that the inverse is achieved by feedback rather
than open-loop computation. It is achieved by what is known in analog
computing as an "implicit" computation.
One last point. A simple switch can now change the system in diagram 2
back and forth between real-time and model-based control. The switch
simply determines whether the perceptual signal entering the comparator
comes from the real perception or the synthetic one. When the connection
is to the real perception, the adaptation of the model continues as
before; the model continues to be updated even though not being used for
control. The obvious advantage is that there is no need to anticipate
disturbances in the real-time mode, and control can be maintained over a
range of parameter variations in the real world. When real-time
perception is lost, this loss can be detected and the switch can be
thrown to the model-based control mode, so that pseudo-control can be
continued at least for a short time. Even the Extended Kalman Filter
model requires a means of detecting that the real-time input has been
lost, so that the rapid adaptations can be stopped.
···
-----------------------------------------
By investigating human control processes with the above models in mind,
we should be able to say which version of adaptive control is the better
one. If the second model is the better one, we should find that upon
loss of input, the first thing that happens is what would happen in a
real-time control model, probably a tendency for the output to start
changing rapidly. But after a short delay (when some higher system
realizes that the input has been lost) the input switch will be thrown
to the other position, and the model-based control mode will be seen, at
least for some short time. The person will control an imagined
perception, with the output actions becoming progressively less
appropriate to the actual state of the environment. Eventually, the loss
of actual control will be detected through errors in other systems, and
higher systems will cease to use the affected control system.
The same investigations, I predict, will show that many control tasks do
not involve model-based control at all. When the input is lost, control
is lost and is never restored even approximately until the input
returns.
Without experiments, there is little reason to debate the relative
merits of these models. They are all plausible, but they are not all
defensible by data about real behavior.
-----------------------------------------------------------------------
Best to all,
Bill P.