[From Bill Powers (960228.0600 MST)]

I've been trying to think about model-based control in the context of

PCT, and the difference between symbolic and analog computation. I'm

quite sure that model-based control does happen, and that in general

outline it happens more or less as Hans Blom and the "modern control

theorists" propose. I doubt rather seriously, however, that it is

implemented in a brain in the same way Hans implements it in a computer.

I also doubt that it exists at the lower levels of control.

As a method of designing artificial computer-driven control systems to

operate "plants" of known nature, model-based control seems to be a

powerful method (although I don't know how well it works in practice --

is this how most contemporary control engineers now design their control

systems?). The question is whether it is also a general model of control

processes in living systems (from the bacterium to the human being).

We need to know how long a person or animal can go without sensory

feedback before losing control. If this turns out to be only a very

short time for a specific control task, then there are much simpler

arrangements than the one Hans proposes that could give the appearance

of continued control for a brief period after loss of sensory

information. If loss of sensory feedback in a particular task leads to

an immediate runaway effect, then clearly there is no model-based

control associated with that task. The basic factual question is, "Do

people act as if they are using model-based control, and if so, under

what circumstances?" The answer can be provided only by experimentation;

there is no way to answer the question on hypothetical grounds, or

through qualitative and essentially data-free anecdotes.

In a comparison of a very simple PCT model and Hans' very elaborate

model-based control model of a simple tracking experiment, Hans' model

controlled slightly better than the PCT model did. But both models

controlled two orders of magnitude better (in terms of remaining error)

than real people do in the same task: a tenth of a percent as opposed to

10 percent RMS error (at the level of difficulty used). Of course Han's

model was superior in one regard to the PCT model in that it adapted

itself to the situation while the PCT model was simply given fixed

parameters. This adaptation was somewhat limited, in that when feedback

was interrupted, Han's model lost control within a fraction of a second,

essentially just as fast as the PCT model did. But I would like to put

that consideration aside for now, because there are other simpler

methods of adaptation that would also work, particularly if the overall

model has to imitate only the imperfect control that a real person shows

rather than reaching some ideal degree of control.

The basic principle of control that Hans proposes is a method of

adjusting a model and of producing an output that affects both the real

world and the internal model. The internal model receives the output

signal, and responds to it by producing a perceptual signal. For

adaptation, this synthetic perceptual signal is compared with one

derived by sensing the real world, and the difference is used to modify

the model.

With adaptation complete, the model is used by the system in two ways.

First, it produces a synthetic perceptual signal for comparison against

a real one. This involves running the model _forward_. Second, the same

model is also used in its inverse form: its inverse is used to convert

the reference signal, open-loop, into an output signal. This is

something I have only recently realized. When I originally drew a

diagram of Hans' system, I drew an internal feedback loop, as if the

synthetic perceptual signal were being compared with the reference

signal, and the error was producing the output. Hans drew his diagram in

a similar way. But in fact, there is no internal feedback loop of that

kind. The synthetic perceptual signal is not compared with the reference

signal.

What is done instead is to let the adaptation procedure adjust the

parameters of the model; then the model's inverse, with those

parameters, is used open-loop to convert the reference signal into an

output signal. The internal "loop" is closed only through transferring

the parameters from the forward model to the inverse of the model.

I think this is a correct diagram of the organization of Hans' model,

where previous diagrams have been, at best, misleading. Notice that the

synthetic perceptual signal is NOT part of an internal loop that

includes the output path. It is used only in adjusting the parameters of

the model.

In a computer program, it is no problem to compute the forward form of

the model and also its inverse. The designer of the program knows the

general form of the real-world system (with parameters of the model

being left for the adaptation to adjust), and therefore can also compute

the inverse form (assuming that one exists). The parameters can be

stored in memory or a register, then recalled for use when evaluating

the output of the inverse function.

In one sense, a nervous system can do the same thing. In Hans' computer

program, we have the evidence that a nervous system (Hans') can set up

symbolic computations, work the math with pencil and paper to compute

the forward and inverse models, and write a computer program to

implement the computations. However, one is permitted to wonder whether

the nervous system has any _other_ way to do the same thing, without

going through the step of conversion to symbols and the execution of

procedures that one has to go to school to learn.

In particular, I am struck by the practical difficulties involved in

having both a forward function and an inverse function, with the

parameters from one being transferred for use in the other. Much has

been made of the perfectness of this method, since in principle the

controlled variable can be made to match the reference exactly, whereas

the PCT model must always contain some error to make it act. But now we

see that this perfection depends critically on being able to compute one

function that is the exact inverse of another. In symbolic mathematics

this can often be done, but if we require that these functions be

represented by neural networks that operate without using discrete

symbols, then not only must the inverse be perfectly computed, but it

must be computed by a different network from the network computing the

forward function. Further, the changing parameters of the forward

function (represented, no doubt, by synaptic weightings) must be

continually and perfectly transferred (somehow) from the forward to the

inverse neural network.

I'm not trying to shoot down the whole concept of adaptive control here;

only the unrealistic method used to implement it, and the spurious claim

that this method is inherently perfectable. The most serious defect of

this implementation is the duplication of functions: the necessity of

doing the computations of the model twice, once forward and once in

reverse. This duplication is unnecessary.

There is, in fact, a way to achieve the same result within any

_practical_ limits of perfection without this duplication of functions.

All that is required is to set up an internal loop that works as a PCT-

style control system. The adaptive part of the model can be left exactly

the same.

ref signal

>

v

--------------->---- COMPARATOR

> >

> > error signal

> v

> OUTPUT FUNCTION

> >

> > output u

synthetic --<--- FORWARD MODEL<-----

perception | x' ^ |

> param | adjust |

\ | |

-> adaptive process |

/ | system

real | |

perception | y |

- - - - - - - - - - - - - - - - - - - - - - - - - - -

> > environment

x <----- REAL WORLD <------ u

f(u)

Now there is an output function of the same kind that would be used in

any PCT model. The loop inside the controlling system is a PCT loop. Its

dynamic characteristics are adjusted for stability, and its gain is made

as high as possible or as high as needed (an integrator would produce

effectively infinite gain at low frequencies). The synthetic perception

is directly compared against the reference signal, so only the error

signal enters the output function. This means that small errors in the

functions affect the difference between the synthetic perception and the

reference signal, and are strongly corrected. Also, the same output

function can work over a wide range of parameters of the forward model,

now the only form of the model used by the system. A simple adaptive

function associated with the output function (not shown) and using only

the error signal as input can maintain stability of the system. This

adaptation does not have to be explicitly coordinated with the main

adaptive process.

Notice that if the synthetic perception x' is maintained near the value

of the reference signal, then the output u is effectively the inverse

model function of the reference signal. The difference between this and

the original diagram is that the inverse is achieved by feedback rather

than open-loop computation. It is achieved by what is known in analog

computing as an "implicit" computation.

One last point. A simple switch can now change the system in diagram 2

back and forth between real-time and model-based control. The switch

simply determines whether the perceptual signal entering the comparator

comes from the real perception or the synthetic one. When the connection

is to the real perception, the adaptation of the model continues as

before; the model continues to be updated even though not being used for

control. The obvious advantage is that there is no need to anticipate

disturbances in the real-time mode, and control can be maintained over a

range of parameter variations in the real world. When real-time

perception is lost, this loss can be detected and the switch can be

thrown to the model-based control mode, so that pseudo-control can be

continued at least for a short time. Even the Extended Kalman Filter

model requires a means of detecting that the real-time input has been

lost, so that the rapid adaptations can be stopped.

## ···

-----------------------------------------

By investigating human control processes with the above models in mind,

we should be able to say which version of adaptive control is the better

one. If the second model is the better one, we should find that upon

loss of input, the first thing that happens is what would happen in a

real-time control model, probably a tendency for the output to start

changing rapidly. But after a short delay (when some higher system

realizes that the input has been lost) the input switch will be thrown

to the other position, and the model-based control mode will be seen, at

least for some short time. The person will control an imagined

perception, with the output actions becoming progressively less

appropriate to the actual state of the environment. Eventually, the loss

of actual control will be detected through errors in other systems, and

higher systems will cease to use the affected control system.

The same investigations, I predict, will show that many control tasks do

not involve model-based control at all. When the input is lost, control

is lost and is never restored even approximately until the input

returns.

Without experiments, there is little reason to debate the relative

merits of these models. They are all plausible, but they are not all

defensible by data about real behavior.

-----------------------------------------------------------------------

Best to all,

Bill P.