[From Bill Powers (960704.1900 MDT)]

RE: Hans Blom's adaptive model.

I did a search on "Kalman Filter" on WWW today and got 27,000 hits. I

guess this idea is pretty popular. Just before the Fort Lewis computer

went on an unscheduled vacation, I was looking at a paper by a guy in

Chapel Hill (UNC department of computer science) purporting to be an

"introduction" to Kalman Filters. The mathematical notation is beyond my

comprehension, but putting the words together with what I could grasp, I

think I got a better picture of how the Kalman filter works, at least in

the application Hans describes. I hope you will tell me how far off the

track I am, Hans.

The method seems to involves adjusting the parameters of a function to

make the value of the function match some particular value or time

function. In doing this, an error measure is calculated, and then, using

the form of the model, the partial derivative of the error with respect

to each parameter is calculated. Then, using the partial derivatives

found in this way, each parameter is adjusted by a small amount

(proportional to the partial) in the direction that would tend to

decrease the error measure, and the value of the function and of all the

partial derivatives is computed again. If the steps by which the

parameters are adjusted are not too large, this procedure will finally

converge to some stable set of parameters and a minimum in the error

measure.

Along with this basic process, there seems to be some use made of

uncertainty measures. As near as I can figure, they are used to adjust

the amounts of change in the parameters on each step. I suppose this is

done so the convergence won't be unduly affected by noise, or perhaps so

that noise fluctuations won't be treated as systematic errors.

Is this anywhere near right?

## ···

--------------------------------

The model proposed by Hans, I now realize, is not really a closed-loop

model as we understand it in PCT. Here is a very simple diagram of how

it works.

> Ref signal

^ |

> modeled v

> perceptual ----> Inverse of

> signal / function f'

> adj /params |

> / |

> Model of |

x'<--- environment, <------ |

> function f' |

> ^ |

v (adj | params) |

\ | |

Extended Kal. Filter | system output signal,

/ | u

^ perceptual signal |

> >

> >

> Real |

x <-- environment <-------

function, f

("the plant")

For this model to operate, the form of the real environment function

must be known, and it must exist inside the controlling system both in

the forward (normal) form and in the inverse form. The function f

describes not only the environmental "plant" but the properties of any

transducers and actuators that convert the signal u into a physical

effect on the plant. If, for example, the actuator is a human arm, and

the plant is some device operated by positioning the arm, the function

f' must include the kinematics and dynamics of the arm itself as well as

the plant. The inverse function must include the inverse kinematics and

dynamics of the arm and also the inverse of the plant function. There is

nothing in this model that can generate f' or its inverse; both of these

functions must be given before adaptation can begin. Adaptation consists

of adjusting the parameters of f', and somehow also adjusting the same

parameters in the inverse form of f' to the same values.

The Kalman Filter algorithm adjusts the parameters of the model function

f', so that when the real environment and the model are driven in

parallel by the output signal u, the modeled perceptual signal x'

behaves the same as the actual perceptual signal x. I am assuming here

that the real perceptual signal and x are the same thing. If they are

not, then the model must also include the properties of the perceptual

input function just prior to x'.

The model f' is not used in the actual control process. Its only purpose

is to provide a way for the Kalman Filter algorithm to determine the

parameters of the external function f. As those parameters change during

the adaptation process, they are copied into the inverse model that is

driven by the reference signal. It is the reference signal operating

open-loop through the inverse of the model function that actually

generates the output signal u. There is no comparator or closed loop in

this part of the model.

The only proposed advantage of this organization over a simple negative

feedback control system (other than its adaptive properties) is that it

can continue to operate in a disturbance-free environment after the

perceptual signal x is lost. Variations in the reference signal will

continue to be converted to changes of output that would be appropriate

if the environment did not change. The modeled perceptual signal x'

would continue to vary as dictated by the changing reference signal.

To achieve the same effect with an ordinary PCT control system is not

easy. Of course very little effort has been devoted to trying to achieve

this effect -- there are certain not 27,000 references to papers on that

subject.

Both models would require a special detector to report the loss of the

feedback signal, so that the absent input signal would not be treated as

a real input value of zero. This detector would have to actuate some

device that disables the normal operation of the system: for the

adaptive Kalman Filter model, it would have to disable the adaptation

process. The adjustment required in the PCT system might be complex,

although there may be ways of organizing the system so that only one

signal need be clamped to zero. Systems with integrating outputs will

hold their outputs when the error goes to zero. In a multileveled

system, the highest derivative might be held constant, or might be

blindly set by a reference signal.

There are clearly some advantages to the Kalman Filter approach, in

terms of achieving continued operation after loss of feedback

information. However, the most critical question in relation to models

of living systems is to what extent living systems have this capacity.

Obviously, if control processes by living systems degenerate when

feedback is lost, a model that continues to operate properly would be

the wrong model. The correct model would show the same degeneration of

control (if any) that the real system shows. If the purpose of a

control-system model is to imitate a living system, finding the "best"

control organization is not the goal; finding the best organization

would be appropriate in industrial design, but not in modeling

organisms.

Also, there are alternatives to the architecture of the adaptive Kalman

filter model. It is possible that rather than the same control process

continuing when feedback is lost, some alternative control process is

started which uses different feedback information to achieve the same

end. An example is entering information onto a computer screen. When a

familiar sequence is created on the screen over and over -- a login

process is a common example -- the system is reliable enough that it is

not necessary to observe the echoed characters during typing, although

normally one does so. If the system response is slow, one can type the

correct letters by watching the keyboard instead of the screen, or for

touch typists, by feeling the correct keystrokes. Errors of typing can

be detected from visually or tactilly observing which keys have been

depressed; errors are essentially never introduced between the keypress

and the character that appears on the screen. If the typing is done

correctly, eventually the expected sequence of letters will appear on

the screen.

There are many cases where we adjust things not by watching the variable

to be adjusted but by watching something on which that variable reliably

depends. While one could adjust the temperature of a room just by

feeling the air temperature and moving the setting lever on a

thermostat, the more common way is to set the lever to a calibrated

mark. Only if this resulted in a long-term felt temperature error would

one use the larger control loop, and even then one would probably just

change the target number to which the lever is set. It's quicker to

perform a calibrated action (which is itself an example of perceptual

control) and rely on the constancy of nature to produce the desired

result. I think we do this whenever it is possible and whenever precise

control is not critical; we use the lowest level of control that will

actually produce the desired higher-level result, near enough. Naturally

if the environment changes, the higher-level loop has to be brought into

real-time action, to reset the target value of the calibrated action.

And of course in many situations there is no calibrated action that will

make control easier. These are situations where unpredictable

disturbances are frequent, and in which the effects of actions on the

environment are not exactly enough reproducible.

This kind of possibility narrows the range of situations in which true

adaptation is required. And of course it does not call for any control

system to go on operating normally when feedback information to that

system is lost. One simply changes control systems -- or, realistically,

control simply fails.

I am convinced that in some kind of behaviors, probably at the higher

levels, a model similar to the one Hans proposes might be appropriate.

The greatest disadvantage of this model, however, is that it requires

the nervous system to know the form of an environmental feedback

function, and to be able to compute its exact inverse, or at least one

valid inverse. This kind of knowledge is hard to account for in lower-

level systems -- it's hard to imagine the midbrain or the brainstem

computing the inverse of a complicated function, and hard to explain how

the internal model acquires the correct basic form which is necessary

before parameter adjustments can achieve anything.

At higher levels, where cognitive functions are to be expected, complex

operations are more to be expected (although it's still embarrassing not

be be able to give even a hint as to how they are carried out). However,

at lower levels of control behavior, the evidence is very strong that

interruption of feedback pathways grossly distorts behavior and often

makes it impossible. Few people can thread a needle or drive a car

blindfolded. The adaptive model that continues to produce organized

behavior in the absence of feedback would simply be the wrong model for

these kinds of behavior.

The Kalman Filter approach seems to be a general method for

systematically adjusting the parameters of a function to achieve a

desired input-output relationship. It could be applied in many ways

other than the particular way it is used in Hans' model. Also, there are

other methods for doing the same adjustments, involving different and

often much simpler algorithms. My artificial cerebellum model does the

same kind of thing, but uses no symbolic calculations at all (it's

basically an analog computation). The author of that paper I mentioned

above said that the basic principle is a principle of feedback control.

It's the same basic idea as "backward propagation." You set a desired

outcome, compare the actual outcome with it, and use the error as a

basis for adjusting something about the process that leads to the

outcome. There are probably many methods for doing this, of which the

Kalman approach is one. I expect that we have not yet found the simplest

of such processes, or the kinds that would be most amenable to

implementation in a nervous system.

-----------------------------------------------------------------------

Best,

Bill P.