[From Bill Powers (960710.1430 MDT)]

Bruce Abbott (960710.1035 EST) --

To achieve synchrony of two (or more) oscillators, all that is

required is negative feedback.

Actually, no feedback at all is needed. Synchrony of free-running

oscilllators can be achieved simply by perturbing them at intervals

close to the free-running frequency, like pushing on a pendulum at

nearly the same point in each cycle.

The reason I don't think that menstrual cycles are under control is

precisely that they can be synchronized by such minor influences as

pheremones. If they were actually under control (with respect to

frequency or phase) it would be difficult to change them. The fact that

a few molecules of a chemical can alter the cycle doesn't show that the

chemicals are superpowerful; it just shows that what they're affecting

isn't hard to change.

## ···

-----------------------------------------------------------------------

Hans Blom, 960710 --

Uh? Another misunderstanding? What you call "disturbance", I call

"noise". For me, this translation is automatic; the literature that

I have read does not talk about "disturbance" but about "noise".

As Rick Marken pointed out, we're talking about different things.

Disturbances do not have to be randomly distributed or have a mean value

of zero as you usually seem to assume; in fact most of them aren't of

that nature, because they're caused by perfectly regular and systematic

physical processes. Some of them, like gravity, are constant.

At any rate, what I think of as "noise" in perception would be simply

uncertainty about the state of the perception. In my experience,

perceptions are not in themselves uncertain -- for example, when I

perceive the brightness of a light, I perceive a steady or smoothly

changing value of brightess. I do not perceive a brightness b, plus or

minus some band of uncertainty. When I perceive a table, I do not

perceive a range of shapes (whether I'm filling in parts of the scene

from imagination or not), but simply the shape that I see. I can't find

any unpredictable fluctuations in that perception.

When disturbances do occur, I know of them through obvious changes in my

perceptions. The television set may unaccountably become dimmer. I am

not uncertain about that dimming; to me it seems that the brightess of

the screen has decreased from one value to another, and there is no

uncertainty about either value. And I can correct the change in

brightess by using the brightess control, bringing it back to the level

I prefer -- again, without any uncertainty.

Try to look again. Now with the mindset: what do my sensors truly

capture at this moment. That is extremely hard; we just don't

operate that way. And so we confuse what we perceive with what we,

sort of, reconstruct out of something internal that has been fed by

earlier observations.

You're talking about the mechanisms that (hypothetically) give rise to

perception, not about the experience of perceptions. If I hallucinate

the smell of pizza, the hallucinated smell is perfectly clear and not at

all noisy. What I'm talking about is the direct experience of the world

of perception, without any obbligato (a persistent or background motif)

of commentary or explanation or conjecture about what is causing it. I

don't experience any reconstruction; it may be going on, but it's not

part of the world of perception. I experience only the outcome of

whatever processes lead to perception. And that outcome is not normally

obscured, fuzzy, irregular, uncertain, or noisy. It is what it is.

When a perception IS noisy, it's clearly not normal. The room is nearly

blacked out so all objects are barely discernible, or the music is

overlaid with hisses, pops, crackles, and squeaks, or the TV picture is

full of snow and drifting bands of light and dark. Noise appears in

perception when it should: when the signals are close to the lower

threshold of detection, or when other signals are interfering

significantly. And then it is perfectly clear to the perceiver that

something is preventing normal perception.

-----------------------------------------------------------------------

Hans Blom, 960709b --

Glad to know I got somewhere near an understanding of the strategy of

the Kalman Filter.

Overall, you understand the approach but your reluctance to use (or

inexperience with) probabilities may make that it _seems_ beyond

your comprehension.

From what I can see, the basic strategy doesn't depend on probabilities,

but on partial derivatives. The probabilistic aspects of it come in when

the data contain a lot of noise, so there is some doubt about whether

the extrapolation is valid. In that case, the method lowers the weight

given to the predicted value and increases the weight of the measured

value. That seems simple enough to me. The concept isn't what throws me;

it's the supercompact notation, and the fact that the method is

described in terms of multiple variables. I never could keep subscripts

straight, and I always had trouble with matrix manipulations because I

couldn't visualize what was going on.

Basically, the method is a "real time" curve fit. You may be

acquainted with the least squares method of fitting a best straight

line through a number of points, resulting in two parameters: a

slope and an offset.

Yes, that's basically how Welch and Bishop explained it: it's a running

least-squares computation that is always as up-to-date as it can be.

They also say "The Kalman filter estimates a process by using a form of

feedback control."

I remember telling you, somewhen in the remote past, that my home

heating system does not have a thermostat. It is a feedforward

system that adjust the temperature of the circulating water to the

outside temperature, which is measured. (That such a system works

as well as it does may depend on the fact that in the part of

Europe where I live weather changes are not extreme and houses have

better insulation.)

I think it also arises from the fact that Europeans are much more rugged

than Americans, and don't demand very much of their thermostats. I'm

curious, though: just where does your system measure the temperature?

And how does it handle days when sunshine is pouring in the windows, as

opposed to days when it's cloudy? What happens when it's cold outside

and the wind is blowing? What happens when you build a fire in the

fireplace? What happens when someone opens a door?

Now assume that this adjustment takes place in real time, so that

the operating curve is always well tuned. At the lowest level, we

still have a feedforward control system, but one level up is the

(control?) system that continually adjusts the operating curve.

This is, in short, how Kalman Filter based control systems work. As

you see, this is different from the PCT paradigm.

Yes, it is. It seems to me that this system must be quite slow to

respond to novel circumstances.

For this model to operate, the form of the real environment function

must be known

Here, "known" must be interpreted in the same sense as we "know" a

straight line from the computed offset and slope fitted through a

collection of points. So we'd better say that it must have been

"modelled" than to say that it must be "known".

No, it means more than that. You assume a certain form of the

environment equation, which may or may not be appropriate. The offsets

and slopes of which you speak concern only the parameters in the

equation; you have to have nearly the right form of the equation, with

general parameters, before you can start adjusting parameters. If you

assume that the plant response is proportional, but it really involves

an integral or a derivative, no amount of adjusting the model parameters

will give you good control. The only way around this is to start with a

model that is completely general, containing all possible integrals and

derivatives and polynomials in every variable, each with an adjustable

parameter, and then hope that your method will discard those elements of

the general equation that don't apply.

Let's speculate a bit about what an internal model in a human could

be used for, apart from moment-to-moment control. Basically, what

is available is a description of _what can be done_: imagine a

pattern of actions and observe, again in imagination, what the

outcome will be in terms of perceptions. In short, an internal

model allows planning: imagine action (t) for t=0 to t=T, and you

will, within the accuracy of the model, observe perception (t) for

t=0 to t=T.

Yes -- that is roughly what I said about planning in B:CP. I agree that

this sort of thing must happen, at the higher levels of perception.

There are clearly some advantages to the Kalman Filter approach, in

terms of achieving continued operation after loss of feedback

information. However, the most critical question in relation to

models of living systems is to what extent living systems have this

capacity.

Do you know of another mechanism that "explains" planning? You

might, of course, that planning does not exist. But that does not

solve the problem.

I agree that this mechanism is probably involved in conscious planning.

I suspect it is not done by using the computational methods of the

Kalman Filter. I do not agree that it is involved in lower-level control

processes.

Obviously, if control processes by living systems degenerate when

feedback is lost, a model that continues to operate properly would be

the wrong model.

Some time ago, I proposed a test: have the feedback signal go away

occasionally in a tracking experiment and observe the subject's

behavior. What happens? A) The subject stops, complaining that he

cannot track when no feedback is available. B) The cursor remains

at a constant position as long as there is no feedback. C) The

cursor follows a non-constant trajectory that is some sort of

extrapolation of its previous path. D) ... Only A would indicate

that no internal model exists. B would occur with the simplest

model thinkable: a zero order hold. C might be a first or higher

order hold.

What happens in a tracking experiment is normally A. The subjects say

"Are you crazy? How can I track when I can't see the cursor?" Once in a

while we see B): the subject freezes. And occasionally we see C. In all

cases, tracking immediately (or in a second or two) deteriorates, errors

growing tenfold or more.

If the break in visibility is brief enough, subjects will continue to

move the handle and, if no large change in the disturbance occurs during

that time, the actual cursor position will look like an extrapolation of

the curve and may remain close to the target. However, the time over

which this happens is so brief that it can be accounted for without

invoking an internal model. I would look more seriously for an internal

model if there were evidence that the model took over and produced some

extended pattern of output changes. But that doesn't happen. What seems

to happen is that the last velocity of the handle simply continues.

... control _theory_ might shed some light on what is possible and

what not, and therefore suggest some of the tests that might be

performed. Bruce Gregory describes this aspect of the value of a

theory extremely well.

I agree to both sentences.

I am convinced that in some kind of behaviors, probably at the higher

levels, a model similar to the one Hans proposes might be

appropriate. The greatest disadvantage of this model, however, is that

it requires the nervous system to know the form of an environmental

feedback function, and to be able to compute its exact inverse, or at

least one valid inverse.

This is a misunderstanding. In the first place, "smooth" functions

can be well approximated by only a few terms of their series

expansion, and many of the "laws of nature" appear to be smooth.

That means that a simple model can usually provide a pretty

realistic approximation of reality. Second, exactness is not

required, precisely because control goes on: errors will be

corrected upon their perception. Control -- the continuous

availability of feedback information -- allows simple

approximations to be effective.

I like this much better than what you've been saying. I simply can't

believe that the brain is doing all those computations that appear in

your programs. But if there is a simple approximation to these methods

that can be done without such elaborate arithmetic, I would be much more

interested. I agree that the closed-loop aspect of the process will

forgive many inaccuracies and approximations.

This kind of knowledge is hard to account for in lower-

level systems -- it's hard to imagine the midbrain or the brainstem

computing the inverse of a complicated function

Hardly more difficult than imagining the brain to compute a multi-

plication or an integral...

Far more difficult, because approximate multiplication and integration

can be done by analog methods, even by a single neuron, while computing

the general inverse of a function requires symbol manipulation and the

formal operations of mathematics.

Few people can thread a needle or drive a car blindfolded.

But _all_ people who drive a car occasionally take their eyes off

the road without causing an immediate accident.

They do this not by controlling a model but simply by holding the

steering wheel in a fixed position. You are dodging my example, though.

Can't you explain why a person can't thread a needle or drive a car

blindfolded?

---------------------------------------------

Regarding the artificial cerebellum model:

I have found a more interesting version of this model that I'll send to

you first. If I send you an executable version, do you have a machine

you can run it on (a PC)? If not, getting it running might be a pain,

because I use two machine-language routines for the algorithm itself --

when I worked this out, back in the mid-80s, Pascal and C were just too

slow on the machine I then had. Perhaps I could try rewriting that part

in Pascal, if you can't run the object code immediately.

In the version I want to send you, the dynamic model of the arm from

Little Man v2 is used, and a two-level control system operates each

joint. The lower level controls (angular) velocity by varying torque,

and the higher level controls position by varying the reference level

for velocity. A total of four "artificial cerebellum" modules are used

for the four output functions of the control systems, so the entire

model adapts from scratch.

Let me know if you'd rather have an all-Pascal version; if so, I don't

know if I can get the revised program running before we leave for the

CSG meeting (the 16th). If you can use object code, I'll send a MIME64

version of a self-extracting .zip file (along with all the Pascal source

code and the assembler listings).

----------------------------

I want to bring up a point about hierarchical control. If you'll look at

the first-level test modes of Little Man V2, brought up with F1, F2, and

F3, you'll see that the behavior of the joint angles becomes a very

simple function of the reference signals. The perceptions representing

joint angle respond to changes in the perceptual signals as if there

were a single-pole Tchebyschev (sp?) filter connecting them.

Without the first-level control systems, the arm would behave like a

very wobbly mass on a spring, with not much damping. A step-change in

the driving signal to the motor neuron would result in a large damped

oscillation of position, and thus of the signals representing position

and velocity. These oscillations would be complex because of coupling to

other limb segments.

It's possible, of course, that an adaptive system at a higher level

could still control the position signals it receives by emitting the

right driving signals to the motor neurons. But the model that this

system would need would have to represent the second-order differential

equation that describes the arm's response to driving forces, as well as

all the nonlinear and cross-coupling terms that arise because of

linkages to other arm segments. The reference signal would have to pass

through the inverse function, too, so the system would have to know the

form of the inverse function as well as the forward function.

However, with the lower feedback control systems in place, the higher

system experiences a much simpler relationship between the driving (now

reference) signals and the resulting position signals. A step-change in

the reference signal results in a quick asymptotic rise of the position

signal to the specified level (of nearly the form 1 - exp(-kt)). The

interactions among the control systems operating different joints all

but disappear; each joint control system operates independently of the

others, or appears to do so from the standpoint of the higher system.

All signs of underdamping are gone.

In short, the model that the next level of control uses can be a simple

first-order linear differential equation: almost trivially simple in

comparison with what would be required without the lower-level control

systems.

Your remarks on using simplified models reminded me of this. As I say, I

simply can't believe that the brain actually performs all the

computations that the Kalman Filter approach seems to call for. But I

could believe that some simple approximation to the Kalman approach

might actually exist in the nervous system, particularly if there were a

way to simplify the environment that a given control system has to deal

with -- meaning, of course, to simplify the MODEL that the control

system needs in order to operate adaptively in the Kalman manner.

What would make me sit up and take notice would be a hierarchical model

in which some VERY simple approximation to the Kalman method is used to

do some model-based controlling of an environment simplified by lower-

level control systems.

-----------------------------------------

Without nonlinearities, the "plant" in the A.C. demonstration is

described bym(d2x/dt^2) - k1(dx/dt) - k2(x - x0) = [driving force].

What is the inverse of this equation that you would use in your

model?

I don't understand the question. Do you want an explicit expression

for x as a function of the driving force? Solve the differential

equation. It's a simple second order system. What's the problem?

Actually I said this the wrong way. Given the behavior of x, the above

equation allows us to calculate the driving force u(t) that would

produce that behavior. That is already the inverse of the forward model

The forward model would be the solution of the above differential

equation: it would express x as a function of the driving force, u. The

general solution is a rather complex thing which I don't need to go look

up, I hope. But it is the model of the environment. You need both that

model and the inverse expression to use your "feed-forward" method. The

forward model is used to find the parameter values; the inverse model

(given above) is used to convert the reference signal into the output u.

---------------------------------

If I had the equations of motion of the jointed stick, I'd love to

try a PCT model of this [balancing three jointed sticks]. I'm not very

handy with dynamical equations -- I'd have to have them given to me.

Each stick is described by a formula like the one you give above.

Not quite -- the sticks interact with each other. You need all the

cross-coupling terms and all that, which I'm not up to producing.

Perhaps you could find the actual equations in some report on this

stick-balancing feat.

-----------------------------------------------------------------------

Best to all,

Bill P.