[From Bill Powers (950607.0630 MDT)]

Hans Blom (950606) --

I gave a lengthy reply yesterday to your paragraph asking about the

hierarchical control concept. I hope it is now at least starting to be

clear that the PCT model in general is designed to control at many

levels, not just the lowest. I did not discuss how a control system at a

given level may operate by setting references for multiple systems at

lower levels, and control perceptions derived from many perceptual

signals of lower levels, but at least we have a start.

Now for the rest of the post...

RE: the calibrated gas engine:

This is a great example of what the Kalman Filter approach in

effect does: as a higher level perception, it computes the

(parameters of the) calibration curve. But it computes those

parameters ON LINE, so when the calibration curve changes, this

will be known and can be used -- as long as there are observations.

If not, the old (and now outdated) calibration curve will still be

used.

Bill Leach already caught this one but I'll repeat it. If the Kalman

filter has continuous input from the speed of the engine, then there is

a tachometer sensor on the engine and a negative feedback control system

would be a far better and simpler way to control the speed. Loss of the

tachometer signal (a rarity) need not cause a disaster; the engine speed

can be limited by other means.

RE: Kalman filter adjusting output function

In theory, that is perfectly possible: a "best" inverse of the

environmental feedback function could well be computed. In

practice, however, this might prove to be very difficult or

impossible, especially when the environmental feedback function

contains a significant delay; its inverse would then need to

contain a perfect predictor and thus is not physically realizable.

A forward model does not have those problems: a model of something

physically existing IS physically realizable.

I don't see that a "perfect" predictor is needed; we're not trying to

produce "perfect" performance. A mathematical model that performs

perfectly wouldn't be a model of any real system.

Actually when a control system engineer is designing a control system

that has an environmental feedback function with a delay in it, he is

doing exactly what my suggested Kalman filter would be supposed to do:

set the system parameters for the best stable control. Since this is

successfully and frequently done, it must be possible. The problem with

a purely mathematical analysis is that it may show that the system

equations don't have a realizable solution, which discourages the

designer because the math can't answer the question, "Well, then, how

close CAN we get to an exact solution?" The answer usually is "Close

enough for any purposes we can think of."

You:

>>... a "controller" is a (good) controller only in particular

>>environments. Or, equivalently, whether a system is a (good) con-

>>troller depends on the environment in which it finds itself.

Me:

>This is true, but some controllers are able to work without change

>in a wider range of environments than others.

You:

Sure. The technical term is "robust". Some controllers are more

robust than others.

Oh. So you knew all the time that some controllers have a wider range of

control than others. Silly of me to mention it. >:(

The literature (and my own research) suggests that a PID-

controller, for instance (the type that you mostly use is a PI-

controller; the I stands for the integrator that you use in your

output function, the P comes into effect at low frequencies through

the integrator's "leak"), continues to function well if the parame-

ters of the world (delay time, gain, dominant time constant) change

by a factor of at most 2 to 3.

Depends on _which_ parameters change and on your criteria for adequate

performance. If the time constants or delays change, yes you can run

into instabilities. If the gain changes or static nonlinearities appear

(a more common kind of change in the environment), the control system

can handle a far wider range of variations without any substantial

degradation of performance -- at least 10:1 and often larger.

RE: open-loop, closed-loop, and adaptive closed-loop control

the slightest change in any aspect of the

environment or in the transfer function will directly affect the

result, making it different from the desired result; control will

disappear.

Now make this open loop "controller" adaptive, so that it continu-

ously updates the "calibration curve" (or its equivalent for a

dynamic system): control will reappear.

You're jumping the gun; that was my third case. If you have continuous

feedback information available, your best bet is to use a simple

negative feedback control system; that will add robustness, as

illustrated in the example about an operational amplifier.

If you want to compare this op amp diagram with the functioning of

an organism, you should note that the feedback element models the

environment, which will generally not be a simple resistor. It

might contain a delay, an integrator, a varying gain or what have

you. Replace the feedback element by a complex impedance and

calculate when you have a control system: only under severe

restrictions...

Not that severe. My point in using this example was that you can freely

substitute a great variety of feedback elements for the resistor in

order to do different calculations, and the system will still perform

within specifications (typically, 0.1% computing accuracy). While the

error signal at the input might vary over a factor of 20 from one

configuration to another, the gain of the op amp is so large that even

the largest error still does not induce a computing error greater than

0.1% (or to say that differently, the error for a resistive load is so

low, on the order of one part in 10^5, that it can grow 20 to 100 times

before becoming comparable to uncertainties in the values of the passive

components).

Things can be improved, however, if it is within your power to

change the series element into a complex impedance as well, the

characteristics of which you can change. If so, you have

implemented adaptive control.

Yes, exactly. That is the next stage that I would expect, adding

adaptive control to adjust the parameters of a closed-loop negative

feedback control system. That is what the reorganizing system in the PCT

model is supposed to do. But what you are doing is dropping the

continuous negative feedback control aspect, and substituting an

adaptive system to modify an _open-loop_ system. That is why I said your

model seemed like a step backward.

On further study, I discovered that in your model, when set as

appropriate for negligible observing noise, the system effectively

becomes a negative feedback control system (of low loop gain, but that

could be corrected), with no real-time adaptation going on (the world-

model continues to adapt, but it has no effect on the ongoing behavior).

In order to retain the feature of continuing to act reasonably when

feedback is lost, your model requires an external system that detects

the loss of signal and switches the system so it now controls the

internal world model; the real plant is then affected in a way

appropriate to a disturbance-free environment.

... The real point is that this "control system" manages to keep

its controlled variable, the voltage at its negative input,

precisely matching the voltage at the reference input in all these

cases, with no adaptation or other adjustment to compensate for the

radical changes in the series and feedback elements.

Analysis will show you that this is not true. A strict relationship

must exist between the series element and the feedback element if

the diagram above is to represent a control system rather than an

oscillator.

Analysis may show you that, but experience with real analog computers

will tell you otherwise. Of course there are choices of components that

will create an oscillator; that is often done deliberately, for example

to create a tuned filter or a signal generator. But the "strict

relationship" of which you speak is actually a very broad and permissive

condition that allows you to use everything from an inductor through a

resistor through a capacitor as the feedback element, without any change

in the op amp and without exceeding the error limit of 0.1% computing

accuracy. All you are saying is that there are limits; of course there

are. But they are enormously broader than the limits of an open-loop

system.

And it does this with absolutely no advance information about what

voltage waveform will be applied to the input. In fact, that is the

point of this analog computing component: it will compute a

particular function of ANY input waveform, the value of the function

being continuously represented as its output voltage.

This is not true either. It may be that the response to certain

wave forms (say DC) is perfectly all right, whereas the response to

other wave forms (say square waves) shows tremendous over- or

undershoots. Have you ever calibrated the input section of an

oscilloscope?

I used to make my living designing and building electronic systems. Of

course I have calibrated the input section of an oscilloscope. And how

do you do that? You apply a repetitive square-wave signal to the input

(kindly provided inside the scope by Tektronix), adjust the peaking so

you get the fastest rise-time without overshoot, adjust the capacitive

probe compensation to eliminate low-frequency droop, adjust the balance

of the differential amplifier for correct DC zero, and adjust the local

feedback resistor for quantitatively accurate DC response. The

oscilloscope response is then as good as it can be for ANY POSSIBLE

INPUT WAVEFORM from DC to the maximum frequency of response.

An operational amplifier is designed to maintain its zero accurately,

and to have the correct frequency-gain rolloff to work properly with a

wide range of feedback elements. The gain-bandwidth product, as Bill

Leach commented about modern op amps (I haven't used one for over a

decade), can be up to 500 million. The internal compensation is set to

allow capacitive feedback at all frequencies. So the op-amp works in

analog computations with an accuracy of 0.1% or better for resistive and

capacitive loads at frequencies at least up to 100,000 Hz.

You may be able to pull rank on me when it comes to mathematical

analysis, but not when it comes to practical electronics :->

You may want to check how well (white) noise can be "resisted" in

the case of an op amp. I have redrawn your diagram and added the

noise as an extra voltage source in series with the feedback

element. ...The qualification "can resist" ought to be quantified.

:

All the frequency components of the white noise up to about 100,000 Hz

can be resisted (at the input) with an accuracy of 0.1% by the op amp

that Bill Leach described. The input section of an oscilloscope is

essentially a high-gain, ultra-high-speed op-amp with resistive feedback

elements, capable of handling frequencies up to 100 megahertz with 2%

accuracy. If noise were added in the position you indicate, it would be

reduced at the input by a factor of at least 1000 over the frequency

band in which the op amp is designed to work. The output would vary

equally and oppositely to the disturbance fluctuations over that

frequency band. Don't forget that in the op amp, the equivalent of the

controlled variable is the voltage at the negative input, with the

reference signal being the voltage at the positive input. The output

voltage is merely the means of control, the system's action (your u).

I maintain that a disturbance can be the better resisted the better

it can be modelled.

How are you going to "model" a white noise disturbance? The modeled

disturbance would have to be adjusted not only to match the mean, RMS

amplitude, and variance of the real disturbance, but to create

instantaneous fluctuations in exact synchronism with, and of the same

amplitude as, those of the external noise source.

And for that matter, how are you going to model even a low-frequency

disturbance that doesn't follow any analytically-describable waveform

and that doesn't repeat in a predictable way? If I gave you a sample of

the disturbance (of any length you like), you could do a Fourier

analysis and then create a waveform generator that would recreate the

sample, but that would be good only for the sample that has already

occurred; it would not apply to the next sample. Even if I gave you a

sine-wave disturbance, you might be able to adjust a sine-wave generator

to match it, but within a few thousand cycles (or less) your generator

would start to get out of phase with the external sine-wave, and you

would end up adding more disturbances instead of correcting for them.

There is simply no practical way to create a model of an external

disturbance that will work for more than a very brief time.

Mathematically, of course, it's simple: if the external disturbance is

D*sin(1/t), you just write that equation for the modeled disturbance and

the real disturbance will be exactly cancelled forever. But just try to

realize that cancellation in a physical system!

"Noise" is just a convenient category in which we place waveforms that

don't fit any handy analytical form. Basically, noise is just a

variation in some variable. The magnitude of the variable rises and

falls with time. If a noise-disturbance is presented to a control

system, the control system will oppose the resulting variations of the

controlled variable as fast as it can, getting rid of the slow

variations and leaving the ones too fast for it to oppose. Often a

controlled variable will be associated with a mass of some sort, which

by itself will help filter out high-frequency disturbances.

A disturbance is a disturbance, whether we can categorize its behavior

or not. In a control system there is no reason to handle "noise"

differently from any other category of disturbance waveform. Unless, of

course, you think that you have to be able to model a disturbance in

order to oppose its effects.

Since white noise cannot be modelled, it is the base line. If part

of the noise can be modelled and used for control purposes, this

would enable us to place an extra (modelled noise) voltage source

in series with the series element, which partially compensates the

world-noise.

You're dreaming. This is a totally impractical proposal, and totally

unnecessary. Negative feedback control will oppose disturbances up to

the bandwidth limit of the control process, and you will never be able

to do better than that by any means short of magic. Your "modelled

noise" source would never stay in synchronism with the real noise

source; you'd just be adding more noise in quadrature to the noise that

already exists.

Bill Leach says:

Without trying to quote some more of your text... are you sure that

you could design a model based controller for one of the simpler PCT

tests and obtain similar results (including the sorts of changes

made to the typical experiment)?

And you say:

Yes. Regrettably (for Bill Powers) the very exact specifications

that would be required to establish the QUALITY of control would

result in an equally exact model specification.

Ok, put your model where your mouth is. I don't believe you can do this.

It is extremely simple to set up an experiment with a human subject and

then fit a control-system model to it that will behave the same way the

human subject does, within 5% RMS of the handle movements, for the

disturbance used during the experiment AND ALSO FOR ANY SUBSEQUENT

RANDOMLY-GENERATED DISTURBANCE.

Our specifications for quality of control are very simple: the subject's

handle movements should match those of the model within 5% RMS of the

range of handle movements, when you lay one trace over the other. The

requirement is simply that the model should behave the same way the real

person does.

I should point out that we generate tables of random disturbances which

are unknown beforehand both to the person and to the model. If you try

to meet this challenge, you will have to do it under the same

conditions. Perhaps you can figure out a way for your model to derive

the form of the disturbance from the same information that is available

to the subject: the behavior of the cursor and target on the screen. If

so, fine, do it. I don't think you can do it.

Want me to send you a program for carrying out the experiment?

## ···

----------------------------------

... since optimal controllers directly tune their parameters in

such a fashion that optimal control (i.e. control that could not be

better given the control criterium and the model) results, no other

controller could be better.

What you're saying is just a tautology: if there were an optimal

controller, by definition it would adjust the parameters of control

(including its own parameters) in the best possible way. But this by no

means shows that you have found any optimal controllers; to demonstrate

that, you would have to show that no other form of controller could do

any better. You're talking in terms of a _particular design_ for a

controller, and of course for any particular design there is a set of

parameters that makes it work better than for any other set of

parameters. But that does not mean that this design works better than

any other possible design would work if also adjusted optimally.

Perhaps what leads you to speak in absolutes is the fact that your way

of designing a world-model and a controller for it produces performance

that is not just "optimal" -- it is _perfect_. What could be more

optimal than a system which makes x = xopt, not just nearly but with

infinite exactness, in a single iteration, no matter how short a

physical time is represented by one iteration?

But this very perfection should make us suspicious; the system you are

modeling doesn't have any physical properties, and neither does the

world-model or the control system. The control system can act in no

time; the external system can react to inputs in no time. One begins to

suspect that what you have is not a system model, but only an algebraic

identity. We get x = xopt because there is nothing else we can get when

the equations for the world model and those for the controller are

combined. The world-model plus controller make x = xopt for any values

of parameters whatever, provided that there is no division by zero

involved. So adjusting the parameters makes no difference in how the

world-model works. That is rather peculiar. If x = xopt for any values

of the parameters, why does not xt = xopt for any values of the

parameters? Why do the parameters make a difference in the response of

xt to u, when they do not make any difference in the response of x to u?

Somewhere, hidden in your equations and manipulations, there is the

equivalent of dividing by zero.

If the real external system is a physical system, then it can't behave

the way the world-model does. The length of time represented by one

iteration makes a difference; if that time is shortened, the physical

variables can change correspondingly less in one iteration. This basic

physical fact is not represented in the world-model. If it were

represented, you would no longer get x = xopt in one iteration; you

would find that the difference, xopt - x, would depend on the

parameters, the passage of time, fluctuations in the disturbance, and

fluctuations in the reference signal.

So your model is not an "optimal" model; it is an _imaginary_ model,

working in a massless, forceless, timeless universe. This is why I am

confident that you will not be able to find any form of this model that

will fit the behavior of a real person. In the real person, xt does NOT

equal xopt, so in any world model, x had better not equal xopt, either.

------------------------------------

Note that the value of pvv thus acts as a switch: pvv = 0 means

"control the actual value of xt" and pvv > 0 means "control the

output of the world model." This switch is actuated by an outside

intelligence.

This is the switch that toggles what you call the "imagination con-

nection". It is, however, a "soft switch" so that control can be

PARTIALLY based on both internal prediction and external

observation. That is how we humans often operate, I think: we have

a course inner model and use our observations only to fill in the

moment-to-moment details.

Yes. This is why I am still interested in the adaptive system with a

world-model, despite its serious defects. There must be some way to

combine the best features of the world-model system and the high-gain

negative feedback control system, so the system can resist disturbances

accurately without having to predict or model them, and so it can

operate with incomplete input information.

A hierarchical model may help a lot, because as I showed in a previous

post, a complex system can be broken down into a hierarchical system in

which each control system is very simple and handles only one simple

variable. This means that the "world model" used in any one control

system can also be extremely simple; only when you look at the behavior

of many systems of this kind in a hierarchical array do you see a result

that looks complex. I think that the hierarchical approach greatly

simplifies all control problems, including problems of adaptive control.

Each control system, after all, has only a simple task in a hierarchical

model: to make its perceptual signal match the reference signal it is

given, in one dimension. A proper world-model, then, only has to provide

a single signal of the appropriate magnitude, for any one controlling

subsystem. The model of the whole external world does not exist in any

one control system; it is distributed over all of them.

------------------------------------

RE: the model

4. Independent disturbances applied to both y and xt. All relevant

parameters manually set to agree with disturbance amplitudes.Now the control system makes x = xopt, and xt = xopt + (disturbance

of xt). There is no resistance to disturbances of xt. The model

output x follows the reference signal xopt, but the real system

output does not.

This is the most complex situation: the model attempts to

discriminate between what is "real" (in the outside world) and what

is due to bad sensors (inner defects). Sometimes such a

discrimination is not possible, e.g. when both disturbances covary.

Correction: the model does not attempt to make this discrimination. It

must be told when there is a direct disturbance of y. If it is not told

this, it will treat those disturbances just as it treats disturbances of

xt, and produce an output that compensates for the combined effects of

the two disturbances. It will then create fluctuations in xt that are

equal and opposite to fluctuations in the disturbance of y, _causing_

rather than correcting errors in the real xt.

This is essentially W. Ross Ashby's concept of a regulator,

described in _Design for a Brain_.

Thanks for the reference. I'll look him up.

Ashby, W. R. (1952); _Design for a brain_. New York, John Wiley & Sons.

Actually a more directly relevant reference is

Ashby, W. R. (1956); _An introduction to cybernetics_. New York, Wiley.

See especially the section on "Control" starting on page 213.

---------------------------------------

RE: hierarchy again

That is the other discussion: if the "world" changes significantly,

the controller must change significantly as well, i.e. it must be

adaptive.

In a hierarchical model, the lowest-level variables under control are in

an environment the least likely to change in significant ways. The

control loops are short and the variables are dealt with in isolation

from other variables. There are few interactions, because effects of

other variables are treated as disturbances and essentially cancelled.

So the lowest level of control creates a set of perceptual variables

that are adjustable simply by changing reference signals, and only

rarely would there be any occasion for adaptation of a serious nature

(some simple adaptations like gain changes can be handled locally).

Each added level both creates more abstract variables to control, and

simplifies and regularizes the world with which still higher systems

must interact. The latitude provided by negative feedback makes many

kinds of changes in the world invisible to higher systems, unimportant

in their operation. So the hierarchical architecture has the happy

effect of minimizing the demands on adaptive processes that work at any

given level. This is especially true in contrast to a system in which

the entire control process has to be carried out by a single highly

complex one-level system, in which adaptation may require the

simultaneous adjustment of large numbers of parameters -- all in the

right direction. A difficult problem in adaptation for a single-level

system may well turn out to become a modest set of simple adaptations in

a hierarchical system.

----------------------------------------------------------------------

Best,

Bill P.