[From Bill Powers (950607.0630 MDT)]
Hans Blom (950606) --
I gave a lengthy reply yesterday to your paragraph asking about the
hierarchical control concept. I hope it is now at least starting to be
clear that the PCT model in general is designed to control at many
levels, not just the lowest. I did not discuss how a control system at a
given level may operate by setting references for multiple systems at
lower levels, and control perceptions derived from many perceptual
signals of lower levels, but at least we have a start.
Now for the rest of the post...
RE: the calibrated gas engine:
This is a great example of what the Kalman Filter approach in
effect does: as a higher level perception, it computes the
(parameters of the) calibration curve. But it computes those
parameters ON LINE, so when the calibration curve changes, this
will be known and can be used -- as long as there are observations.
If not, the old (and now outdated) calibration curve will still be
used.
Bill Leach already caught this one but I'll repeat it. If the Kalman
filter has continuous input from the speed of the engine, then there is
a tachometer sensor on the engine and a negative feedback control system
would be a far better and simpler way to control the speed. Loss of the
tachometer signal (a rarity) need not cause a disaster; the engine speed
can be limited by other means.
RE: Kalman filter adjusting output function
In theory, that is perfectly possible: a "best" inverse of the
environmental feedback function could well be computed. In
practice, however, this might prove to be very difficult or
impossible, especially when the environmental feedback function
contains a significant delay; its inverse would then need to
contain a perfect predictor and thus is not physically realizable.
A forward model does not have those problems: a model of something
physically existing IS physically realizable.
I don't see that a "perfect" predictor is needed; we're not trying to
produce "perfect" performance. A mathematical model that performs
perfectly wouldn't be a model of any real system.
Actually when a control system engineer is designing a control system
that has an environmental feedback function with a delay in it, he is
doing exactly what my suggested Kalman filter would be supposed to do:
set the system parameters for the best stable control. Since this is
successfully and frequently done, it must be possible. The problem with
a purely mathematical analysis is that it may show that the system
equations don't have a realizable solution, which discourages the
designer because the math can't answer the question, "Well, then, how
close CAN we get to an exact solution?" The answer usually is "Close
enough for any purposes we can think of."
You:
>>... a "controller" is a (good) controller only in particular
>>environments. Or, equivalently, whether a system is a (good) con-
>>troller depends on the environment in which it finds itself.
Me:
>This is true, but some controllers are able to work without change
>in a wider range of environments than others.
You:
Sure. The technical term is "robust". Some controllers are more
robust than others.
Oh. So you knew all the time that some controllers have a wider range of
control than others. Silly of me to mention it. >:(
The literature (and my own research) suggests that a PID-
controller, for instance (the type that you mostly use is a PI-
controller; the I stands for the integrator that you use in your
output function, the P comes into effect at low frequencies through
the integrator's "leak"), continues to function well if the parame-
ters of the world (delay time, gain, dominant time constant) change
by a factor of at most 2 to 3.
Depends on _which_ parameters change and on your criteria for adequate
performance. If the time constants or delays change, yes you can run
into instabilities. If the gain changes or static nonlinearities appear
(a more common kind of change in the environment), the control system
can handle a far wider range of variations without any substantial
degradation of performance -- at least 10:1 and often larger.
RE: open-loop, closed-loop, and adaptive closed-loop control
the slightest change in any aspect of the
environment or in the transfer function will directly affect the
result, making it different from the desired result; control will
disappear.
Now make this open loop "controller" adaptive, so that it continu-
ously updates the "calibration curve" (or its equivalent for a
dynamic system): control will reappear.
You're jumping the gun; that was my third case. If you have continuous
feedback information available, your best bet is to use a simple
negative feedback control system; that will add robustness, as
illustrated in the example about an operational amplifier.
If you want to compare this op amp diagram with the functioning of
an organism, you should note that the feedback element models the
environment, which will generally not be a simple resistor. It
might contain a delay, an integrator, a varying gain or what have
you. Replace the feedback element by a complex impedance and
calculate when you have a control system: only under severe
restrictions...
Not that severe. My point in using this example was that you can freely
substitute a great variety of feedback elements for the resistor in
order to do different calculations, and the system will still perform
within specifications (typically, 0.1% computing accuracy). While the
error signal at the input might vary over a factor of 20 from one
configuration to another, the gain of the op amp is so large that even
the largest error still does not induce a computing error greater than
0.1% (or to say that differently, the error for a resistive load is so
low, on the order of one part in 10^5, that it can grow 20 to 100 times
before becoming comparable to uncertainties in the values of the passive
components).
Things can be improved, however, if it is within your power to
change the series element into a complex impedance as well, the
characteristics of which you can change. If so, you have
implemented adaptive control.
Yes, exactly. That is the next stage that I would expect, adding
adaptive control to adjust the parameters of a closed-loop negative
feedback control system. That is what the reorganizing system in the PCT
model is supposed to do. But what you are doing is dropping the
continuous negative feedback control aspect, and substituting an
adaptive system to modify an _open-loop_ system. That is why I said your
model seemed like a step backward.
On further study, I discovered that in your model, when set as
appropriate for negligible observing noise, the system effectively
becomes a negative feedback control system (of low loop gain, but that
could be corrected), with no real-time adaptation going on (the world-
model continues to adapt, but it has no effect on the ongoing behavior).
In order to retain the feature of continuing to act reasonably when
feedback is lost, your model requires an external system that detects
the loss of signal and switches the system so it now controls the
internal world model; the real plant is then affected in a way
appropriate to a disturbance-free environment.
... The real point is that this "control system" manages to keep
its controlled variable, the voltage at its negative input,
precisely matching the voltage at the reference input in all these
cases, with no adaptation or other adjustment to compensate for the
radical changes in the series and feedback elements.
Analysis will show you that this is not true. A strict relationship
must exist between the series element and the feedback element if
the diagram above is to represent a control system rather than an
oscillator.
Analysis may show you that, but experience with real analog computers
will tell you otherwise. Of course there are choices of components that
will create an oscillator; that is often done deliberately, for example
to create a tuned filter or a signal generator. But the "strict
relationship" of which you speak is actually a very broad and permissive
condition that allows you to use everything from an inductor through a
resistor through a capacitor as the feedback element, without any change
in the op amp and without exceeding the error limit of 0.1% computing
accuracy. All you are saying is that there are limits; of course there
are. But they are enormously broader than the limits of an open-loop
system.
And it does this with absolutely no advance information about what
voltage waveform will be applied to the input. In fact, that is the
point of this analog computing component: it will compute a
particular function of ANY input waveform, the value of the function
being continuously represented as its output voltage.
This is not true either. It may be that the response to certain
wave forms (say DC) is perfectly all right, whereas the response to
other wave forms (say square waves) shows tremendous over- or
undershoots. Have you ever calibrated the input section of an
oscilloscope?
I used to make my living designing and building electronic systems. Of
course I have calibrated the input section of an oscilloscope. And how
do you do that? You apply a repetitive square-wave signal to the input
(kindly provided inside the scope by Tektronix), adjust the peaking so
you get the fastest rise-time without overshoot, adjust the capacitive
probe compensation to eliminate low-frequency droop, adjust the balance
of the differential amplifier for correct DC zero, and adjust the local
feedback resistor for quantitatively accurate DC response. The
oscilloscope response is then as good as it can be for ANY POSSIBLE
INPUT WAVEFORM from DC to the maximum frequency of response.
An operational amplifier is designed to maintain its zero accurately,
and to have the correct frequency-gain rolloff to work properly with a
wide range of feedback elements. The gain-bandwidth product, as Bill
Leach commented about modern op amps (I haven't used one for over a
decade), can be up to 500 million. The internal compensation is set to
allow capacitive feedback at all frequencies. So the op-amp works in
analog computations with an accuracy of 0.1% or better for resistive and
capacitive loads at frequencies at least up to 100,000 Hz.
You may be able to pull rank on me when it comes to mathematical
analysis, but not when it comes to practical electronics :->
You may want to check how well (white) noise can be "resisted" in
the case of an op amp. I have redrawn your diagram and added the
noise as an extra voltage source in series with the feedback
element. ...The qualification "can resist" ought to be quantified.
:
All the frequency components of the white noise up to about 100,000 Hz
can be resisted (at the input) with an accuracy of 0.1% by the op amp
that Bill Leach described. The input section of an oscilloscope is
essentially a high-gain, ultra-high-speed op-amp with resistive feedback
elements, capable of handling frequencies up to 100 megahertz with 2%
accuracy. If noise were added in the position you indicate, it would be
reduced at the input by a factor of at least 1000 over the frequency
band in which the op amp is designed to work. The output would vary
equally and oppositely to the disturbance fluctuations over that
frequency band. Don't forget that in the op amp, the equivalent of the
controlled variable is the voltage at the negative input, with the
reference signal being the voltage at the positive input. The output
voltage is merely the means of control, the system's action (your u).
I maintain that a disturbance can be the better resisted the better
it can be modelled.
How are you going to "model" a white noise disturbance? The modeled
disturbance would have to be adjusted not only to match the mean, RMS
amplitude, and variance of the real disturbance, but to create
instantaneous fluctuations in exact synchronism with, and of the same
amplitude as, those of the external noise source.
And for that matter, how are you going to model even a low-frequency
disturbance that doesn't follow any analytically-describable waveform
and that doesn't repeat in a predictable way? If I gave you a sample of
the disturbance (of any length you like), you could do a Fourier
analysis and then create a waveform generator that would recreate the
sample, but that would be good only for the sample that has already
occurred; it would not apply to the next sample. Even if I gave you a
sine-wave disturbance, you might be able to adjust a sine-wave generator
to match it, but within a few thousand cycles (or less) your generator
would start to get out of phase with the external sine-wave, and you
would end up adding more disturbances instead of correcting for them.
There is simply no practical way to create a model of an external
disturbance that will work for more than a very brief time.
Mathematically, of course, it's simple: if the external disturbance is
D*sin(1/t), you just write that equation for the modeled disturbance and
the real disturbance will be exactly cancelled forever. But just try to
realize that cancellation in a physical system!
"Noise" is just a convenient category in which we place waveforms that
don't fit any handy analytical form. Basically, noise is just a
variation in some variable. The magnitude of the variable rises and
falls with time. If a noise-disturbance is presented to a control
system, the control system will oppose the resulting variations of the
controlled variable as fast as it can, getting rid of the slow
variations and leaving the ones too fast for it to oppose. Often a
controlled variable will be associated with a mass of some sort, which
by itself will help filter out high-frequency disturbances.
A disturbance is a disturbance, whether we can categorize its behavior
or not. In a control system there is no reason to handle "noise"
differently from any other category of disturbance waveform. Unless, of
course, you think that you have to be able to model a disturbance in
order to oppose its effects.
Since white noise cannot be modelled, it is the base line. If part
of the noise can be modelled and used for control purposes, this
would enable us to place an extra (modelled noise) voltage source
in series with the series element, which partially compensates the
world-noise.
You're dreaming. This is a totally impractical proposal, and totally
unnecessary. Negative feedback control will oppose disturbances up to
the bandwidth limit of the control process, and you will never be able
to do better than that by any means short of magic. Your "modelled
noise" source would never stay in synchronism with the real noise
source; you'd just be adding more noise in quadrature to the noise that
already exists.
Bill Leach says:
Without trying to quote some more of your text... are you sure that
you could design a model based controller for one of the simpler PCT
tests and obtain similar results (including the sorts of changes
made to the typical experiment)?
And you say:
Yes. Regrettably (for Bill Powers) the very exact specifications
that would be required to establish the QUALITY of control would
result in an equally exact model specification.
Ok, put your model where your mouth is. I don't believe you can do this.
It is extremely simple to set up an experiment with a human subject and
then fit a control-system model to it that will behave the same way the
human subject does, within 5% RMS of the handle movements, for the
disturbance used during the experiment AND ALSO FOR ANY SUBSEQUENT
RANDOMLY-GENERATED DISTURBANCE.
Our specifications for quality of control are very simple: the subject's
handle movements should match those of the model within 5% RMS of the
range of handle movements, when you lay one trace over the other. The
requirement is simply that the model should behave the same way the real
person does.
I should point out that we generate tables of random disturbances which
are unknown beforehand both to the person and to the model. If you try
to meet this challenge, you will have to do it under the same
conditions. Perhaps you can figure out a way for your model to derive
the form of the disturbance from the same information that is available
to the subject: the behavior of the cursor and target on the screen. If
so, fine, do it. I don't think you can do it.
Want me to send you a program for carrying out the experiment?
ยทยทยท
----------------------------------
... since optimal controllers directly tune their parameters in
such a fashion that optimal control (i.e. control that could not be
better given the control criterium and the model) results, no other
controller could be better.
What you're saying is just a tautology: if there were an optimal
controller, by definition it would adjust the parameters of control
(including its own parameters) in the best possible way. But this by no
means shows that you have found any optimal controllers; to demonstrate
that, you would have to show that no other form of controller could do
any better. You're talking in terms of a _particular design_ for a
controller, and of course for any particular design there is a set of
parameters that makes it work better than for any other set of
parameters. But that does not mean that this design works better than
any other possible design would work if also adjusted optimally.
Perhaps what leads you to speak in absolutes is the fact that your way
of designing a world-model and a controller for it produces performance
that is not just "optimal" -- it is _perfect_. What could be more
optimal than a system which makes x = xopt, not just nearly but with
infinite exactness, in a single iteration, no matter how short a
physical time is represented by one iteration?
But this very perfection should make us suspicious; the system you are
modeling doesn't have any physical properties, and neither does the
world-model or the control system. The control system can act in no
time; the external system can react to inputs in no time. One begins to
suspect that what you have is not a system model, but only an algebraic
identity. We get x = xopt because there is nothing else we can get when
the equations for the world model and those for the controller are
combined. The world-model plus controller make x = xopt for any values
of parameters whatever, provided that there is no division by zero
involved. So adjusting the parameters makes no difference in how the
world-model works. That is rather peculiar. If x = xopt for any values
of the parameters, why does not xt = xopt for any values of the
parameters? Why do the parameters make a difference in the response of
xt to u, when they do not make any difference in the response of x to u?
Somewhere, hidden in your equations and manipulations, there is the
equivalent of dividing by zero.
If the real external system is a physical system, then it can't behave
the way the world-model does. The length of time represented by one
iteration makes a difference; if that time is shortened, the physical
variables can change correspondingly less in one iteration. This basic
physical fact is not represented in the world-model. If it were
represented, you would no longer get x = xopt in one iteration; you
would find that the difference, xopt - x, would depend on the
parameters, the passage of time, fluctuations in the disturbance, and
fluctuations in the reference signal.
So your model is not an "optimal" model; it is an _imaginary_ model,
working in a massless, forceless, timeless universe. This is why I am
confident that you will not be able to find any form of this model that
will fit the behavior of a real person. In the real person, xt does NOT
equal xopt, so in any world model, x had better not equal xopt, either.
------------------------------------
Note that the value of pvv thus acts as a switch: pvv = 0 means
"control the actual value of xt" and pvv > 0 means "control the
output of the world model." This switch is actuated by an outside
intelligence.
This is the switch that toggles what you call the "imagination con-
nection". It is, however, a "soft switch" so that control can be
PARTIALLY based on both internal prediction and external
observation. That is how we humans often operate, I think: we have
a course inner model and use our observations only to fill in the
moment-to-moment details.
Yes. This is why I am still interested in the adaptive system with a
world-model, despite its serious defects. There must be some way to
combine the best features of the world-model system and the high-gain
negative feedback control system, so the system can resist disturbances
accurately without having to predict or model them, and so it can
operate with incomplete input information.
A hierarchical model may help a lot, because as I showed in a previous
post, a complex system can be broken down into a hierarchical system in
which each control system is very simple and handles only one simple
variable. This means that the "world model" used in any one control
system can also be extremely simple; only when you look at the behavior
of many systems of this kind in a hierarchical array do you see a result
that looks complex. I think that the hierarchical approach greatly
simplifies all control problems, including problems of adaptive control.
Each control system, after all, has only a simple task in a hierarchical
model: to make its perceptual signal match the reference signal it is
given, in one dimension. A proper world-model, then, only has to provide
a single signal of the appropriate magnitude, for any one controlling
subsystem. The model of the whole external world does not exist in any
one control system; it is distributed over all of them.
------------------------------------
RE: the model
4. Independent disturbances applied to both y and xt. All relevant
parameters manually set to agree with disturbance amplitudes.Now the control system makes x = xopt, and xt = xopt + (disturbance
of xt). There is no resistance to disturbances of xt. The model
output x follows the reference signal xopt, but the real system
output does not.
This is the most complex situation: the model attempts to
discriminate between what is "real" (in the outside world) and what
is due to bad sensors (inner defects). Sometimes such a
discrimination is not possible, e.g. when both disturbances covary.
Correction: the model does not attempt to make this discrimination. It
must be told when there is a direct disturbance of y. If it is not told
this, it will treat those disturbances just as it treats disturbances of
xt, and produce an output that compensates for the combined effects of
the two disturbances. It will then create fluctuations in xt that are
equal and opposite to fluctuations in the disturbance of y, _causing_
rather than correcting errors in the real xt.
This is essentially W. Ross Ashby's concept of a regulator,
described in _Design for a Brain_.
Thanks for the reference. I'll look him up.
Ashby, W. R. (1952); _Design for a brain_. New York, John Wiley & Sons.
Actually a more directly relevant reference is
Ashby, W. R. (1956); _An introduction to cybernetics_. New York, Wiley.
See especially the section on "Control" starting on page 213.
---------------------------------------
RE: hierarchy again
That is the other discussion: if the "world" changes significantly,
the controller must change significantly as well, i.e. it must be
adaptive.
In a hierarchical model, the lowest-level variables under control are in
an environment the least likely to change in significant ways. The
control loops are short and the variables are dealt with in isolation
from other variables. There are few interactions, because effects of
other variables are treated as disturbances and essentially cancelled.
So the lowest level of control creates a set of perceptual variables
that are adjustable simply by changing reference signals, and only
rarely would there be any occasion for adaptation of a serious nature
(some simple adaptations like gain changes can be handled locally).
Each added level both creates more abstract variables to control, and
simplifies and regularizes the world with which still higher systems
must interact. The latitude provided by negative feedback makes many
kinds of changes in the world invisible to higher systems, unimportant
in their operation. So the hierarchical architecture has the happy
effect of minimizing the demands on adaptive processes that work at any
given level. This is especially true in contrast to a system in which
the entire control process has to be carried out by a single highly
complex one-level system, in which adaptation may require the
simultaneous adjustment of large numbers of parameters -- all in the
right direction. A difficult problem in adaptation for a single-level
system may well turn out to become a modest set of simple adaptations in
a hierarchical system.
----------------------------------------------------------------------
Best,
Bill P.