------------------- ASCII.ASC follows --------------------
[Hans Blom, 950601]
(Bill Powers (950531.0845 MDT))
Something finally sank in. ... How simple!
You got it! Once you truly understand something it IS simple,
although to others it may be abacadabra...
My way of modeling a simple control system must have puzzled you
greatly, because customarily I have represented the environmental
feedback connection as a simple constant, without in any way
indicating what approximation is hidden by this representation.
No, I'm familiar with your way of modeling. It is the classical,
basic approach to control, and it demonstrates that control can work
well even if our assumptions about the "world" are inaccurate --
which my demo shows as well. Part of what I wanted to demonstrate is
that if our knowledge (world-model) about how we can influence what
is out there improves, control can become even better. One aspect of
that is that sensory information that does not contribute to better
control (observation "noise") can and should be disregarded. The
classical approach is not able to do that; Kalman Filtering can,
because it incorporates learning of what part of the observation is
significant (can be used in control) and what not (what can and must
be disregarded).
An aside re influence vs. control, a discussion topic of sometime ago
in which I did not participate: I think that a discrimination between
the two is useless. In a model-based control approach, we can only
"influence" if there are unmodelled dynamics that "influence" the
state of the world as well. We can "control" (in the sense of fully
determine) only if the map fully coincides with the territory --
which can never be the case except in artificial worlds that we built
ourselves and thus known up to the smallest detail.
Note that the loop gain of this system is Kf/Kf = 1.0.
The notion "loop gain" is not applicable here. It is an important
notion in the basic, classical approach, but not here. See the ANOVA
explanation discussion below.
There is only one problem: we must know d in advance in order to
compute the correct o to cancel it, and build that value of d into
the equation of the controller.
You got it! Replace "know d in advance" by "have an internal model of
d" and you're there. For as far as d is not truly random, an internal
model CAN be built (see below), so the system will have at least
partial advance knowledge. The truly random part is of course unpre-
dictable/unanticipatable (is that a word?), but, BECAUSE it is truly
random, it will average out and for that reason can be disregarded.
Unless, of course, the amplitude of the truly random part of d is so
large that no good control is possible at all, whatever control
method one attempts to use.
If d is a function of time, d(t), then we must build d(t) into the
controller and time it so it is synchronized with the external d(t)
acting on qi.
You got it. That is part of what in the demo is called "prediction".
But prediction is possible only in so far as we can build a model of
d. The unpredictable part of d cannot be modelled and cannot be con-
trolled.
So we might easily come to the conclusion that a control system
can't resist disturbances that are not represented in the equation
of the control system. This seems to be your claim, Hans, and what
your professor from long ago taught you.
Yes. It correlates with the everyday observation that one cannot use
tools that one is not familiar with -- that one doesn't even recog-
nize as tools.
This explains a peculiarity in your program, the way the function
for computing the random noise variable is organized. This function
returns a new value of the random noise only on every second call;
Meta-remark: see how easy it is to come to an incorrect explanation
cq. world-model? It is a miracle that convergence to the correct
values sometimes occurs at all!
If the environmental feedback function is something other than a
constant of proportionality, then we can't necessarily use the same
form of the controller function, because of the system dynamics.
Exactly: the controller function depends on the system dynamics. That
is what I have attempted to get across for some time now, in diffe-
rent words: a "controller" is a (good) controller only in particular
environments. Or, equivalently, whether a system is a (good) con-
troller depends on the environment in which it finds itself.
The trick that Kalman Filtering attempts -- and often succeeds in --
is to get to know / learn / track the system dynamics in order to
adjust its controller function so that good, or even "optimal"
control results. Even if the environment is initially unknown. Even
if the characteristics of the environment change.
Whereas in PCT models YOU have to plug in gains, integration time
constants and such, in my demo equivalents of this are done automa-
tically.
That is probably more fooling around with Hans' model than anyone
wanted to read.
Bill, I was greatly, and very pleasantly, surprised to discover the
depth toward which you have analyzed what goes on in my demo. Few
people over thirty are still willing to learn complex new theories.
Inquisitive minds want to know. Yours sure is!
(Bill Powers (950531.1540 MDT))
... This bothers me. Why should an adaptive system need a friendly
external helper to tell it what variances to expect? If this is
intended as a model of an organism, there is certainly no such
friendly helper.
Why necessarily an "external" helper? It might be an INTERNAL helper,
one that we are genetically endowed with, or it might be a different
internal system, either at a higher hierarchical level or running in
parallel. The "world" and the model of my demo were very, very limit-
ed -- yet still complex enough to be almost incomprehensible to most
-- and I have only given hints that procedures are known to build
INTERNAL models of these variances.
I am not willing to go into all the details -- my discussions have
lost their interest for most readers of the list already. But to use
an analogy, although very different in detail: Compare the "extern-
al" information that is required about the noise variances with the
"external" information in the form of highest level reference level
settings that your PCT controllers need.
Considering the critical effects of these parameters, it seems
rather an anticlimax to find that the adaptive control system can't
find the right values itself.
No. As I said:
It is, however, possible to design methods to estimate the proper
values of pvv and pff if these cannot be pre-specified.
You're talking about an engineered system, not a living system.
Neither. I am talking about a feasibility proof: can it be done at
all? Therefore I engineer an (admittedly tiny) system and demonstrate
that an accurate world-model can be built based only on the informa-
tion that is obtained from observations (y) and actions (u), with
everything else internal "estimates"/guesses. And, that such a system
can learn to control from scratch (as a baby does), and can relearn
to control after major unpredictable changes in the world (as after
any type of life crisis). And, that it is not necessary for the
internal estimates to be "correct" (in the objective sense, i.e.
at=a, etc.) for control to be of good quality (as in neurosis). And,
that is possible to discover which part of our observations is useful
for control purposes and which part can be neglected.
Thus, besides "irrelevant side effects" I now introduce the notion
"irrelevant observations" (note: irrelevant only with respect to a
well defined goal or set of goals).
If we know that model-based control CAN be done, we can start looking
for its consequences in living systems. Evolution MIGHT have disco-
vered the same or similar mechanisms in its "survival of the fit-
test", which MIGHT coincide with an inexorable progression toward
living control systems that are ever better able to adjust their
"controller function" to their environment, i.e. better exploit their
niche.
It may help to know that a model-based controller will achieve con-
trol of a better quality -- in the mean square sense -- than your
classical type of controller, given an accurate model. That is why I
will not take you up on your offer to compare the quality of control-
lers designed by you and by me. Necessarily, your description of the
operating conditions of the controller (the "world" in which it
lives) would provide such an excellent model that you are bound to
loose such a contest :-).
It is difficult to get to understand Extended Kalman Filter theory
from playing with a demonstration, especially when you are
convinced that you can explain everything in terms of a different
theory (PCT versus SR, for instance :-).
I thought I was doing a pretty good job of trying to understand your
program, although I admit that my lack of familiarity with the
mathematical techniques is a handicap.
Yes, it seems that things suddenly snap into place. Your exclamation
"How simple!" might be a sign that in certain areas convergence has
taken place and "certainty" has been reached.
Basically, it is an on-line multiple regression estimation method,
very much like the ANOVA that psychologists are familiar with.
That isn't much help to me, as I have never used such methods.
OK, a basic explanation with an almost trivially simple example.
(Don't accuse me of being simplistic here; I know I am). This
procedure is for a single input single output system without
dynamics.
Assume that you have a set of pairs of observations y and actions u.
Plot those pairs:
> *
^ | * *
> > * *
> > * * *
y | * *
> * *
> * *
> *
>
>-------------------------
u ---->
Now fit a best straight line through those points. This line gives
you two parameters, offset and slope. That is the first step. Now
plot the residuals: the deviations of the points from the line of
best fit. The quality of the fit can be described in terms of the sum
of squares of the residuals. If this sum is zero, you have a perfect
fit. If the sum is small enough to your taste, you might be satisfied
at this point. If not, continue with fitting curves through the
residuals.
Obviously, a best straight line fit through the residuals would yield
a line with zero offset and zero slope; that doesn't help. But a
different function might be fitted: a square law curve, a parabola, a
sine, or what have you. If so, you have discovered a better fit with
more parameters. Plot the new residuals again and check whether they
are small enough. If not, iterate.
What results is a model y = f (u), maybe something like
y = a + b * u + c * u * u
-1
For control purposes, we need u = f (y), i.e. a method to compute
some u given a prescribed y. Inverting f isn't really necessary: the
fit may also be performed directly on u in terms of y rather than on
y in terms of u, yielding maybe something like
u = d + e * y + f * sqrt (y)
Now imagine this same process in more dimensions -- y and u multi-
dimensional. If necessary, one can introduce "hidden variables" if
the system has dynamics, e.g. when there is a relation between y[i+1]
and y[i] (from here on, psychologists are usually flabbergasted). The
goals are the same, however: to discover a model y = f (u) and, for
control, the inverse of f.
Now the problem of searching for a best f is unconstrained and unim-
plementable for the general case. That changes when only a finite
subset of fitting functions is considered. Usually straight lines
suffice. If so, a particularly simple fitting procedure can be
implemented, something well within the complexity bounds of the
central nervous system.
That was ANOVA, much simplified. And please don't ask me about the
"hidden variables" thing... way too complex for a discussion list.
ft stands for everything that we cannot (or do not want to) model
and is, lacking more information, assumed to be best represented as
normally distributed zero average white noise ...
This is my main hangup. It seems that you have given up on actually
counteracting the disturbances in detail. If you represent the
disturance as a normally distributed zero average white noise, you
are not going to be modelling the actual perturbations that a real
environment induces in the controlled variable, and so will not be
able to build a system that can counteract them.
In my simplified ANOVA description, no more parameters can be extrac-
ted from the residuals as soon as the residuals resemble white noise.
As long as the residuals have more structure, a better model can be
built. I have indeed given up on actually counteracting white noise
disturbances, because that just is physically impossible in a causal
system (a system that cannot know the future).
The assumed disturbance is very different from real disturbances
encountered in a real world. It almost seems that you believe there
is no way to counteract unpredicted disturbances.
We are talking at cross purposes again: "real" disturbances that are
not white noise can be modelled (see the above procedure) until the
residuals ARE white noise, which cannot be modelled anymore.
Or are you speaking ONLY of the special case of a controller that
relies on an internal world-model, and realize that there are other
kinds of controller that don't need to make this approximation?
No, I am speaking in general. No controller, whatever the design, can
control away white SYSTEM noise (whereas white OBSERVATION noise can
be adequately rejected). But as soon as the noise shows more struc-
ture, that structure can be modelled and controlled away. In your PCT
models, you usually rely on such structure, often imposed by demand-
ing that the disturbances be low frequency. If the disturbance is low
frequency compared to the sampling rate/control interval, one partial
fit is obvious already: d [k+1] = d [k] doesn't "explain" everything,
but it IS a fine PARTIAL fit. An even better fit might be a linear
extrapolation of the line connecting the previous two values. Well,
you get the idea.
Basic in all these considerations is that you cannot know more than
what you have observed. The basic material for the model is the set
of y, u points. You can fully exhaust all information that is con-
tained in those data until only white noise residuals remain. BUT NO
MORE. Thus a variant of "it's all perception": "its all perception
and action".
In a real control system, the computed output is not just what is
required to bring the controlled variable to its reference level.
You are confusing "real" control systems with the type of control
system that you are familiar with (meta-remark: convergence should
not occur before all data are in). Let me assure you that there exist
REAL Kalman Kilter based control systems!
Well, this tells me that your model can work only in carefully
selected environments. That rather restricts its usefulness.
I keep trying to explain that EVERY controller is a controller only
in "carefully selected environments". That truly rather restricts the
usefulness of ANY "controller". You say essentially the same thing:
If the environmental feedback function is something other than a
constant of proportionality, then we can't necessarily use the same
form of the controller function, because of the system dynamics.
Can we finally agree on this?
Have you taken into account that a slow phenomenon can only be
learned slowly as well?
Have you determined that this is the correct explanation?
Yes. Let me explain. Assume that the disturbance is a very slow sine
wave that starts at zero. If you observe this disturbance over a
short time only, you will find an almost perfect fit with a straight
line segment. If convergence is too rapid, i.e. reached within this
short observation time interval, you have learned an incorrect model.
Given even the tiniest bit of random observation noise, there is no
way in which anybody (or any system) can establish that the disturb-
ance IS GOING TO BE a sinewave. The best prediction is the extra-
polation of the straight line. If the observation noise is finite,
this implies that the "true" world-variable will go to infinity.
Wrong conclusion! Only if convergence is forced to be slow enough
(based on enough data points) will it be possible to establish that
the disturbance is not correlated with any parameter that is useful
for control.
... If your system requires all disturbances to average to zero, you
are not designing a system for the real world.
You aren't getting this yet. A non-zero-average disturbance component
is EASY to model and EASY to control. In my u-y diagram above, just
shift all dots one position up or down. No problem: one offset para-
meter will have a different value but the model is not essentially
different.
Are you sure that you observed the rules concerning ft, pff, vt,
and pvv?
That was my point; no, I didn't.
Apply a theory to a domain where it isn't applicable and you get
funny results...
Can you explain why it is necessary to tell the model what
magnitude of disturbance to expect?
The model assumes TWO types of disturbances: one type (the
observation noise) must be disregarded, the other type (the
not yet modelled dynamics) is used to (further) tune the model
parameters. Very different things, which must be discriminated
between.
But the model isn't discriminating between them; the programmer
initializing the model is [by specifying pff and pvv].
That is true for my simple demo. Again, there are methods... Let's
not try conflagrate all hierarchical levels, before we even have the
basics right.
A hint: focus your attention on the two types of noise/uncertainty
and their very different natures; difficult but most enlightening.
Greetings,
Hans