------------------- ASCII.ASC follows --------------------
[Hans Blom, 950529]
(Rick Marken (950524.1000))
... Hans' program exhibits no "model-based" control at all
(Rick Marken (950524.2145))
... but I was also wrong about Hans' model -- it does control
Thank you. So does it or doesn't it? Let's see...
... Hans' model does control but it controls poorly. Most important,
what control this system does exhibit is a result of control of the
perceptual representation of xt.
The "poorly" needs to be better defined. And, of course, all control
ultimately depends on information that is somehow available about "the
world out there".
What is new in my demo is, in comparison to the "standard" PCT-control-
lers:
1) an internal model of the regularities of "the world out there"
can be built, so that for instance a square wave pattern
reference level can be accurately tracked with essentially no
error and zero response delay;
2) this internal model is also available to control (Bill Powers:
"pseudo-control") in periods where no sensory information at
all is available;
3) this internal model may become so trustworthy that it is more
relied on than the perceptions; this, as well as 2) above, was
my counter-example where the generalization "control of per-
ception" breaks down and must be replaced by "control of (by?)
internal representations".
Nothing of this is new, of course. 1) Tracking a regular ("predictable")
pattern with approximately zero delay has frequently been encountered in
human operator response speed studies. 2) An example: because humans have
forward looking eyes, the complete visual field cannot be surveyed at any
single moment of time. Yet, especially in inimical environments, it is
important to always have an accurate "map"/model of what goes on behind
our backs. This "map" must be periodically updated, but after each update
it may serve us for some time -- or not, if not updated soon enough. 3)
An example: we have learned to rely on Newton's (or Einstein's) laws more
than on our inaccurate perceptions re positions/speeds/accelerations of
falling bodies. In fact, Newton himself trusted "regularities" more than
his (rather poor) observations; curve fitting isn't new...
(Bill Powers (950525.0530 MDT)) replying to Rick Marken (950524.2145)
One problem is that when you set the reference signal to a constant
value, the Kalman filter has no basis for calculating partial
derivatives and can't adapt properly until it has has some experience
with the faster disturbance.
You touch a crucial point: learning occurs only when there is error.
Error has to be introduced somehow (e.g. in system noise/"unmodelled
dynamics" or in a varying reference level) for learning to be able to
occur.
I believe the net result is that hidden inside this adaptive model with
a Kalman filter is an ordinary control system that works through the
external world in the usual way once adaptation is complete.
That depends upon what you consider an "ordinary" control system. In the
demo, control works as follows. When adaptation is complete, a=at, b=bt,
c=ct, and pax=0. The "world" acts according to
xt [k+1] = ct + at * xt [k] + bt * u [k] + system noise
The model offers the prediction
x [k+1] = ct + at * x [k] + bt * u [k]
The system noise, being zero on average, is disregarded. Set x [k+1] =
xopt, specifying that we want x [k+1] to be at xopt in one step. Remember
also that model variable x [k] is known; it is our best estimate thus far
of the "world" variable xt. In fact, if the observation is noise-free, y
= xt = x. The first (y = xt) because the observation is noise-free, the
second (x = y) because it is KNOWN to the model that the observation is
noise-free.
xopt = ct + at * x [k] + bt * u [k]
Now solve for u [k]:
u [k] = (xopt - ct - at * x [k]) / bt
= (xopt - ct - at * y [k]) / bt,
the latter line for noise-free observations. This is what the control
routine, in all its simplicity, does.
The primary difference from the simple control system is that if
present-time inputs are lost, the controller can continue to produce the
same pattern of outputs and thus continue to drive the external system
in the appropriate way, as long as there are no independent disturbances
and the external parameters remain the same.
In the above control algorithm, no direct reference is made to the
observation y. Whenever it is missing, we can still generate new values
for x [k+1], x [k+2], etc, using the prediction equation
x [i+1] = c + a * x [i] + b * u [i] for any i
and we can still solve for u [i]. Thus it is the internal model that
generates "pseudo"-observations x [i] that serve as a basis for control.
The quality of the control depends on a) the fact that some of the
world's dynamics may not be modelled and hence cannot be predicted, and
2) the accuracy of the current estimates of the parameters a, b and c.
Under a loose definition of the word "control", the system can be said
to continue controlling without present-time sensory feedback. It does
not continue controlling under the meaning of control that we use in
PCT, where an essential part of the definition is resistance to
disturbances.
If you define "control" as equivalent to "control of perceptions" or,
equivalently, to resistance to disturbances (which can of course only be
resisted if they are observable, then my demo does not "control". It also
controls when there are no perceptions or when perceptions are corrupted
by random or "arbitrary" variations. So, given my "loose" definition of
the word "control" (which I take from the control literature), can we
agree that we indeed have found a counter-example of "control of percept-
ions" that functions well under some (albeit maybe severely restricted
circumstances?
We can now see that the primary effect of Hans' model is to enable the
controlling system to continue the same pattern of output variations in
the absence of direct sensory feedback from the real controlled
variable.
Not just in the complete ABSENCE of direct sensory feedback; also when
direct sensory feedback is inferior in quality compared to the feedback
provided by the internal model.
The same result can, I believe, be achieved by a hierarchical control
system.
But this statement is in direct conflict with your axiom that control is
"control of perceptions". In my opinion, and as the demo shows, control
should not be "control of perceptions" when those perceptions are
untrustworthy.
Maybe we can salvage the term "control of perception" if we take "in-
ternal observations" of model variables such as x to be "internal" per-
ceptions. This may not be so funny as it sounds. In order to know about
the world, we consult both our (momentary) perceptions of that world AND
our accumulated knowledge about the world. What else would we have memory
for?
However, before we put too much emphasis on this ability to operate
blind we should ask whether it has an important role in explaining human
behavior (which is, after all, the primary purpose of PCT).
Again, it is not only the ability to operate blind that I find interest-
ing. It is the ability to somehow store knowledge about the world that we
can use in our control tasks IN ADDITION TO our direct perceptions.
It is one thing to design an adaptive control system; it is another to
show that it is a model of real human behavior.
I consider it superfluous to demonstrate that (real-time) adaptation/
learning exists in organisms with a nervous system. It remains to be
shown HOW adaptation works in such organisms. I maintain that all kinds
of (real-time) adaptation have common underlying features, and that my
demo shows some of those, be it in a very crude way. That brings us to a
more philosophical point:
There is a tendency in computer-science circles to design a system that
operates in some clever way, and then look around for behaviors that can
be interpreted (and often overinterpreted) as fitting it. This is the
wrong way around. We must start with observations of real human behavior
in circumstances where we can measure accurately what is going on, and
then search for a model, clever or simple, that will reproduce the
behavior under all reasonable variations in circumstances that we can
think of.
This tendency exists not only in computer-science circles (which, by the
way, I do not consider myself to belong to), but in all of science. There
simply is no other way. I have accepted by now that "observations" are
always model-driven: there is so very, very much to observe that we must
set limits to what we WILL observe, what we will describe (in language or
in formula), what we deem important and what not. And what we can observe
is restricted, as you note, by what we can accurately measure, i.e. in
terms of what is already known to us. This embeds observations within a
culture: the type of observations that we can make will always depend on
whichever instruments we have and on our pre-existing knowledge of what
is observation-worthy.
Science, whether we like it or not, is mostly hypothesis- or theorem-
driven: someone invents a theory about how things relate (in terms of
measurables and ought-to-be measurables) and then must proceed to
demonstrate that this particular theory works well. Theoretical physics
is, I think, much more a driving force than observational physics. It is
a stroke of genius to invent "invisible" concepts like "acceleration",
"black hole" or "continental drift". We have no inkling yet, I think, in
which terms to describe what we intuitively feel to be the most important
aspects of "real human behavior".
(Rick Marken (950525.0900)) to Bill Powers (950525.0530 MDT)
... as Hans himself noted in his post, there is no control of xt at all
when the model is blind (y is essentially all noise).
I have noted no such thing. When the model is blind, there is control of
x (see above: the controller attempts to bring x toward xopt). And inas-
much as x is an accurate representation of xt, xt is controlled as well.
If you define "control" as being identical with "control of perception",
there can of course be no control if there is no perception. But you must
realize that you then use the term "control" in a very idiosyncratic way,
which, moreover, makes the P in the title BCP tautological, to say the
least. So what do you mean when you say:
... So Hans' model can control. But it controls only when it has
available a perceptual representation (y) of the controlled variable:
it can't control blind.
One thing that my demo was meant to show is that there may be an
"internal equivalent" x to the perception y that can be used when y is
missing, and that, as long as the internal representation is accurate,
control can proceed even without observations. So is this control or not?
We seem to have a terminology problem. Perhaps Bill's "pseudo-control" is
a good idea. Then we can rephrase
So I was wrong to conclude that Hans' model cannot control. But that
mistake occurred because I assumed that Hans was saying that his model
could control xt without perceiving it.
into something like
Hans' model can pseudo-control but not control xt without perceiving it.
How is that?
But there is something more where I said
... "control of perception" does not apply to model-based control if
the perception is disturbed and the model adjustment algorithm is
given the information that it is.
The "perception" is y = xt + noise/disturbance vt. I propose that it
would be better to attempt to control xt than y. This can be done by a
'filter' operation. This 'filter' is based upon the information that is
accumulated by somehow "processing" observations and fitting them into
the internal model, which can subsequently filter out (some of) the
noise. Of course, xt is not directly accessible, but x might be xt's
accurate internal representation.
... It looks like there is control of a real world variable (xt) while
the perception (y) of that variable is ignored.
Yes, LOOKS LIKE phrases it well. Of course it is x that is controlled and
not xt. But x might represent xt better than y does.
The missing Post from Bill Powers (950523.2115 MDT), that came to us by
way of Rick Marken (950525.1500) [thanks, Rick!] deserves more thought
than I have time for at this moment. I'll come back to it later.
Greetings,
Hans