[From Bruce Abbott (96019.1245 EST)]
Hans Blom, 960718 --
Hans, it's been a while since you presented your adaptive model-based
controller and discussed its workings, and my memory of it has become a bit
fuzzy. If you wouldn't mind, I wonder if you would remind me about some
details. What, precisely, does the controller _know_ or _learn_ about the
World in which it operates? Does it "know" or learn the effect of its own
actions on the CEV?
Regards,
Bruce
[Hans Blom, 960722]
(Bruce Abbott (96019.1245 EST))
Hans, it's been a while since you presented your adaptive model-based
controller and discussed its workings, and my memory of it has become a bit
fuzzy. If you wouldn't mind, I wonder if you would remind me about some
details. What, precisely, does the controller _know_ or _learn_ about the
World in which it operates? Does it "know" or learn the effect of its own
actions on the CEV?
It LEARNS how its actions affect its perceptions. Thus, it discovers
a function p = f (a), where we call f a WORLD MODEL. The general
method is to cross-correlate actions and perceptions. It is this world
model f that becomes KNOWN. But only to some precision, because we
will generally find that f is not 100% accurate. This can be
expressed as p = f (a) + n, where n stands for that part of the
perception that cannot be "explained" by the action. This is the part
of the model that is important for control. A complete model must
also include perceptions c that are not influenced by our actions,
such as the sun rising and setting. This expands the model to
p = c + f (a) + n
The term c can, of course, be discovered by not acting (a = 0) or by
cross-correlation. In the latter case, c "explains" everything that
is not due to a. This seems to be one area where people have
problems. As any psychotherapist can tell, it is frequently difficult
(not only for patients) to distinguish between what _I_ caused
(through action a), what is the result of the action of the laws of
nature as we understand them (c) and what is unpredictable and
unreproducible (n). Artificial systems have these same problems, of
course, due to haphazard correlations that will always exist between
p and actions a, that do NOT contribute to p (accidentally synchronous
events rather than cause-effect relationships). An example is praying
for rain. It is inevitable THAT rain will come (it belongs to c), it
is more or less random WHEN it comes (it contributes to n), but we
might THINK it is caused by a.
Theory shows that this problem is due to insufficient averaging,
which may be aggravated by differential averaging for positive and
negative instances (biased perception).
How to act, once a model like this has been discovered? Well, the
model says that
f (a) = p - c - n
so, given some desired perception p, we can "compute" the action a by
inverting f and plugging in p, c, and n. Since n is unpredictable
("noise"), the computed action will generally not take p all the way
to its desired value immediately; there is some "action uncertainty"
depending on how large n is and thus how good the model f.
If we are aware of this action uncertainty, we will know that our
actions cannot result in perfect results if our model is inaccurate.
If we think that our model is more accurate than it really is, we
will often be surprised. Usually, this surprise takes the form of
disappointment (in our predictive powers, if you analyze this more
fully), but sometimes also of elation or the experience of "grace"
(more is possible than I could ever have predicted). In either case,
we ought to enlarge our estimate of the influence of n and decrease
our estimate of the accuracy of f.
If we go all the way in this, we "know" that n explains (almost)
"everything" and f (almost) "nothing". This realization is the
foundation of wisdom, as expressed in "the more I know, the less I
know I know". It is the realization that our models cannot be
anything but coarse approximations. But this is "high level"
knowledge that is not easily acquired. Theory shows this as well.
Only if we have good models of c and f can we start to get to know
something about the properties of n.
The Kalman filter is an approach to generating the knowledge, that is
the terms c, f (*) and (the statistical properties of) n. A separate
approach (orthogonal, but interacting) is required for control.
A nice introduction to the mechanics of the Kalman approach of
solving this problem can be found at
http://www.cs.brown.edu/stc/education/course95-96/Kalman-Filters/
kalman.html
which is also rapidly found when you use the Alta Vista web browser
searching for "Discrete Kalman Filter".
By the way, a nice description of the Inverted Pendulum (balancing a
stick on the palm of your hand) can be found at
http://robocop.anu.edu.au:80/docs/InvPend/thesis.html
Greetings,
Hans