feedback / feedforward

[Hans Blom, 931108]

Glad to see that "feedforward control" is of some interest to some people.
The discussion contained some very valuable remarks and may be the start
of something new. Exciting!

(Rick Marken (931104.1100))

                      The conditions under which feedforward
works are VERY RARE in real behavior.

That you perceive this may be due to your never having looked for the
phenomenon.

          note that this implies that people control for using
a particular type of "control"; people use feedforward "control"
only while they are percieving "well-known patterns" and feedback
"control" otherwise.

Yes, I thought I said something like that. You must add, however, that
"people MUST use feedforward "control" while they are not perceiving any
feedback related to the task at hand".

This was the point of my question; even if there is feedforward
"control" of some behaviors (and I think there is -- such as the
ballistic movement of the eyeball) feedforward must always PART of
a feedback control loop. It just has to be; there is nobody else
"watching" what is happening besides the behaving system itself so
there is no one around to "tune up" the feedforward system when it
is no longer producing the "right" results.

Yes, tuning (learning) is undoubtedly a feedback process.

(Tom Bourbon [931104.1345])

Each of us [Rick, Bill, Tom] has ... said ballistic actions would work
in a disturbance-free world. The problem is, we don't live in that kind
of world. Pre-planned ballistic actions do indeed produce reliable
results in the clean rooms where wafers are manufactured, but that is not
our home and robots do that work.

Conceptually, we can divide the world into a predictable and an unpredict-
able (noisy) part. Every modeller knows that and builds his models like
that. We can find out about the predictable part. That is science, and
that is learning as it goes on in humans. Without any predictability, no
control -- feedback or feedforward -- is possible. My only thesis at this
point is that some aspects of the world are so reliable that feedforward
control might be possible. This thesis is supported by the fact that we
know some of the laws of nature to an extremely high accuracy. In some
cases, it is wise to rely more on those laws than on our more disturbance-
prone observations of what goes on. One of the limitations, in my opinion,
of current CSG is that it assumes a fully error-free input function, i.e.
a perfect observer. There are many cases where observations are fuzzy,
inaccurate, or plainly in error. Have you ever visited a court session
where a number of witnesses had to provide testimony about what exactly
happened in a traffic accident?

But that is not the only issue. In cases where the feedback information is
either incomplete or inaccurate (noisy), an internal model might provide
some of the information that would otherwise be lacking. Think of this in
statistical terms: you have TWO sources of knowledge or measurements, both
inaccurate. One is internal (your accumulated experience), the other is
external (your new perception). The combination of both provides a better
estimate than either alone, i.e. a smaller standard deviation. That is
also the procedure that a judge applies upon hearing all the testimonies
in a traffic accident case -- ideally.

Also, you did not address Bill's distinction between continuous and
sampled control. Many of the actions you describe as ballistic are nice
examples of sampled control.

Sampled control does not alleviate the problem in any way. Missing samples
are just as bothersome. In fact, whenever a sampled control system is
designed in such a way that it can tolerate missing samples (not all are),
an internal mechanism must be available that "predicts" what those missing
samples ought to have been. Some methods naturally have such a mechanism
as part of their method, so that nothing extra needs to be built in. The
Kalman filter is such a method.

Right. But once it has learned, it might not need feedback anymore, or
only occasionally, or only under special circumstances.

Tom: Occasionally, as in moment to moment. Special conditions, as in
while living in a variable body in this variable world.

You postulate that tuning (learning) goes on all the time. I am not so
sure about that; for me, this is a subject open for discussion.

Finding optimal solutions is indeed difficult. Fortunately, living
control systems don't need to do that, so they don't do it -- unless, of
course, they also happen to be control engineers who earn their livings
building optimal controllers.

I see nature as a grand optimal control engineer, who drops (through a
process called natural selection) inferior designs as soon as better ones
become available. Even small improvements soon become dominant -- where
soon must probably be measured in millennia. In my vision, optimality is
indeed what nature strives for. How this design process works is nicely
illustrated in the new branch of science that deals with genetic algo-
rithms. These algorithms are very robust in that they almost always lead
to optimal solutions even in strangely shaped and/or noisy error land-
scapes.

Is there a reference signal in the "model" and if so what is its relation
to the one we would call an "input?"

A model has no reference signal. Think of the model in this way: a model
provides a "virtual" ("imagined", predicted) feedback signal to a feedback
controller in such a way that "feedback" can continue even if there are no
feedback perceptions. The combination of model plus feedback controller is
called a feedforward controller.

If you want a little experiment with feedforward control: take any one of
the (sampled data) controllers that you designed. Have it output its norm-
al output at the odd sample intervals, and have it output a "feedforward
signal" at the even sample intervals. Experiment with different choices
for the feedforward signal. A good one is the previous output; another one
could be an extrapolation of the two previous outputs. Etc, etc. See how
much the quality of the control deteriorates. I await your reply.

(Martin Taylor 931105 12:10)

I read Hans as saying that in some conditions, conditions that are not
uncommon, there may exist within the hierarchy some ECS that is deprived
of perceptual input and nevertheless manages to produce output that
maintains the CEV at a tolerable level (one that would be produce only
a small error if the perceptual signal were available). It can do this
because the world as seen from its viewpoint is reasonably stable and
predictable.

You read me correctly.

I read Bill, Tom, and Rick as imputing to Hans the claim that all ECSs
in the hierarchy are simultaneously deprived of sensory input, and that
this condition happens often.

So do I. They seem not to understand what I am proposing.

Hans, on the other hand, ignores the normal function of reorganization
in a highly non-linear world, which is to provide mechanisms for dealing
with the situation when the world's behaviour changes (the wheels no
longer grip the road).

I do not ignore reorganization (learning); instead, it is my focus of
attention. I do, however, state that reorganization cannot normally be
instantaneous; it is usually a slow process that preferentially establ-
ishes links between observations and subsequent successful actions. This
"memory", that is usually built up gradually through experience, forms a
"model" that can be used on subsequent occasions, where it allows you to
function in a more or less stimulus-response manner. If I had had to think
up a new, good solution in those few moments when I skidded across the
snow, I would be dead now.

It's clear, and I think generally agreed, that some "control in
imagination" is normal, particularly in the higher layers of the
hierarchy.

Also in the lower layers, I think, but that depends on what you consider
"lower" and "higher". Much of this "imagination" goes on in the motor
cortex, where it leads to finely tuned skills. Ask any professional tennis
player how come he plays so well. He has trained his routine so well, that
much of his repertoire has become feedforward, although he takes care not
to become so predictable that he cannot fool his opponent anymore.

                                             And as Lang and Ham showed
as long ago as the mid-50's even a linear one-level control system with
normal sensory input can work a lot better with the inclusion of a world
model than without it.

It can be proven that ANY control system, even a feedback controller,
needs some kind of world model, i.e. needs to make certain assumptions
about the outside world. Feedback is a mechanism that makes the accuracy
of those assumptions less critical. That is the good news about feedback.
But there are limits. Going outside those limits may make a feedback
controller unstable. That is the bad news.

(John Gardner (931104.1200))

Sigh, I always regret when I speak up because I know that
I'll get roped into another round of engineer-nonengineer
discussions, but I simply couldn't let Hans 'twist in the
wind' like this.

Thanks for your sympathy, but I don't perceive much wind around me. I do
perceive being misunderstood.

                         I believe that the discussion
can take a much more fruitful turn if we examine the
hypothesis that most control systems contain elements of
both.

That was exactly my intention. But I was struggling against an overwhelm-
ing bias, with all those guys focused on feedback and not perceiving any
feedforward. Not anymore, it seems. So now I can embrace my own, multi-
perspective personality once again.

More recently, the Artificial Neural Net community has
proposed a control structure in which a neural net is used
as the feedforward element and it uses the output of the
feedback controller as its 'training signal'. In this
manner, the neural network learns a great deal of the system
dynamics and gradually takes over the load from the feedback
controller.

For me, the switch-over from Adaptive Control Theory to Neural Nets was
easy: the underlying theories are remarkably similar.

(Rick Marken (931105.1500))

"Feedforward" (open loop) elements are already recognized
implicitly in PCT; they are just UNCONTROLLED variables.

Funny terminology: feedforward control results in uncontrolled variables?

                                A variable that is under
feedforward "control" is easily detected using "the test";
disturbances to such a variable will simply not be resisted.

You might be surprised. If feedforward control operates within feedback
control, feedforward might eventually become optimally tuned. It WOULD be
possible to play football on the moon -- although it would soon be a very
different game.

(Bob Clark (931105.2130 EST))

Another view of "Feedforward."

Great discussion!

Clearly, he has some knowledge of skids and how to control them.

Yes. Strange as it may sound, I developped my skidding skills during a
car trip through the Sahara desert. In most places, the Sahara has an
underground of solid sand on which it is easy to drive, interspersed with
wind-blown patches of fine, dry sand. Encountering those is remarkably
similar in feel to driving on ice, but a lot less dangerous. Believe me,
that practice was fun!

                            Thus, having selected his strategy
("procedure?"), he continues to monitor ("pay attention to") the
approach of the metal obstacle and his own actions. Upon detecting
the presence of the edge of the pavement, he shifted his procedure to
other familiar procedures, again selected from his memory.

Yes, that is very much the way I evaluate it. In retrospect, I seemed to
select a well-learned (motor) PROCEDURE that could run its course. Rather
than having to feedback-control at all times, it just seemed required to
make some "discrete" decisions at some critical moments. Don't take me too
seriously now. Those are impressions, hard to make objective.

To the extent that "feedforward" implies "anticipation of future
perceptions," it is necessary to examine memories that resemble
current situations. Such memories can be extrapolated by established
methods. Of course any such anticipation is limited by the relevance
and accuracy of the available data. But even a limited extrapolation
can be very helpful in making decisions (that do not require the
Reorganizing System).

Agreed!

                   They seem to be accepted concepts -- they are
discussed in BCP. However I do find occasional references to
"awareness" and "attention," at least in the form of "monitoring"
on-going events.

I am trying to model these processes as well. It may be established by now
that adjusting the output function improves performance under certain
conditions. The same may be true for the input function. "Attention" could
be a sort of "tuning" of the input function, in which the Kantian "Gewuehl
der Empfindungen" is narrowed down in order to optimally employ the
brain's limited processing powers to solve the problem at hand. But
consider this speculation, for the time being.

(Avery Andrews 931106.2016)

                            Switch off the light, and a perception
of where you are continues to be maintained on the basis of such things
as double integration of signals from the vestibular canals ...

Which I consider a "prediction" by a simple "model", fairly accurate over
short time periods, less so over longer times, because integrators drift.

                                                                 I am,
however, eliminating any in-principle distinction between a perception
and a model, except maybe that a model might be regarded as a perception
based on multiple sources of sensory information.

If the memories/"models" in the brain are considered an alternative source
of perceptions -- if we recognize an inner and an outer world -- every-
thing can be fit into the same information processing/control scheme. The
focus, however, then shifts from control to learning: how do we store and
update memories. I consider this a valuable shift in perspective. It does
not undo the control notions, it adds to them.

The steering-on-an-icy-road incident reveals another subtlety of
control, (perhaps limited to apes?), which is control of expected
consequences.

Isn't all control exactly that? And it is not limited to apes, I think.
When you play a game of chess, you only find out whether you win or loose
after having played some 40 moves or so. Early in the game, the conse-
quences are still very unclear. There is no immediate feedback. If there
were, chess would loose its attractiveness as a game. You have to learn --
through experience or literature study -- what is a good move given a
certain position. The problem is then how to "distribute" the knowledge of
outcome (winning or loosing) over the moves that have been made (the
quality of the individual moves). Standard control theory cannot do that.
Optimal control theory can, but only theoretically; practically, it is as
difficult as computing the solution of the game of chess. Yet, good, prac-
tical -- though suboptimal -- solutions exist for many problems: genetic
algorithms, which describe a form of evolutionary learning closely related
to biological evolution. Even the lowest animals must make many decisions
whose final outcome (the successful bringing into the world of successful
progeny) can only be evaluated (in the process of evolution) at a later
time.

So again we have a choice: are we following pattern-action rules
(when you see dirt on the floor, sweep it up), or controlling for
predicted consequences (controlling for a perception `this will/will not
happen', or `this is/is not likely to happen'?

I don't see the difference. Sweeping the floor is done to control for
consequences as well, though they only may be social consequences ("if I
don't, society will frown on me; I don't like being frowned on; so I will
do it").

The difference may be hard to discern, because much of the reasoning
whereby consequences are predicted also takes the form of pattern-action
rules, but the pattern-action-rules should be able to be made to
betray their presence by a certain rigidity and a tendency to be
followed even when they are clearly serving no useful purpose.

That is the basic problem of learning, whether by an individual or by a
society: learning is based on a correlation between action and outcome.
Essentially, learning is a "superstitious" mechanical process. Learning
halts as soon as success is reached and you run out of ideas. Eventually,
another designer will find an even better solution. A rain dance delivers
rain, guaranteed -- if the tribe keeps it up.

In Hans' particular case, we don't know whether he (a) figured out
from first principles that yanking the wheel would be a bad idea
(b) had gotten some serious training and actually been taught what
to do under these circumstances (c) read something somewhere, and
managed to apply the knowledge. I would see (a) or (c) as genuine
control of predicted consequences (controlling for `I will be less likely
to loose control of the car'), while (b) might involve a significant
degree of pattern-action rule.

You might guess. Be practical. "Figuring out from first principles" cannot
be done in a few seconds, if the situation is complex. Have you read any
of the AI literature on "theorem provers" and how much time they take?
"Reading" and "apply[ing] the knowledge" are very different things. You
cannot learn to play tennis by reading about it, no matter how much.
Training/experience is the answer, i.e. building a model, a "best" stimu-
lus-response connections.

(Bill Powers (931106.1115 MST))

<schematic deleted>

Now we have a reference signal that enters the output function
directly, driving it to produce an output in a calibrated
feedforward way. The same reference signal enters a comparator
where it is compared with the perceptual signal p as usual. The
error signal adds to and subtracts from the direct effect of the
reference signal on the output function.

Now you're talking, Bill! Give you a stimulus, and you switch over into
creativity mode! :slight_smile:

(Martin Taylor 931106 20:45)

This doesn't say that biological systems do the same, but it is a usual
presumption that evolution is pretty good at finding successful tricks,
so it would not be surprising if they do.

There are more indications (genetic algorithms) that evolution is a great
optimal problem solver.

(Bill Powers (931106.2020 MST))

                                      when we get around to
testing control when the feedback information is periodically
blanked out, my new hybrid scheme, similar to the Lang-Ham
scheme, will be required.

Greatgoing, Bill! Keep me up to date on how you proceed!

Greetings,

Hans