assorted replies

[HANS_BLOM2]

[Hans Blom, 950511]

Assorted replies and comments follow.

(Rick Marken (950510.0915))

... I think it is very likely that any ability to maintain
apparent control while a perceptual input is briefly lost can
probably be handled by the existing hierarchical control model.
... I think the "prediction" that occurs in control of
imagination can be handled on the input side of a perceptual
control process that uses the perceptual control system that has
last its perception.

Please demonstrate.

Will do.

Thank you. This promises quite a lot, however. My suggestion is,
therefore, to limit your demonstration to one small but for me very
significant part: explain, on the basis of PCT models, how approx-
imately zero response times can be observed when a subject tracks a
repeating waveform. That would provide you with a clear and un-
ambiguous challenge, I think.

(Bill Powers (950510.1100 MDT))

···

Here is an example of an arbitrary disturbance waveform:

                   *
            * *
        * * * *
      * **** * *
    * * * * **********
    * * *
***** * *
    1 2 3 4 time --> * *

The first increase is a step and hence cannot have been predict-
ed, nor controlled away. But once it has been observed, it could
be the basis of further predictions. A model can compared to a
succinct description/resume of past observations: chances are
that if something was true in the past, it will be true in the
future.

You are assuming that this same waveform will occur again in the
future.

No, I must have expressed myself badly. I meant repetition WITHIN the
waveform. I also meant short-time predictions. See the numbering that
I added on the time axis. The jump at 1 is unpredictable, but the
increase in amplitude going from 1 to 2 can serve as a predictive
model for how the amplitude is going to change in the next episode,
going from 2 to 3. Actually, in the curve above, even such a simple
model/extrapolation would be accurate most of the time. Sometimes not
so accurate, and only completely wrong where step changes occur. But
if we assume band-limited noise with low frequencies only, and if the
updates of the model occur at a high enough frequency, the predict-
ions WITHIN THE WAVEFORM will be fairly accurate at all times.

But when I speak of an "arbitrary" waveform, I mean a non-repeat-
ing waveform as well as one occupying a low-pass bandwidth. When
I say "unpredictable" I don't mean unpredictable in the manner of
white noise, but unpredictable in the sense that the waveforms are
non-repeating -- as far as any observer (or the control system
itself) is concerned, patternless.

So we agree. But I maintain -- see above -- that under these
conditions a fairly accurate model can be maintained at all times.

Actually, in putting that step in, I violated my own condition >that

the waveform be a low-pass waveform -- in a low-pass waveform

there could be no steps.

Right. A physically realizable (finite gain, finite bandwidth)
control system would not be able to control those steps away. A
physically realizable model cannot accurately predict in the presence
of step changes.

In fact, an important theoretical finding shows that control and
identification are "dual" problems, in the sense that they are
governed by similar equations and thus can operate with similar
quality under similar circumstances. Theoretically, there is no basis
to assume that non-model-based control can be better than model-based
control, nor the other way around, given full information about the
"world" (the object to be controlled). That does not mean that under
different circumstances (a model that is not complete or a control
law that is not based on full information) one or the other, or a
combination of both, might be better.

Sometimes it helps to be in academia where rigidly proven theorems
float around ;).

           It is not necessary that the control system have any
information about the fluctuations in disturbances that are going
to affect xt.

Let me qualify this. If the control system had full information about
all the fluctuations of all the disturbances, it could control
perfectly. Indeed, "control" would not be necessary. If the control
system has no information at all about the (fluctuations in the)
disturbances (amplitude, bandwidth), it might not be able to control
well, if at all. Usually we find ourselves somewhere in the middle:
some things we know, other things we do not know. My first lesson in
control engineering was -- and the prof repeated it often -- that the
quality of a controller is the better the more knowledge it is given
about the thing that it must control.

If the reference signal xopt is constant, and if the above
waveform enters as an additive term in the real system so as to
contribute to the state of xt, the resulting behavior of xt could
be made constant enough (through fluctuations of u that oppose the
disturbance) that the effects of the disturbance are undetectable.
This is not a matter of just cancelling out the "predictable part"
of the disturbance. The entire disturbance would be cancelled.

                         ^^^^^^
This cannot be true, as you know, in a real control system. I find it
interesting to analyze under exactly which conditions disturbance
cancellation is at its worst and how it can be improved.

At a meta-level it is an important remark, however, because it points
at the area where our differences in interest are: you are impressed
by how well a simple control scheme works (under steady state con-
ditions, when learning, if necessary at all, has been completed),
whereas I am more concerned with obtaining ever better control, i.e.
learning per se.

But the world out there is not inaccessible to your system: you
have a variable y which represents (is a perception of) the
real-world variable xt. True, from the standpoint of the system,
there is no way for it to know of xt directly; all it knows is y.
Therefore all it can control is y. But if y = f(xt), then in the
real world something is being controlled that is the inverse
f-function of y.

You are forgetting something here: the noise in y. It is not y that
is controlled; control is quite robust in the face of even large
noise levels in y. Have you tried that?

In a negative feedback control system, it is y that is actually
being controlled (behavior is the control of perception).

One of the first applications of Extended Kalman Filtering was in
early satellite tracking and control. In those times, radar ranging,
for instance, was fairly inaccurate/noisy, so the observations as
such could not be used. Some kind of filtering was required in order
to minimize the amount of fuel that would be expended in the control
actions. I have no idea in how far sensor noise is an important
consideration in organisms. If it is not, you are right in that we
can know some of the world's aspects/dimensions directly, be it
through a coordinate transformation.

... A closed-loop negative feedback control system does
not have to have any knowledge of these external variables, nor
how they are going to behave in the future.

I maintain that this assertion is false. At least approximate
knowledge of the system to be controlled is required in order to
arrive at a good enough control system. In my blood pressure
controller, for instance, the patient's sensitivity for the drug
could vary (unpredictably but slowly) by a factor of 80. No fixed
design controller can handle that; some kind of adaptation to and
thus modelling of the sensitivity is required in order to bring that
variability down to a factor of 2 or 3, which a standard (PID)
controller can handle. This is like playing ping pong, not neces-
sarily in the dark, where the mass of the ball can -- slowly, but
unpredictably -- change by a factor of 80. It takes quite a soph-
isticated controller to play this game well...

Greetings,

Hans

[HANS_BLOM1]

[Hans Blom, 931122b]

(Bill Powers (931117.0815 MST))

Generally, you seem to have an accurate understanding of this almost
infinitely brief and sloppy introduction into adaptive control theory.
Just a few extra comments.

You mention that if the disturbance is not a Gaussian noise, the
mathematics becomes intractable. What isn't clear here is how d
is treated: are its effects on the state vector computed moment
by moment, or in a statistical way? If the only problem is
mathematical intractability, you could use simulations to solve
the equations, so that isn't a basic problem.

Yes it is. Remember that we are talking about calculated PREDICTIONS which
have a statistical nature. Now the theoretical probability distribution of
the prediction can, even if we start with a nice Gaussian distribution, be
shown to contain ever more additional moments, up to infinity. Somewhere
the number must be truncated, in order for the system to be physically
realizable. That is one fundamental reason why so many practical long term
predictions soon start to loose accuracy.

where ref(k) denotes the future trajectory for x to follow. It
will not surprise you that the control parameters p, q, r and s
(a constant) can be computed from the prescribed reference
trajectory, the estimated parameters a, b, c and d, and their
(co)variances.

Here we have the first major departure from PCT. The reference
vector specifies a future trajectory for x(k), the state of the
external system, to follow. From the standpoint of an engineer
who can see x, this makes perfect sense. But if we're talking
about living control systems, all the control system knows is
y(k).

The theory makes a distinction between what the system OBSERVES (y) and
what the system KNOWS (the probability distribution of x).

           It has no direct information about x(k).

No, but it has indirect information, through y.

                                             Therefore its
reference signal can specify only the desired state of y(k),
because all adjustments of parameters must be made on the basis
of what the controlling system can know of the world outside it
through its sensors. All that information is contained in y(k);
there is no other path for information to get inside the control
system.

Think of y as a (noisy) observation, and x as its filtered, and hence
presumably purer, stored analogy.

             There is no engineer standing by to tell the living
control system it is really trying to control x(k). So the system
cannot set reference states for x(k).

Why not? The variable x is an "internal variable", that can be "perceived"
(recollected from memory) much in the same way as the "external variable" y.

                                               But the living
control system can use only information that enters it through
its senses; it has no other way of knowing what is going on
outside it. When it controls, all it can control is the
representation of the external world that is contained in its own
perceptual signals, y(k).

No. The vector y represents all CURRENT observations. The vector x is an
"internal variable" that is available as well.

The principles of optimal control will probably be very useful in
future developments of PCT. But the game will have to be played
according to slightly different rules. Everywhere that x(k)
appears in the controller equation, you must substitute y(k).

What you propose is actually a special case of what the theory already
supplies: no measurement noise and full observability (in the outside
world) of all components of x. The latter is frequently unnecessary. The
internal model might contain position, velocity and acceleration of an
object, whereas only position is actually measured.

This will undoubtedly reduce the ability to deduce the absolutely
optimal control system, because some of the information about
x(k) is lost: all that is not transmitted to the control system
by the matrix m, the input function.

Au contraire, as my French colleague would say: it is a much EASIER

situation: perfect sensors of all modalities that exist internally. The
general case is harder. You would have to MODEL velocity and accceleration
from position data only, in the example above.

                                 The basic rule with regard
to modeling living control systems is that only information
available to the controlling entity can be used in the
optimization of the control processes.

Yes. And this information is of two types: that available to the senses,
and that available in memory to the "inner senses".

But that is exactly the situation that a living control system
faces. It does not know the real true Boss Reality. It knows only
what it can perceive, which is only a limited transformation, m,
of What Is.

Optimal control theory describes how "Inner Reality" can be calibrated
using "real true Boss Reality". It also describes the limitations of this
process.

(Mary Powers 931117)

You say: "feedforward" denotes a type of control in which no
"feedback" (perception of outcome) is required. Feedforward
normally refers to a process of computation of output based on
inputs only.

Inputs of what? As I understand control systems, their inputs ARE
perceptions of outcome. What other inputs might there be?

Inputs that "perceive" data stored in memory, or from models "running" in
memory, in parallel with Boss Reality.

This doesn't make sense. The steering wheel doesn't compute
anything, ever. The linkage is a passive mechanical device. If
the steering wheel is turned, the wheels turn. Gracing it with
the name feedforward makes more of it than it actually is,
doesn't it?

What would you call it? A linkage between steering wheel and wheels seems
such a stupid simple example that we might forget how clever an invention
it really is! I could maintain that it "computes" a function y = f (x),
with the additional beautiful property that f is fixed, constant, and
very, very difficult to disturb. Who would want feedback in a situation
like that? Would you trust your car more if that linkage WERE feedback-
controlled? Actually, the whole purpose of ALL feedback systems is to
attempt to realize linkages as sturdy and reliable as that steering wheel
one!

(Rick Marken (931117.1330))

In my view, feedforward has an important place in
control, and is most fruitfully combined with feedback control.

You make this claim even though you have presented NO evidence (other
than the notorius "walk in the dark" anecdotes) that feedforward is
needed to explain any aspect of human behavior.

Rick, this time I really find it too hard to attempt to answer you. What I
invariably find necessary when reading someone else is a slight "suspen-
sion of disbelief", the basic assumption that the other person is not
crazy and might have something to contribute, even if he uses a different
language or has a different world model. Almost invariably, I find that a
difference in language/terminology is the basic problem in interhuman com-
munication about basic issues between consenting adults. I prefer not to
coomunicate with people who I perceive as not willing to "make sense" of
what I say.

(Bill Powers (931117.1500 MST))

You mention the fact that the performance changes when the
disturbances become more predictable. That fact interests me
greatly, and one day I hope to extend the model by another level
to try to account for that difference. Perhaps feedforward will
then prove to hold the solution.

"Feedforward" is not the central theme. Internal maps are, and what can be
done with them, I think.

                                 Remember, we are not designing
a control system, but trying to figure out the design of a system
that already exists.

That is what I want to do as well. In "reverse engineering" experiments
one can at least demonstrate some of the laws that must necessarily under-
ly learning and operating in changing environments.

Your description of your proposed tracking experiment is clear
now.

Great.

  What is the advantage of scrolling the display to show past
positions of the target?

It is an aid to make the regularity of the movement of the target more
apparent to the subject. It increases the subject's learning speed
tremendously, so that a one minute learning interval is all you need in
many cases. The latter makes it much easier on the researcher as well.

As soon as the target was more or less predictable, the
operator soon learned to make use of the regularities. This
resulted in ridiculous results like zero or even negative delay
times.

Why is that a ridiculous result? It's what you observed, isn't
it? Did you give up on the modeling at that point, or did you try
to find a model that would behave the same way?

What do you think I'm talking about all the time? A subject's ability to
PREDICT!

By the way, while you were doing these experiments, did you ever
realize that it was a perception that the person was trying to
control?

No. We were always output-oriented: the subject should USE his perceptions
to DO something. You have shown me the other side of the coin.

                         Even now, as I type, my fingers
are remaining within about 1% of their total positional range,
falling on the keys I want to hit perhaps 98 or 99 percent of the
time.

I don't know whether you are a good typists, but high speed typing is
another example of feedforward control. Just ask professional typists.
They will report that frequently they realize that they are going to make
a typing error before it occurs. Feedback just would not allow the speed.
High speed typing REQUIRES feedforward, ddespite the occasional error...

From another post:

All that a control system needs to know about the environment is
that if it acts in certain ways, perceptions change in certain
ways.

I have proposed something like this in the past. It requires, however,
that the actions/outputs of the control system must be available to its
computing machinery. This is something that the current PCT-model lacks.
Are we converging here as well?

Greetings,

Hans