MCT and PCT models

[From Bill Powers (970331.0030 MST)]

Still processing old posts in the attempt to understand what Hans Blom is
talking about. This runs in parallel with my objections to what I understand
of his model. One part of me may be saying, "This doesn't make sense," but
another part is always asking, " ... but how _might_ it make sense?"

Hans Blom, 970327--

Two paradigms converse with each other. Says one to the other: "I
don't understand a word you say". Responds the other: "What did you
say? I don't understand you at all".

....

In fact, the Kalman Filter equation of what a model-based
controller does when _processing an observation_ is

x-model := w1 * x-model + w2 * x-observation

where w1 and w2 are _weights_ which depend on the accuracies (known
or estimated in/by the model) of both x-model and x-observation. Thus
x-model is a weighted average of the internally generated prediction
x-model and the observation. This is the general formula, but we can
distinguish two extremes:

...

I could instead have written a code line
somewhere in my program such as

x := 0.0 * p + 1.0 * xx

where in my code p is the predicted model-x and xx the direct
observation. Would that have helped?

Yes, it would have helped. What this tells me that I didn't understand the
way the Kalman filter was being used, and thus the significance of your
condition that the observation be "error free." The significance is that
when the observation is error free, w1 is set to 0 and w2 is set to 1.

Thus I now understand that when the observation is error free, the MCT model
is _exactly_ the PCT model at least in form. The internal world-model is not
used at all! If you could read block diagrams, you would have seen that none
of my diagrams reflects what you are saying here. The situation you are
describing is this (omitting the Kalman Filter machinery and any other
crucial parts I may still not understand or you may not have mentioned):

                           r
                           >
        [Sum]----------->[COMP] -->-
        > > >
      [w2] [w1] [output computation]
        > > >
        > x'<--[World-Model]<-----u (output)
        > > CONTROLLER
   =====|==========================================================
        > > ENVIRONMENT
        > <----u
         --<-x <--[Real Object]
                               <--- d

In the case of noise-free observation, w1 is 0 and w2 is 1 (I assume that w1
and w2 always sum to 1). At this extreme, the control loop uses ONLY
real-world observations generated (in our examples) by the exact physical
model of the Real Object!

In this case, a loss of input from x would result in immediate loss of
control, just as in the PCT model; i.e:

The only workable values of w1 and w2 are 1 and 0, if control is to be
simulated upon loss of input from x. Suppose that they were each 0.5. If the
world-model is exactly correct, loss of the x input would lead to the
feedback from the world model's x' being 0.5 times the value of the
simulated x, so instead of matching x' to r, the system would match 0.5*x'
to r. This would result in the wrong output u. The Kalman filter routine (or
some separate subsystem) would have to detect this error and set w1 to 1 and
w2 to 0. So there would be a transient error while this switch was being
made. The point is that correct operation in the absence of real-time
sensing of x requires that w1 = 1 and w2 = 0.

With your handicap concerning block diagrams ;-), you probably didn't
understand a "compromise proposal" I made some months ago, in which I showed
exactly the above diagram, with a switch in place of w1 and w2. The switch
could be thrown so the feedback signal entering the comparator came from
either the real-time input x or the simulated input x'. This is exactly
equivalent to saying that [w1,w2] can have only the values [0,1] and [1,0].
As I have pointed out above, these are the only values that will properly
handle the situation when detection of x is lost. Somehow, the controller
must be able to distinguish between the loss of the connection from x and
x's value actually going to zero, and throw the switch appropriately. In my
compromise proposal, I didn't specify how this distinction was to be made,
and you haven't specified it either (in your programs, the _programmer_
knows that input is lost at a particular time, and arbitrarily sets the
weights to 0 and 1 as required, at the right moment, rather than letting the
Kalman filter assign the weights -- which it would do incorrectly).

So this clears up one mystery. In the case of noise-free observation, the
MCT model is, in form, simply the PCT model.

Your case 2 is presented in a misleading way:

2) x-observation is or can be extremely erroneous. This is the
"missing observation" or "walking in the dark" case. The above
formula now has w1=1 and w2=0 and reduces to x-model := x-model,
resulting in open loop behavior.

When you say that the x-observation is or can be extremely erroneous, you
are saying that the _perception_ of x (called y in your earlier models) can
be a misrepresentation of x. But the only kind of misrepresentation I have
seen offered is that y is x plus a zero-average high-frequency noise term.
This, however, is only one way in which y can be erroneous. In the general
case, the non-random component of y can be some function of x, so that

y = f(x) + noise.

The simplest kind of sensing error is a calibration error: y is proportional
to x but not equal to it, even when there is no observation noise. The
Kalman filter approach can't detect an input calibration error, because y
represents its only knowledge of x. All that the KF approach can do is to
assume that certain variations in y are random and discount them by giving y
less weight and x' more weight. A systematic calibration error, either of
scaling or of linearity, cannot be detected. As a result, the Kalman filter
will adjust the controller for best control of y, but x will not match the
reference level except where the calibration curve crosses the assumed
linear relation between y and x (if it does).

The PCT model, of course, has exactly the same problem, except that the PCT
model doesn't claim to control anything more than the perception.

The confusion may arise because you cannot see the model "in" the
computer code. That is right: it is "behind" the code. It will need
to explicitly appear _in_ the code only when measurements are noisy.
But that is not the case in our example. And it would make the PCT -
MCT comparison far more difficult to investigate -- except in terms>
of performance...

I do have a problem with understanding a model when essential parts of it
are omitted!

Bill, what gives you the perception that I'm a blathering idiot? I do
know what I'm talking about. And it's not your lack of knowledge that
worries me, but your attitude. When you say things like this in your
posts, that is -- _not_ in your attempts to reach understanding...

What gives me this impression is your way of leaving out crucial
explanations, as above. It doesn't help anyone else if YOU know that there
is a w1 and a w2 being adjusted by some background process, but fail to let
anyone else in on this fact. The impression that both Martin Taylor and I
got was that you simply couldn't tell the difference between a continuous
physical process and the MCT model's discrete internal simulation of it. You
never explained that this internal model wasn't even being used.

You said once that you don't intend to give a tutorial on the Kalman filter.
But how are you going to communicate what you mean if you don't do exactly
that? How many more aspects of the MCT model are there that you're keeping
to yourself?

Also, Hans, the mere fact that a lot of people accept the textbook
explanations of MCT is no indication at all that these explanations are
error-free. How many people who espouse this approach are simply passing on
the arguments and interpretations they all read in these same textbooks? And
how many have gone down to the nitty-gritty, iteration-by-iteration level
that we're looking at? I would expect that, as in any other field, there are
a few innovators who generated these ideas, and many followers who are
content to accept the pre-packaged explanations and interpretations given by
these few people. If the innovators happened to share a common
misinterpretation of the relation of the model to the world -- if they all,
for example, use an inexact non-physical approximation of the actual
physical plant to represent what the world is actually doing -- who is going
to call attention to that, and work out its implications? Certainly not
those followers who take the correctness of the arguments on faith, and that
eliminates the vast majority of those who accept the MCT approach. And
certainly not the innovators themselves, if all they are interested in is
proving that they are right.

So who does that leave to keep the theoreticians honest, but outsiders like me?

Best,

Bill P.

[Martin Taylor 970401 11:15]

Bill Powers (970331.0030 MST)] to Hans Blom, 970327--

So who does that leave to keep the theoreticians honest, but outsiders
like me?

What keeps me listening to Hans is that he _isn't_ a theoretician. He's
a practitioner who makes MCT control systems that function in life-critical
situations, and also understands the theory on which he devlops those control
systems. That says to me that if he says something that seems to me as
if it can't work, or doesn't make sense, then it is more likely that there's
a communication malfunction than that Hans's systems would kill.

I do have a problem with understanding a model when essential parts of it
are omitted!

>Bill, what gives you the perception that I'm a blathering idiot? I do
>know what I'm talking about. And it's not your lack of knowledge that
>worries me, but your attitude. When you say things like this in your
>posts, that is -- _not_ in your attempts to reach understanding...

What gives me this impression is your way of leaving out crucial
explanations, as above. It doesn't help anyone else if YOU know that there
is a w1 and a w2 being adjusted by some background process, but fail to let
anyone else in on this fact. The impression that both Martin Taylor and I
got was that you simply couldn't tell the difference between a continuous
physical process and the MCT model's discrete internal simulation of it. You
never explained that this internal model wasn't even being used.

That has nothing to do with the problems I have been having with Hans. It's
quite a separate issue. My problem is with the ability of a discrete
simulation to model a continuous one in such a way that the experimenter
can rely on the simulated observations. One of the issues is that there is a
general proposition that you can't rely on _any_ simulated effect in which
something important changes "in one step," where the "one step" is one
simulation sampling interval. Hans keeps talking about the theodolite
controller observing as often as the simulation sampling interval allows,
and _nevertheless_ making a "one-step" correction. In such a situation, the
simulation says _nothing_ about what may happen in the simulated
continuous world.

I suspect that there are many unsaid background assumptions beyond the ones
you have already winkled out. I see another possibility in:

In the case of noise-free observation, w1 is 0 and w2 is 1 (I assume that w1
and w2 always sum to 1). At this extreme, the control loop uses ONLY
real-world observations generated (in our examples) by the exact physical
model of the Real Object!

In this case, a loss of input from x would result in immediate loss of
control, just as in the PCT model; i.e:

The only workable values of w1 and w2 are 1 and 0, if control is to be
simulated upon loss of input from x. Suppose that they were each 0.5. If the
world-model is exactly correct, loss of the x input would lead to the
feedback from the world model's x' being 0.5 times the value of the
simulated x, so instead of matching x' to r, the system would match 0.5*x'
to r. This would result in the wrong output u. The Kalman filter routine (or
some separate subsystem) would have to detect this error and set w1 to 1 and
w2 to 0. So there would be a transient error while this switch was being
made. The point is that correct operation in the absence of real-time
sensing of x requires that w1 = 1 and w2 = 0.

Let's suppose that the filter works as do many neural systems, not only as an
absolute value, but as a value that is compared with local references (local
in time and space, that is). And let's suppose, for the sake of argument
rather than as a proposal, that the local reference comparison allows the
filter to make a crude assessment of the reliability of the value.

Now, _any_ filter, neural or not, can change its output only as fast as its
bandwidth allows. If the incoming data input is greatly different from its
immediate past value _and_ from the value predicted by the filter, its
reliability might _immediately_ be reduced, switching the weights
instantaneously (nearly) from 1:0 to 0:1, resulting in almost no transient
at all.

The same effect might occur if the perceived value (from the sensory input)
were derived not just from a simple magnitude of input, but from a magnitude
related to context. When both context and magnitude suddenly go to zero,
the reliability of magnitude might well be suddenly dropped and the switch
of weights made.

The Kalman filter can determine its own reliability of estimate, based on
its bandwidth. By the time the autocorrelation function of its equivalent
finite impulse response has gone to zero, so has its reliability, and if
there is _any_ data input the weight on the data input would then go to
1.0.

I don't think that the system has any means of determining whether the
output is wrong, unless it has access to sensory data input. So with the
loss of data input, if there exists a local-comparison way of assessing
the reliability of the filter output, the system would seem likely to
track with an error that increases as the autocorrelation of its FIR
decreases.

The only situation in which the weights would go to 0.5:0.5 would be if
there exists an estimate of the incoming data magnitude, equal in reliability
to the data prediction. That's independent of my suggestion above. As I
understand it, the sum of the weights must always be unity. The question is
whether a sudden vanishing of the data input is equated to a zero value
for the data input.

Like I said, I'm not providing this as a proposal, but I am guessing that
something like it may be one of the underlying missing assumptions that
get in the way of communication.

Martin

[From Bill Powers (970401.1557 MST)]

Martin Taylor 970401 11:15--

My problem is with the ability of a discrete
simulation to model a continuous one in such a way that the experimenter
can rely on the simulated observations. One of the issues is that there is
a general proposition that you can't rely on _any_ simulated effect in
which something important changes "in one step," where the "one step" is
one simulation sampling interval. Hans keeps talking about the theodolite
controller observing as often as the simulation sampling interval allows,
and _nevertheless_ making a "one-step" correction. In such a situation,
the simulation says _nothing_ about what may happen in the simulated
continuous world.

Yes, this is a serious problem, which is a strange one for an experienced
practitioner of control system design to present to us. Hans' model requires
that the peak output needed to produce these "one-step" corrections be
inversely proportional to the sampling interval, leading to extremely large
values of output as Dt approaches zero. Han's comment on this "feature" of
his model was that nobody told him that infinite output was to be avoided:
"you wanted perfect control, I gave you perfect control." In some circles,
this is known as "malicious compliance."

The Kalman filter can determine its own reliability of estimate, based on
its bandwidth. By the time the autocorrelation function of its equivalent
finite impulse response has gone to zero, so has its reliability, and if
there is _any_ data input the weight on the data input would then go to
1.0.

The chief problem with the Kalman filter approach is that the filter itself
has to be considered completely reliable and noise-free. In any realistic
model, every process has to be considered susceptible to random variations.
The PCT model, with its small requirement that error not go exactly to zero
is continuously being compared with the hypothetical Kalman Filter and
discrete controller which achieve perfect control in one iteration through
infallable computations of unlimited precision, acting through massless
effectors on a world which can make finite state-changes in no time.

As you said in one reply to Hans, there is really nothing that the Kalman
filter design does that couldn't be done by a hierarchical architecture of
control systems. All that seems to be missing is the motivation on the part
of people with enough mathematical skill to develop the hierarchical
alternative.

Best,

Bill P.

[Hans Blom, 970407e]

[From Bill Powers (970331.0030 MST)]

In fact, the Kalman Filter equation of what a model-based
controller does when _processing an observation_ is

x-model := w1 * x-model + w2 * x-observation

where ...

I could instead have written a code line
somewhere in my program such as

x := 0.0 * p + 1.0 * xx

where in my code p is the predicted model-x and xx the direct
observation. Would that have helped?

Yes, it would have helped. What this tells me that I didn't
understand the way the Kalman filter was being used, and thus the
significance of your condition that the observation be "error free."

My understanding was that we were not talking about the Kalman Filter
which, after all, implements the model-building part of an MCT
controller. I understood that I could use its _result_, the model,
and that we were to concentrate on _control_, having the model
available. I did just that.

Thus I now understand that when the observation is error free, the
MCT model is _exactly_ the PCT model at least in form. The internal
world-model is not used at all!

This is a misunderstanding. The output of the model-building part is
the model of the environment equation. In case of a perfect model, it
_is_ the environment equation. The responsibility of the control part
is then to substitute xref for x in the model (environment) equation
and solve for u. This is exactly what the controller does. So the
internal world-model is fully used...

The situation you are describing is this (omitting the Kalman Filter
machinery and any other crucial parts I may still not understand or
you may not have mentioned): ...

This is forcing the MCT paradigm into the straightjacket of the PCT
formalism. I object to your block diagram for several reasons:

1. it is insufficiently exact and leaves too much to the imagination.
I need hard math, not soft pictures. They leave far too much to be
desired.

2. The PCT comparator is, wherever it might occur, replaced by a
minimizer (of a function that _can_ be minimized, such as a
multi-dimensional sum-of-squares). Thus an MCT controller always
finds a "best possible" (optimal) solution -- the minimum.

3. The PCT single-dimensional reference level is abolished in favor
of a multi-dimensional one, offered to the minimizer. Thus an MCT
controller's objective might, for instance, be stated as finding the
minimum of

(x-xref)^2 + 2*(y-yref)^2 + 0.25*(z-zref)^2 + 5*u^2

where x, y and z are observations and u is one of the controller's
outputs. I do not see how this can be given a PCT translation,
because it would "confuse" PCT's hierarchical levels. Note that this
MCT formulation allows what you call "conflicting goals" to be
formulated. In MCT there are no "conflicts"; there are only "optimal
(best, given the requirements) solutions".

In PCT, it is still unclear what the controller's goal is. Goal is
usually identified with reference level, even if the reference level
cannot be reached. In MCT, the goal is specified explicitly -- it is
the minimization of a specific function.

So this clears up one mystery. In the case of noise-free
observation, the MCT model is, in form, simply the PCT model.

Yes. Very much like Einstein's equations reduce to those of Newton in
case of velocities far lower than c.

Bill, I don't understand you. You always harp on more research to be
required on PCT, yet you have disregarded the enormous research done
on control. PCT is based on 1950's servo-control, which at the time
did not have an adequate theory behind it. Now it has, and it has
been shown to be one particular form of a far more generally
applicable class of control formalisms. One effect of the research is
that PCT or "servo control" is now much better understood -- in what
it can do and what not, in what its advantages and disadvantages are.
You still want to explain everything in terms of a theory that
everyone else in the field recognizes as inadequate and
underdeveloped. And as a result of not knowing other methods and
their advantages, you do not see the disadvantages of PCT. The main
one being that, however cleverly _you_ can design what is now called
an "adaptive controller", you cannot come up with a general enough
scheme that will let the _controller_ (re)design or (re)tune itself.
But that problem has been solved a long time ago! And when I tell you
about some of these newer approaches you doubt every small step
beyond what you know, as if I and the whole world of control theory
attempt to deceive you. I find your paranoia utterly frustrating.

Bill, what gives you the perception that I'm a blathering idiot?

What gives me this impression is your way of leaving out crucial
explanations, as above. It doesn't help anyone else if YOU know that
there is a w1 and a w2 being adjusted by some background process,
but fail to let anyone else in on this fact.

Bill, I'm just rehashing simple textbook stuff; I'm hardly secretive.
I've told you so before. I have tried to explain a thing or two, but
experience such an incredible resistance every time that no progress
seems possible. And I really would like you to become acquainted with
(modern!) control theory so we can talk at a more meaningful level.

In frustration,

Hans

[Martin Taylor 970408 13:25]

Hans Blom, 970407e]

3. The PCT single-dimensional reference level is abolished in favor
of a multi-dimensional one, offered to the minimizer. Thus an MCT
controller's objective might, for instance, be stated as finding the
minimum of

(x-xref)^2 + 2*(y-yref)^2 + 0.25*(z-zref)^2 + 5*u^2

where x, y and z are observations and u is one of the controller's
outputs. I do not see how this can be given a PCT translation,
because it would "confuse" PCT's hierarchical levels.

This isn't intended as a comment on the rest of your message, but is to
help you see how it could be given a PCT translation.

x, xref, y, yref, z, zref, and u must all be inputs to the perceptual
function of some control system. There's no problem with this, since
any signal anywhere potentially can be observed. In the "classic" PCT
hierarchy, "-ref" variables are not perceived, but there's no reason
they should not be in any specific configuration.

The perceptual input function of this control system produces
  (x-xref)^2 + 2*(y-yref)^2 + 0.25*(z-zref)^2 + 5*u^2
as its output. This is compared with a reference level of zero, and the
resulting error signal is the input to the control unit's output function
(a leaky integrator, perhaps). The resulting output is distributed to the
reference inputs of controllers for x, y, z and u, where they are added
with weights learned through reorganization to the x-ref, yref, and z-ref
values to provide actual reference values for x, y, and z respectively.
The system attempts to influence its output only when one or more of
x, y, z and u differs from its reference (u having an implicit reference
value of zero).

That's a simple-minded PCT translation. But why bother, when x, y, and z
are already being controlled to match their reference values. You could
effectively do the same job by controlling for u = 0, which tends to conflict
with the other controlled variables being near their reference values. With
appropriate weightings on the gains, your function could be achieved in
that way as well.

And as a result of not knowing other methods and
their advantages, you do not see the disadvantages of PCT. The main
one being that, however cleverly _you_ can design what is now called
an "adaptive controller", you cannot come up with a general enough
scheme that will let the _controller_ (re)design or (re)tune itself.

I don't understand why you say that. Are you asserting that the general
reorganization scheme has been shown to fail? If so, in what way does it
fail? It incorporates both complete redesign and continual retuning. Are
you saying it demonstrably doesn't work, or that it theoretically can't
work?

Martin

[From Rick Marken (970408.0950 PDT)]

Hans Blom (970407e) --

Bill, I don't understand you. You always harp on more research to be
required on PCT, yet you have disregarded the enormous research done
on control.

The research Bill is talking about is research that tests the PCT
model of behavior. You have to do (the appropriate kinds of) studies of
the behavior of living control systems to test the PCT model of
purposeful behavior.

PCT is based on 1950's servo-control

Closer to 1930's, I'd say.

which at the time did not have an adequate theory behind it.

The theory didn't have an adequate theory behind it? What in
the world are you talking about, Hans?

You still want to explain everything in terms of a theory that
everyone else in the field recognizes as inadequate

Inadequate? PCT accounts for the basic facts of purposeful behavior;
it is still not clear that MCT can do that. And I have already
developed an integral control demo that shows that humans exhibit no
control at all over the variable they can no longer perceive. MCT
explains a phenomenon (the ability to control "in the blind") that
doesn't exist. For some reason, I'm not impressed.

And as a result of not knowing other methods and their advantages,
you do not see the disadvantages of PCT. The main one being that,
however cleverly _you_ can design what is now called an "adaptive
controller", you cannot come up with a general enough scheme that
will let the _controller_ (re)design or (re)tune itself.

Why is it important that the controller retune itself? What is the
advantage of that (either in terms of explaining behavioral data
or in terms of designing a working controller) over having two
control systems, one that controls a variable and another that controls
(tunes) the first controller?

Bill, what gives you the perception that I'm a blathering idiot?

Just read what you write, Hans. It's really pretty obvious;-)

Best

Rick

[From Bruce Gregory 9970408.1340 EST)]

Hans Blom is:

        (1) An enthusiastic supporter of MCT
        (2) A nom de guerre of Bruce Abbott
        (3) A disturbance testing for the variables controlled by
                Bill, Martin, and Rick
        (4) All of the above
        (5) None of the above
        (6) Not enough information to tell
        (7) Not worth worrying about

All entries become the property of CSGNet. Decision of the
judges is final. In case of ties, duplicate prizes will
_not_ we awarded.

[Hans Blom, 970410b]

(Martin Taylor 970408 13:25)

3. The PCT single-dimensional reference level is abolished in favor
of a multi-dimensional one, offered to the minimizer. Thus an MCT
controller's objective might, for instance, be stated as finding
the minimum of

(x-xref)^2 + 2*(y-yref)^2 + 0.25*(z-zref)^2 + 5*u^2

where x, y and z are observations and u is one of the controller's
outputs. I do not see how this can be given a PCT translation,
because it would "confuse" PCT's hierarchical levels.

This isn't intended as a comment on the rest of your message, but is
to help you see how it could be given a PCT translation.

x, xref, y, yref, z, zref, and u must all be inputs to the
perceptual function of some control system. There's no problem with
this, since any signal anywhere potentially can be observed.

There _is_ a problem, and I've discussed it before: in the standard
(H)PCT model, the controller does not know what it does, i.e. it does
not observe u. Bill Powers has emphatically said so on several
occasions. Are you proposing an extension to the standard model?

In the "classic" PCT hierarchy, "-ref" variables are not perceived,
but there's no reason they should not be in any specific
configuration.

That would be another extension. It, too, would effectively abolish
the nicely layered structure of the hierarchy, I think, so I predict
that Bill Powers will veto you on this ;-).

The perceptual input function of this control system produces
(x-xref)^2 + 2*(y-yref)^2 + 0.25*(z-zref)^2 + 5*u^2 as its output.
This is compared with a reference level of zero, and the resulting
error signal is the input to the control unit's output function
(a leaky integrator, perhaps).

Yes, something like that ought to be able to do it.

That's a simple-minded PCT translation.

If one accepts the required extensions to the standard model. Do you?

But why bother, when x, y, and z are already being controlled to
match their reference values. You could effectively do the same job
by controlling for u = 0, which tends to conflict with the other
controlled variables being near their reference values.

That's fine as well. But this, too, is not included in the standard
(H)PCT model, where outputs are not perceived. And where the
hierarchy seems to have been invented to prevent exactly these types
of "conflicts", I hesitantly infer.

Are you asserting that the general [PCT] reorganization scheme has
been shown to fail? If so, in what way does it fail?

The PCT reorganization scheme has a very different emphasis than the
MCT scheme. In MCT, learning/optimization is climbing a gradient; if
there is no gradient, there is no way to go, and the MCT model is
stuck. Only "dual control" theory approaches the theme of "on-line
experimentation" by adding "optimal" random elements (a pretty
difficult concept :wink: to the controller's output. But this theory
about "optimal learning" is extremely complex and (thus) not yet
applied in any practical system, as far as I know. Here, PCT seems to
have an advantage, at least in theory. In practice it remains pretty
hazardous to generate outputs if you don't know where they will take
you (sorry for the MCT terminology ;-).

This is where I think a synthesis is required: if there is a
gradient, use it as efficiently as possible. If not, take random
reorganization steps. But only if you believe that a higher
performance can be achieved, i.e. when you believe that you're stuck
in a local maximum. If there is no higher peak anywhere, trying to
find it only results in inferior performance -- although "local
learning/exploration" is required if the peak's position may shift.
But if there is a higher peak, finding it will give a permanent, and
maybe large, advantage. At the risk of losing sight of your previous
peak. That may be why organisms often seem to explore "hesitantly"
from a well established home base, as dual control theory predicts
should be done -- in many cases.

No-one seems to have good answers to these questions yet, as far as I
know, for individuals, although genetic algorithms have provided some
of the answers for populations.

Greetings,

Hans

[From Bill Powers (970411.1329 MST)]

Hans Blom, 970407e--

Thus I now understand that when the observation is error free, the
MCT model is _exactly_ the PCT model at least in form. The internal
world-model is not used at all!

This is a misunderstanding. The output of the model-building part is
the model of the environment equation. In case of a perfect model, it
_is_ the environment equation. The responsibility of the control part
is then to substitute xref for x in the model (environment) equation
and solve for u. This is exactly what the controller does. So the
internal world-model is fully used...

This is not correct. You are not only using the exact _form_ of environment
equation and the exact _parameters_, but exactly the same _values of the
variables_, which is not valid.

I said, in trying to explain why this is an error,

Clearly, at the end of this iteration, the world-model's position
and velocity are different from the real position and velocity. That
difference can then be used to calculate a corrected disturbance for
the _next_ iteration, but nothing can be done about the _current_
iteration which has just finished. You can't go back in time and fix
it.

And you said,

Right.

Now, either you meant "right" or you were just brushing off what I said. If
you meant "right," then you were agreeing that the world-model's VARIABLES
have values different from those of the theodolite in the environment. If
you just substitute the values from the environment model into the
world-model, as you propose doing, you are arbitrarily, and illegitimately,
making the world-model's variables match those in the environment when in
fact they cannot match.

It is this difference between the world-model's variables and those of the
real physical system that leads to the difficulties I found in the way the
MCT model handles the disturbance.

I repeat: you are mistaken in assuming that because the equations and
parameters in the world model are an exact representation of the physical
situation, the _variables_ in the world-model must be the same as they are
in the real physical device. The world-model's variables must be calculated
using the world-model, not the computations of the theodolite's actual
position and velocity (which you do by saying x := xx and v := vv). The
critical event occurs when there is a disturbance applied to the real
theodolite, in the same iteration that the _old_ disturbance (simulated) is
being applied to the world-model. This leads to a difference between the
real velocity and the modeled velocity; indeed, it is this difference that
is used to correct the disturbance on the next iteration.

···

-----------------------------------

In PCT, it is still unclear what the controller's goal is. Goal is
usually identified with reference level, even if the reference level
cannot be reached. In MCT, the goal is specified explicitly -- it is
the minimization of a specific function.

I agree that this is a major difference between MCT and PCT. Martin Taylor
has discussed this difference quite clearly; I will leave it up to him to
deal with this subject.

Bill, I don't understand you. You always harp on more research to be
required on PCT, yet you have disregarded the enormous research done
on control. PCT is based on 1950's servo-control, which at the time
did not have an adequate theory behind it. Now it has, and it has
been shown to be one particular form of a far more generally
applicable class of control formalisms. One effect of the research is
that PCT or "servo control" is now much better understood -- in what
it can do and what not, in what its advantages and disadvantages are.

When I "harp" on more research, I'm speaking of investigating the behavior
of organisms to see what they can control and how they do it; I'm not
talking about developing more general control "formalisms" and then looking
for facts to fit them. I am satisfied that the PCT model as it stands today,
in all its intellectual poverty and decrepitness, gives us an adequate model
for the aspects of behavior we've been able to investigate so far, with the
resources that are available. No doubt we will some day encounter types of
behavior that can't be explained with the PCT model. When that happens, I
hope there will be more advanced models to choose from, and people capable
of applying them and willing to take the time to do so.

You still want to explain everything in terms of a theory that
everyone else in the field recognizes as inadequate and
underdeveloped.

Oh, pooey. Everyone else? I guess that means they must be right, eh?

And as a result of not knowing other methods and
their advantages, you do not see the disadvantages of PCT. The main
one being that, however cleverly _you_ can design what is now called
an "adaptive controller", you cannot come up with a general enough
scheme that will let the _controller_ (re)design or (re)tune itself.

Neither, as I recall, can you. There's not only the problem of "system
identification," which may be bravely discussed but is far from being solved
"generally," but there is the question of who tells the controller what
paradigms to use even for a successful system identification. The designer
is giving the controller a lot of help. This is fine for engineers designing
and building control systems, and more power to them, but is useless for
those investigating living systems, which have to do EVERYTHING for themselves.

But that problem has been solved a long time ago! And when I tell you
about some of these newer approaches you doubt every small step
beyond what you know, as if I and the whole world of control theory
attempt to deceive you. I find your paranoia utterly frustrating.

My paranoia is restricted to doubting what I see before my eyes if I see
mistakes in it. I think you grossly exaggerate the successes in your field;
but of course that is what proponents of an idea always do, don't they? I
don't mind that so much; what bugs me is the condescension.

Bill, what gives you the perception that I'm a blathering idiot?

What gives me this impression is your way of leaving out crucial
explanations, as above. It doesn't help anyone else if YOU know that
there is a w1 and a w2 being adjusted by some background process,
but fail to let anyone else in on this fact.

Bill, I'm just rehashing simple textbook stuff; I'm hardly secretive.
I've told you so before. I have tried to explain a thing or two, but
experience such an incredible resistance every time that no progress
seems possible. And I really would like you to become acquainted with
(modern!) control theory so we can talk at a more meaningful level.

Seems to me that you remarked a while ago that this MCT approach hasn't
caught on in the United States the way it has in Europe. So I wonder whose
textbooks you're talking about. Anyway, I'm willing to go on slogging
through the details if you are -- and the detail that needs looking at right
now is substituting the actual variables for the modeled ones. When you
correct that mistake we can go on through theoMCT6 -- unless you just want
to call the whole thing off.

Best,

Bill P.

[Hans Blom, 970415f]

(Bill Powers (970411.1329 MST))

The output of the model-building part is the model of the
environment equation. In case of a perfect model, it _is_ the
environment equation. The responsibility of the control part
is then to substitute xref for x in the model (environment)
equation and solve for u. This is exactly what the controller
does. So the internal world-model is fully used...

This is not correct. You are not only using the exact _form_ of
environment equation and the exact _parameters_, but exactly the
same _values of the variables_, which is not valid.

The _values of the variables_ that are used are the observations
(perceptions, feedback information). The MCT controller is a feedback
controller as well, you know ;-). And that the internal model's
variable values (its fallible predictions) are 100% corrected by the
observations (and thus effectively discarded) is due to the knowledge
of the controller that the observations are error-free, whereas its
predictions are not.

If you meant "right," then you were agreeing that the world-model's
VARIABLES have values different from those of the theodolite in the
environment.

This is right at the moment the controller makes its prediction
(called p in the program) of what the next observation is _going to
be_. The observation itself is not yet available, so the controller
only has its prediction to rely on.

If you just substitute the values from the environment model into
the world-model, as you propose doing, you are arbitrarily, and
illegitimately, making the world-model's variables match those in
the environment when in fact they cannot match.

But as soon as the new (noise-free!) observation _is_ available, it
is used, rather than the imperfect prediction. I have no idea why
that would be illegitimate...

It is this difference between the world-model's variables and those
of the real physical system that leads to the difficulties I found
in the way the MCT model handles the disturbance.

It is just the controller using the fact that the measuring process
is noise-free whereas its prediction process is not (due to the fact
that the disturbance is unknown and can only be estimated). Your PCT
controller is allowed to use that fact as well, if you can think of a
way...

The critical event occurs when there is a disturbance applied to the
real theodolite, in the same iteration that the _old_ disturbance
(simulated) is being applied to the world-model. This leads to a
difference between the real velocity and the modeled velocity;
indeed, it is this difference that is used to correct the
disturbance on the next iteration.

Correct. It is only when the true disturbance is different from its
assumed/modeled value that the controller's prediction p will differ
from the next observation x. This discrepancy (called the "prediction
error") is immediately used to adjust the model of the disturbance to
a new value. In fact, in a model-based controller it is _always_ the
prediction error that provides information about the quality of the
model and that is used to further adjust the model. That must be easy
to understand: if the model is perfect, it will always generate
perfect predictions. If the predictions are not perfect, the model is
not perfect and should be adjusted.

You still want to explain everything in terms of a theory that
everyone else in the field recognizes as inadequate and
underdeveloped.

Oh, pooey. Everyone else? I guess that means they must be right, eh?

This has nothing to do with being "right". It has to do with the fact
that modern control theory is effective in providing a more adequate
and detailed and better understood analysis, even of PCT-type
controllers. And thus of their advantages and limitations compared to
other types of controllers.

Anyway, I'm willing to go on slogging through the details if you are
-- and the detail that needs looking at right now is substituting
the actual variables for the modeled ones. When you correct that
mistake ...

Believe me, Bill, it's no mistake. It's called using the model error
to tune the model. And that is a kind of feedback process ;-).

Greetings,

Hans

[From Bill Powers (970415.1551 MST)]

Hans Blom, 970415f--

If you just substitute the values from the environment model into
the world-model, as you propose doing, you are arbitrarily, and
illegitimately, making the world-model's variables match those in
the environment when in fact they cannot match.

But as soon as the new (noise-free!) observation _is_ available, it
is used, rather than the imperfect prediction. I have no idea why
that would be illegitimate...

"Illegiminate" depends on what rules are being tacitly assumed in the
background. You're implying here that after the prediction is made, a
decision occurs as to whether to use the world-model's output or the
real-time observation. This decision is based on evaluating the noise in the
observation. And you are assuming that this decision always comes out in
favor of using the real perceptual input instead of the modeled input, and
takes no time to make.

I think the difficulty here would become much clearer if we just added a
little bit of observation noise to the system. Now, since the observation is
no longer "perfect," the world-model's output would have to be used instead
of or as well as the actual observation. Your model would have to be
equipped with some way of evaluating the input noise, assigning weights to
the real and modeled values of x, and combining the weighted values to
produce an effective value for comparison with the reference signal. I
suppose the combining would be done in the form of (w1*x + w2*x')/(w1 + w2),
or perhaps a quadrature average. These computations would become a necessary
part of the MCT model, further adding to its complexity and providing
further places where computational errors can occur.

Best,

Bill P.

[Hans Blom, 970422d]

(Bill Powers (970415.1551 MST))

I think the difficulty here would become much clearer if we just
added a little bit of observation noise to the system. Now, since
the observation is no longer "perfect," the world-model's output
would have to be used instead of or as well as the actual
observation.

I refer you back to an earlier demo of mine, where I did exactly
this. The same demo that also changed "the sign of the output
function", as you would say.

The problem with that demo was that it seemed far too abstract and
complex, and that I tried to demonstrate too many features at the
same time. With extremely limited success, alas. But if you check
that program once again, you will find that the model-x is _always_
different from the observation (there called y), even directly after
an observation, as long as there was what I called "measurement
noise".

Greetings,

Hans

[From Bill Powers (970422.2103 MST)]

Hans Blom, 970422d--

The problem with that demo was that it seemed far too abstract and
complex, and that I tried to demonstrate too many features at the
same time. With extremely limited success, alas. But if you check
that program once again, you will find that the model-x is _always_
different from the observation (there called y), even directly after
an observation, as long as there was what I called "measurement
noise".

OK, I'll take your word for it. I think I've explored the MCT model enough
to satisfy me for now that I understand it. I may run across phenomena that
suggest that the MCT model is necessary in a model of the behavior of living
systems. If so, I will probably ask for help in implementing it. However, I
will stick with Newton until it becomes obvious that it is necessary to call
on Einstein.

Best,

Bill P.

[Hans Blom, 970424g]

(Bill Powers (970422.2103 MST))

I think I've explored the MCT model enough to satisfy me for now
that I understand it. I may run across phenomena that suggest that
the MCT model is necessary in a model of the behavior of living
systems. If so, I will probably ask for help in implementing it.
However, I will stick with Newton until it becomes obvious that it
is necessary to call on Einstein.

Eminently reasonable point of view. To me, your understanding of MCT
implies that if I offer a different explanation you will at least
understand me, if not agree with me. That's good enough ;-).

Greetings,

Hans