MCT and PCT

[From Rick Marken (960726.1040)]

Hans Blom (960725b) --

If it [MCT] were incompatible, then HPCT would be wrong and I wouldn't be
here ;-).

It seems to me that Modern Control Theory (MCT) and HPCT are compatible only
when, as Bill P. says, we take a "birds-eye view" of the two theories. Up
close, MCT and PCT seem utterly incompatible: MCT seems to "aspire" towards a
view of behavior as computed output; HPCT is based on the view that all
behavior is the control of input.

Rather than continuing to muck around in the details of MCT and PCT, I
suggest that one good way to compare these theories is in terms of the
insights they provide about the nature of human nature. What would you say
are the essential insights about human nature that come from looking a people
from the MCT perspective? Why, according to MCT, do people have problems?
Why do people fight with one another? What can people do to get along with
others and themselves? I understand how PCT answers these questions and I
understand how these answers are derived from the PCT model of behavior. How
does MCT answer these questions and how are these answers derived from the
MCT model of behavior?

Best

Rick

[Hans Blom, 960726]

(Rick Marken (960726.1040))

It seems to me that Modern Control Theory (MCT) and HPCT are compatible only
when, as Bill P. says, we take a "birds-eye view" of the two theories. Up
close, MCT and PCT seem utterly incompatible: MCT seems to "aspire" towards a
view of behavior as computed output; HPCT is based on the view that all
behavior is the control of input.

It only seems that way. In HPCT, output is "computed" as well by the
internals of the control system. It is the "information" that is used
in the computations that is different. In MCT, the emphasis is on the
importance of "internal perceptions" (memory, or memory derived
variables) for control, which often far outweighs the importance of
our momentary perceptions. In other words: momentary perceptions are
interpreted in the light of the existing internal model. In HPCT,
model parameters are more implicit, but they are present also: in the
weight factors and the type of weighting in each perceptual input or
output function.

Rather than continuing to muck around in the details of MCT and PCT, I
suggest that one good way to compare these theories is in terms of the
insights they provide about the nature of human nature.

Some good questions. Let me try to give some preliminary answers. It
will not surprise you that the answers of MCT are not too different
from those of HPCT. It will not surprise you either that the emphasis
is different.

What would you say are the essential insights about human nature
that come from looking a people from the MCT perspective?

One: Learning comes before control. If learning does not succeed,
there can be no control. Babies. Schizophrenics.

Two: We are all different, because our circumstances (perceptions and
actions) have been different. Our "internal models", that say how we
perceive the world and what we can do, are partly innate (DNA pre-
scribed hardware), partly based on our individual history. Ask any
therapist.

Three: Uncertainties play a major role in how we view the world, but
much less in how we act. The reason is that the internal model is
much less constrained than our motor apparatus. A world view can be
crazy or underdeveloped, whereas the individual may still function
pretty well in society. Just look around ;-).

Four: There are a great many (sub)goals at any one time. Actions will
attempt to control for all of these goals at the same time, as far as
possible. This point of view pretty much dooms The Test ;-).

Five: All subgoals serve one topmost goal. The question is: what is
it?

Six: Control is best in a fully predictable world. Because we want to
control best, we create a predictable world, we want others to react
predictably, and we ourselves react predictably to others, whether we
want it or not.

Seven: Because the world is not fully predictable, it needs to be
explored continuously. Explorations reestablish the up-to-date-ness
of the model.

Eight: Exploration and control require different types of action:
random versus "computable". So some compromise is required. This
compromise depends on the "noise" that appears spontaneously in the
world. In times of war, we seek security and control. When things go
too smoothly we are bored and seek adventure.

Nine: Developing correct perception is much more important than
developing correct action. One might say that just observing a
problem from all angles solves it. Control (physical realization of
the want) is an afterthought. Ask any philosopher: they hardly _do_
anything.

Ten: There are no "unused" perceptions. _All_ perceptions tune the
model, although some may be discarded AFTER it has been established
that they cannot be fit into the model. "Unintended side effects" are
caused by a bad model. In a fully correct model, _all_ actions will
be fully intended and optimal and all perceptions fully foreseen.
Regrettably, our skull is not large enough to contain such an
extensive model. Human skull size has, however, increased quite a lot
over the last few million years.

Corollary of the above: All behavior is optimal. Either we learn from
it and are able to build a better model, or we achieve our goals.
Farfetched? It is what the theory says!

Another corrolary: Every second part of an optimal trajectory is
itself an optimal trajectory. The past does not matter, only in so
far as it determines the current model. In popular terms: today is
the first day of the rest of your life.

Why, according to MCT, do people have problems?

In a scientific theory, the word "problem" denotes something to be
solved. People have to solve the riddle of how to live optimally.
What is "optimal" is fully subjective. The topmost goal cannot be
chosen, lower level goals are idiosyncratic. Discovering "how to live
the good life" is a life long process. Its solution depends on dis-
covery of the "laws of nature" (and of culture) that allow matching
of wants and the results of actions (control). This is much like
finding (or creating) and decorating your "niche" ;-).

If the model (which captures our beliefs) is of a bad quality, some of
th wants (subgoals) contained in them may be pathological (ineffect-
ive). When people have "problems", their model is bad, either in that
they have arrived at wants that cannot be fulfilled or in that they
do not know how to achieve realistic goals. The solution to suffering
is in what Freud calls "reality testing" and in what Buddhists call
"enlightenment", which is much the same thing.

Beliefs are idiosyncratic. Bad (ineffective) beliefs originate in a
bad model. If bad beliefs lead to action, they do not solve but may
destruct.

Why do people fight with one another?

Because one person does not understand how he/she can "control" the
other. A person is like a rock or a chain saw or a chemical plant in
that he/she/it can be controlled only if we get to know how he/she/it
functions. The most important aspect of controlling someone else is to
recognize that they are control systems as well and that they are
also in the process of achieving _their_ goals. I believe that it is
always possible to create a win-win situation in every encounter.
Fighting is suboptimal behavior.

What can people do to get along with others and themselves?

Build better models. Get to know others better. Get to know yourself
better. Identifying the "inner goals" of others is not necessary. It
is sufficient to get to know what the other likes and detests. People
ordinarily give very clear signals about their likes and dislikes.

A major discovery at some point in the learning process is probably
that we are happy when someone else is happy. That perception and
reaction seems to be hard-wired into us. It requires, however, a
careful compromise with being _not_ happy -- and showing it -- when
we are not. If we forget this, we give inappropriate feedback. If we
overdo it, too.

I understand how PCT answers these questions and I understand how
these answers are derived from the PCT model of behavior. How does
MCT answer these questions and how are these answers derived from
the MCT model of behavior?

If I cannot "muck around" in the details, it is pretty difficult to
say how the theory leads to a world view. So I cannot answer the
"how" part unless I present the underlying theory.

Note, however, that the theory says that _my_ model was arrived at
through _my_ personal history. Anyone else's model will be different.
So you don't have to agree with me. Actually, you cannot, given your
different history. So what I say to you now can be no more than
offering text that you might certain perceive. It is up to your model
what to do with it.

Theory says that you have different options. You can consider this
text to be "observation noise". In that case you decide that it is
pure ununderstandable gibberish of which you cannot make any sense.
You can also consider that you might not want to incorporate any of
this into your internal model. In that case it is "system noise".
Although it exists, it is not worth modelling. The third alternative
is to (partly) update your model, because this info contains things
that help you build a better model.

I withdraw the above paragraph. Theory says that you don't have any
alternatives. You will react as your internal model prescribes...

How is this for a start?

Greetings,

Hans

[From Rick Marken (960726.1430)]

Me:

Rather than continuing to muck around in the details of MCT and PCT, I
suggest that one good way to compare these theories is in terms of the
insights they provide about the nature of human nature.

Hans Blom (960726) --

Some good questions. Let me try to give some preliminary answers. It
will not surprise you that the answers of MCT are not too different
from those of HPCT.

Thanks. Your answers reveal, more clearly than I could have imagined, the
enormous difference between the MCT and PCT views of human nature. I'll
write a detailed reply over the weekend.

How is this for a start?

Magnificent.

Best

Rick

[Martin Taylor 960816 11:20]

Hans Blom, 960726 and Rick Marken (960727.2100)

Sorry for the long delay. I didn't catch up on the mail backed up from the
previous trip before I went away for another couple of weeks, and there are
about 170 messages still awaiting perusal. But this interchange is worth
a comment, since I disagree somewhat (and agree somewhat) with both Hans
and Rick. I'll cast this in the same form as Rick's response to Hans.

Hans answered a series of questions about the relationship between model-based
control theory (MCT) and perceptual control theory (PCT), or more correctly
hierarchic perceptual control theory (HPCT). Without the "H", PCT needs
models in order to accommodate some of the results (see Marken's results
on the tracking of a simple sine wave, a year or two ago, in which he showed
that a sinusoidal reference signal improved the fit to actual tracking data;
this signal would have to come from a higher level control output).

In the following, I take HPCT and MCT both in their extreme forms, HPCT
involving no explicit models (though it is acknowledged that they probably
are used at the higher levels) and MCT involving no hierarchy (though it
is acknowledged that hierarchic models are probably more efficient than
single-level complex models). In HPCT, the environment _is_ modelled, though
only implicitly in the structure of linkages (outputs to lower-level
references, perceptual signals to higher-level inputs) and linkage weights.
The difference between the HPCT "model" and the MCT model is like the
difference between a neural network and an expert system. Nowhere in a
neural network (or an HPCT hierarchy) can one discover a specific
representation of any item of "knowledge", whereas in an expert system
(or an MCT model) each item of knowledge is overt and distinct.

···

----------------
Marken queries noted ">>", Blom responses ">".

What would you say are the essential insights about human nature
that come from looking a people from the MCT perspective?

One: Learning comes before control. If learning does not succeed,
there can be no control. Babies. Schizophrenics.

Learning has to be done largely by evolution, and is continued by the
individual. Many animals can walk (fly, or whatever) immediately on birth,
hatching, larval emergence... Only in humans is a quarter of the life-span
devoted to learning how to control effectively within the social-physical
environment. For MCT this means that the _data_ for the models must be
fed into the appropriate storage before birth; for PCT it means that a good
part of the _structure_ of the hierarchy is inborn.

It seems far more probable to me that biological systems have evolved to
grow structures like the structures of their parents than that they have
evolved both to grow structures like those of their parents to serve as
memories _and_ to fill those memories with the same data as filled their
parents' memories at birth.

(But that's an article of faith rather than of fact.)

Hans's statement is the same for MCT and HPCT, but there is a profound
underlying difference between the two in the implications of that statement.

Using Rick's scoring method, I'd say the similarity rating between MCT and
HPCT here is about 2 out of 10 (Rick's score, 5).

2. Two: We are all different, because our circumstances (perceptions and
actions) have been different. Our "internal models", that say how we
perceive the world and what we can do, are partly innate (DNA pre-
scribed hardware), partly based on our individual history. Ask any
therapist.

Here, Rick says:

The "internal models" in MCT are models of the
laws of physics, not perceptual representations of external variables. MCT,
therefore, has to predict that people who live in the same environment
would _have to_ build the same internal models of that environment or
they could not control at all.

This is incorrect, at least according to my understanding of MCT. The
"internal models" are models of what PCT calls "the environmental feedback
function," and possibly of the statistical characteristics of the disturbing
influences. Only, because there is no hierarchy, the environmental feedback
function is incredibly complicated in a one-level MCT system, so the models
must embody a _lot_ of situation-specific knowledge (such as the different
behaviours of the reaction to turning the steering wheel when the road is
(a) uniformly grey, (b) textured (i.e. gravelly), (c) dark and glistening,
(d) shiny (icy), (e) dark and matte....). In PCT, these situation-specific
differences in the environmental feedback functions show up as changes in
the reference values fed to the muscular systems from intermediate level
control systems that have reference values based on moderately complex
perceptions controlled at higher levels. Both MCT and HPCT systems that
have learned to control a car well on dark, matte roads are likely to crash
when first encountering a dark shiny (icy) road; both will probably perform
well by the hundredth such encounter (if they survive).

Similarity score 10. (Rick 5).

Three: Uncertainties play a major role in how we view the world, but
much less in how we act... A world view can be crazy or underdeveloped,
whereas the individual may still function pretty well in society. Just
look around ;-).

I'm afraid I don't understand this well enough to comment or to score. Rick
says:

It seems to me that uncertainties play the opposite role in PCT as they
do in MCT.

They play a different role, but I don't think it is an opposite role. In
both cases, control is better, the lower the uncertainty of the disturbance
given the recent history of the perception.

Just so that my score can be compared with Rick's, I'll give a half-way 5
(Rick 0).

Four: There are a great many (sub)goals at any one time. Actions will
attempt to control for all of these goals at the same time, as far as
possible. This point of view pretty much dooms The Test ;-).

Up until the last statement things were going along fine. Since we have
successfully done The Test over and over again, any theory that says The
Test can't be done has real problems. Anyway, MCT and PCT certainly seem
to differ on this matter.

Here I agree with Rick, but differ on the score. The fact that Hans
misunderstands The Test has no bearing on the similarity between MCT and
HPCT. Score 8 (Rick 0).

Five: All subgoals serve one topmost goal. The question is: what is
it?

PCT would say that many subgoals could (and probably do) serve several topmost
goals. We have also written models to show how simply and elegantly this
hierachical control of perception system can be implemented.

I'd say both statements are correct, except that the "topmost goal" seen
by Hans to be required is the evolutionary "goal" that one has in fact come
from an unbroken line of ancestors stretching back some 4 billion years.
The "goal" of passing on one's genes is not, however, a "goal" in the sense
the word in used in PCT discussions--the value of a reference signal. There
is no controlled perception of passing one's genes into a distant future.
One may hope for grandchildren, but that's not quite the same thing.

When one is comparing the individual control structures implied by HPCT
and MCT, Hans asserts that an MCT structure must have one single goal.
I'm afraid I don't see why, but have to accept his word. I don't see why
the MCT "goal" should not be vector valued, as is everything else in the
MCT system. Anyway, there is no limit to the number of top-level goals
in a HPCT structure, though there is a limit to how many can be simultaneously
satisfied (i.e. how many there can be without conflict).

Similarity score 0 (Rick 0).

Six: Control is best in a fully predictable world.

PCT and MCT agree here, but for different reasons. Still, the general
statement has to count as true for both theories.

Agree. Similarity score 10 (Rick 10).

Seven: Because the world is not fully predictable, it needs to be explored
continuously. Explorations reestablish the up-to-date-ness of the model.

PCT shows that no world model is necessary for control.

HPCT does not show this, as I mentioned at the top of this message. What
HPCT suggests is that no _explicit_ world model is necessary for control.
Rick has not addressed the question of exploration--a question that deserves
a whole long message or several, on its own. Without going into the reasons,
I believe that HPCT and MCT are very close here, though not identical.

Similarity score 8 (Rick 0).

Eight: Exploration and control require different types of action:
random versus "computable".

This is not at all like the PCT perspective. Exploration per se isn't
necessary to tune a world model becuase there is no world model.

I'm not sure what Hans means by "computable." As I understand MCT, the
building of a model requires both the ability to analyze the feedback
function and an ability to assess the statistical structure of the
disturbance waveforms. The first demands that control be relinquished
to some degree, "randomizing" the output _independently of the disturbance_
to see how the jittered perceptual signal correlates with the jittered
output; the second also demands that control be relinquished, at least
in the absence of a good model of the environmental feedback function,
so that the perceptual signal can be taken as a surrogate for the disturbance
(with good control there is very little correlation between the perceptual
signal and the disturbance).

Now, in an HPCT structure that has _not_ been well reorganized, control is
poor to nonexistent, and perceptual signals do correlate with disturbance
waveforms. When it has reorganized, and the perceptual input functions and
the output functions and linkages are such that control is good, there is
very little correlation between any of the perceptual signals and any of
the disturbances. It is during the period of poor control that reorganization
occurs; in Hans's terms, the output actions are "random" with respect to
the disturbances, which permits (MCT), or leads to (HPCT), learning
(reorganization).

Similarity rating 6 (Rick 0).

Nine: Developing correct perception is much more important than
developing correct action.

This makes no sense from a PCT perspective. There is no such thing as a
"correct" perception. In order to control, we have to construct perceptions
that can be influenced in all relevant dimensions.

In HPCT, the overriding "goal" is that all the "intrinsic variables" are
kept within their tolerance limits of the evolutionarily determined
reference levels. To do this, some perceptual signals are controlled,
and one of the facets of reorganization (or of evolutionary "learning")
is to vary the perceptual input functions until the perceptual signals
being controlled have the side effect of keeping the intrinsic variables
under control. Far from saying that "there is no such thing as a 'correct'
perception," HPCT _insists_ that "developing correct perception is much
more important than developing correct action." But this may not be the
sense in which Hans meant it. I assume that it is, and therefore give
a similarity rating of 10. (Rick 0).

Ten: There are no "unused" perceptions. _All_ perceptions tune the
model, although some may be discarded AFTER it has been established
that they cannot be fit into the model.

PCT suggests that some perceptions are controlled; some are not.

I think Rick is talking about something different from Hans. In reorganization,
HPCT does what Hans suggests. Perceptual input functions and linkages
change according to the success that perceptual control has in keeping the
intrinsic variables under control. But in the hierarchy, there are, as
Rick says, perceptions that are not controlled--many of them, at any
specific moment--even though those perceptual signals may well contribute
to higher-level perceptions that _are_ controlled. In Hans's sense, those
"uncontrolled" perceptions are _used_ by the hierarchy. In HPCT, reorganization
tends to eliminate perceptions that are not used, simply because a random
change that eliminates them will not reduce the stability of the intrinsic
variables.

Similarity rating 9 (Rick 0).

Corollary of the above: All behavior is optimal.

I don't think there is a conceptual equivalent to this statement in HPCT.
If there is, I either don't understand the statement or I don't understand
HPCT. In HPCT, behaviour is what it is. It may not affect the controlled
perceptions as efficiently as other behaviour might do for the same hierarchy
of perceptions, and therefore might not be considered as optimal by an
analyst. But what does that mean to the hierarchy that exists?

Similarity rating 0 (Rick 0).

Another corrolary: Every second part of an optimal trajectory is
itself an optimal trajectory. The past does not matter

Well, this is pretty vague but, I've gotta admit that PCT leads to
the same conclusion.

Again I don't understand Hans. It's that word "optimal". If "optimal" means
that the model of the environment is only as good as it is, and the
behaviour based on the model is the reference against which the optimality
of the behaviour is judged, then both this and the previous corollary are
tautologies. If "optimal" means something else, I don't know what that is,
except for evolutionary success. Some control systems leave no
descendants, so they are not "optimal" in that sense. But we all came
from ancestors who did leave descendants, so our ancestors must have been
"optimal", I suppose, even if we are not.

I agree with Rick here. It's vague, but I understand "the past does not
matter" to mean that memories of the past are actually in the present, in
which case we have another tautology. It doesn't matter to a tautology
which view is taken of a control system, so I agree with Rick in giving
a similarity rating of 10.

Rick:

Why, according to MCT, do people have problems?

Hans:

People have to solve the riddle of how to live optimally.

In HPCT, "problems" mean that some aspect of control is imperfect, whether
it be of intrinsic or of perceptual variables. Imperfect control is often
due to conflict between control systems, either within the same body or in
different bodies. But Hans claimed that all behaviour is optimal, so again
I don't understand where the "riddle" is in learning to behave optimally.

Similarity rating a dubious 5 (Rick 0).

Rick:

Why do people fight with one another?

Hans:

Because one person does not understand how he/she can "control" the
other. A person is like a rock or a chain saw or a chemical plant in
that he/she/it can be controlled only if we get to know how he/she/it
functions...Fighting is suboptimal behavior.

Apart from the last sentence, which again seems to conflict with "all
behaviour is optimal", one _can_ construe Hans's statement as agreeing
with HPCT if one wants to. One can also construe it as being in direct
opposition to HPCT. So it is hard to assign a similarity rating. On the
assumption that to "control" another person is to act so that the other
person's actions advance one's own goals, Hans's answer is almost an HPCT
answer. The main different is that in HPCT there is no part of the
hierarchy that "understands how" it works. The "understanding" is
distributed through the structure. When one's actions serve to advance
one's own goals, the structure is less likely to change than when they
don't. And when at the same time one's actions serve to advance another
person's goals, the other person's structure is unlikely to change. So
collaboration is stable whereas conflict is not. The two people's systems
come to "understand" how to "control" each other.

Maybe that's what Hans means. If so, similarity rating 10 (Rick 0). But
that may not be what he means, and the similarity rating could go as low
as zero.

Rick:

What can people do to get along with others and themselves?

Hans:

Build better models. Get to know others better.

That's very vague, but since the essence of MCT is the building of explicit
models, it's probably the best that can be done. In HPCT, the same effect
comes about by reorganization that happens more when conflict impedes good
control than when conflict is removed. Implicitly, then, the answers are
the same, since to "build better models" in HPCT translates into "reorganize
into a structure that controls better."

But, reversing field, Rick says:

PCT would say that this is
_exactly_ the wrong way to go. The first thing to do to get along with
people is to realize that people are control systems and that, therefore,
they cannot be controlled arbitrarily.

Rick is saying that PCT _requires_ the building of models. As I noted at the
top of this message, we acknowledge that models may well be used at the
upper levels of an HPCT structure, but in my similarity ratings I am
assuming that we have an extreme variant that does not use models. By his
own statement, Rick should have given a similarity rating of at least 8,
rather than the zero he did give. I give a similarity rating of about 7.

-------
Rick:

Ok. That's it for this group. There were a total of 15 answers; the
highest possible similarity score is, thus, 150. I gave MCT a total
similarity rating of 30 yielding an average similarity value of 2
out of 10 between MCT and PCT (assuming all answers are equally
important).

My similarity ratings total 90 to 100 out of 150, or an average of around 6
out of 10.

MCT may be the greatest thing since sliced bread (I'll take the sliced
bread, thanks) but it is DEFINITELY not PCT.

Nobody said it _was_ PCT, and I prefer to slice my own bread, thank you.
But the implications of MCT are not as different from those of PCT as
Rick makes out.

Then again, maybe I don't understand MCT very well.

----------------
I can see from the message headers that there were several subsequent
postings on this topic, but it seemed to me worthwhile presenting my own
answers independently of whatever anyone else might have said on these
various issues. Sorry if my comments turn out to have been redundant.

Now for the rest of the 200 backed-up messages.

Martin

[From Bill Powers (960826.0830 MDT)]

[Hans Blom, 960826] --

Hans, thank you for the summary of Prof. Eykhoff's views. They repeat a
number of points I have been trying to communicate. I learned much the
same progression of thoughts when I was learning analogue computing in
the 1950s:

1. The ideal compensator should have a transfer function that is the
inverse of the driven process's transfer function (or, in the frequency
domain, the complex conjugate of it).

2. An approximate inverse of a function can be computed by putting the
function into the negative feedback link of a frequency-compensated
operational amplifier.

3. The best possible model of the environment is the environment itself.

The history of control engineering from the Industrial Revolution to the
1960s is represented by the progression from Eykhoff's first diagram to
the last one: from

     u (t) ----------- ----------- y (t)
     ----->| Q (t) |----->| P (t) |----->
           ----------- -----------

to

     u (t) ----- ----------- ----------- y (t)
     ----->| |----->| inf |----->| P (t) |----->
         + ----- ----------- ----------- |
             ^ - |

···

                                      >

             -----------------------------------------

The first diagram represents attempts to compensate machines by
inserting either a series compensator or a parallel one affecting the
same output. Disturbances were handled the same way -- the temperature
effects on a pendulum were compensated by adding a bimetallic element
that moved a weight oppositely to the change in length of the pendulum,
or by adding a column of mercury with a higher coefficient of expansion
than the main pendulum that expanded toward the pivot. That was a
typical 19th Century approach to the design of precision mechanisms.
That's why they tended to be so big and heavy.

The second diagram represents the realization that all compensations
(other than dynamic) can be done away with if a sufficiently high loop
gain is used and if the sensor that detects the state of the output is
as precise as possible. H. S. Black realized in 1927 that this strongly-
fed-back arrangement also has two great practical advantages: it can
render unimportant even quite large changes in the properties of the
output function P(t), and it can greatly increase the bandwidth of the
response of the output to the input, while still maintaining the output
equal to the input (the reference signal, as you note). Furthermore,
unpredictable disturbances adding independently to the state of the
output are largely cancelled out, without any need to predict what they
are going to be.

The digital revolution had the effect, I think, of turning attention
toward digital computers and away from the lore of the 40s, 50s, and
60s. Digital devices are much easier to work with than analog devices.
They are the ideal input-output machines. There's no need to think in
terms of physical dynamics -- a stepper motor simply turns by the
commanded number of steps. The problem of components with variable
characteristics is done away with: there is only one chance in 1000, and
later 1 in 10^16 or so, of a command not being obeyed exactly correctly.
And Boolean logical is a hell of a lot easier to learn than differential
equations. So we ended up with a whole generation, and perhaps three, of
engineers most of whom "don't do analog."

I think that this trend toward infallible and exceedingly accurate
digital computations also encouraged a mathematical approach to control
in which practical considerations tended to be bypassed. For example,
you say

     One important example of this is "response improvement", where the
     output signal (what PCT calls the perception) ought to be as much
     like the input signal (what PCT calls the reference) as is
     possible, particularly -- in what is probably the most difficult
     case -- when the input signal makes a (step) change from one level
     to another one.

What I learned back in the mists of time was that you don't design a
control system to follow a step-change in the reference signal. While
the mathematical expressions show that if the compensator is the inverse
of the output function a step change will be perfectly reproduced, in
any practical case implementing a step change would require applying
infinite forces to the environment. Trying to approximate the ideal
step-response would result in ridiculous expense in designing infinitely
powerful and fast actuators, which would probably destroy the physical
device as soon as they were turned on. The best practical design
objective was an exponential approach to the steady state after a step
input, or slightly better, an approach with a single overshoot
("minimum-time damping").

As it turns out, reducing the requirement from reproducing a step change
to approximating the step change with a decelerating exponential makes
simple designs much easier. If the environment function is an
integrator, the controller needs only to be proportional. That is, the
controller does not act like the inverse of the environment function,
but, as it were, contains one derivative less than the inverse. The
result is not a perfect response to a step-input, but a transition to
the new state along an exponential curve. The rise-time of the
exponential can be made as short as practically possible, so that real
actuators can follow it without generating destructive driving signals
or forces.

However, I don't want to second-guess modern control engineers. It's
quite possible that the advent of the fast, cheap, and reliable digital
computer has made possible a new approach to the design of automatic
control systems which can be governed by exact mathematics. I harbor the
suspicion that modern control engineers may have forgotten some of the
useful early lore that existed prior to the digital revolution, but
that's neither my business nor my interest.

My interest is in modeling living control systems which must live in a
vast environment full of surprises and unpredictable events. They must
operate not through infallible stepper motors, but by using soft tissue
that can both change in bulk and fatigue with use. The computations they
perform have to be done with chemical reactions inside neurons, and at
the lower levels of organization have a repeatability (I would guess) of
typically about 5%. They can't compute sines and cosines, or do long
division or floating-point multiplication and addition. They can embody
input-output functions, but they can't know that they are doing so, and
can't calculate the inverses of those functions. They are not digital
computers; they offer a very poor platform on which to implement modern
control system designs.

Yet modern control engineering, which turns out not to be quite as
modern as I had imagined, does offer some ideas to PCT, as I have noted.
I believe that higher organisms do sometimes use internal models, but I
don't believe that the internal models that a living control system may
use at the higher levels are nearly as detailed as those in your Kalman-
filter approach, nor that the _primary_ mode of operation is model-based
at any level of organization. I think that a living control system has a
modular and hierarchical design, and does not work as one huge
computation. And I think that may be some knowledge built into the
system, but much less than you assume, concerning disturbances. I think
that the basic unit of design is a simple negative feedback loop in
which real-time, not imagined, perceptions are the primary target of
control and in which specific input signals, not model outputs, are
compared in real time against specific reference signals.

These are the conclusions I currently hold about the relation of MCT to
PCT, in connection with models of organisms. I could be wrong (I can
remember a time in 1977 when I was wrong), but so could your propsals
based on MCT be wrong. If we can ever get PCT into the form of an
experimental science, we will both probably find that we were just
guessing.
-----------------------------------------------------------------------
Best,

Bill P.

[From Rick Marken (960828.0800)]

Bill Powers (960927.1330 MDT) --

Thanks for posting the diagram of the MCT model. At some point in this
discussion of MCT vs PCT someone suggested that the MCT model could
survive a change in the sign of the feedback function because, ofter all,
the "world model" (F) is a model of the feedback function (F(real)) and that
function includes a sign.

While thinking about this in the shower this morning (yes, it's true; I am
ALWAYS thinking about PCT) it occurred to me that this is not necessarily
true. If the sign of the feedback function (F(real)) went from positive to
negative, I believe that y and x' would start to diverge exponentially. Based
on the results of our polarity reversal study (done with living and
simulated control systems), where we actually did change the sign of the
feedback function, it seems to me that this divergence would occur
because y acts as though there were an _increase_ in the coefficient
(amplification) of the feedback function, not a reversal of its sign. So it
seems to me that the Kalman filter algorithm would act to bring y into a
match with x', _not_ by changing the sign of the "world model" (F), but by
increasing the amplification coefficient of the world model, to get x' to
match the exponential "runaway" of y. This would produce an exponential
runaway, just like the PCT model.

I wonder if you could check this out with the copy of Hans' model that you
have? If not, I think I still have a copy and I'll try to get to it later
this week. I think it is quite possible that the MCT model will behave much
like the the PCT model in the polarity reversal situation. This would be
interesting because it would show that the MCT model has no advantage over
the PCT model in a situation where one might think that it would.

Best

Rick

[From Bill Powers (960927.1330 MDT)]

Martin Taylor 960827 13:20

When the perceptual signal is not just a simple copy of the external
controlled variable, the internal world-model must not only reflect the
properties of the environment, but the properties of the perceptual
function, too.

    I'm not clear whether you are talking in general here, about MCT, or
    about PCT.

I see from your discussion that I didn't make myself clear. Here is a
diagram of the MCT model (you can consider that all variables are mucho-
complex vectors):

···

ref sig

                                  -------
                                  > Finv |
                                   ------
                                     >
                         --- |
               x'-<-----| F |<-------|
                > --- |
                > > >
                 \ ------- |u

                 --| Kalman| |

                 / ------- |
                > >
                > >
                y |
                > --------- |
                x-<---| F(real) |<---
                       ---------

The function F in the top center, above "Kalman", is the world-model
that represents F(real) in the environment, below. The reference
signal acts through the inverse of that function, labelled Finv. As
the reference signal varies, u varies and drives both the world-model
and the real external plant function. The real controlled variable x
is supposed to follow the reference signal. This is accomplished
through the Kalman Filter, which adjusts F until its output x' is
following x exactly. Each time that a parameter in F is adjusted by
the adaptive algorithm, the same parameter must be adjusted to the
same value in the inverse function Finv. When x matches x', it is
necessarily true that x = ref signal, because of Finv. What you end up
with is

ref signal ---> Finv -----u ------- F(real) --------x ( = ref)

In my post I was referring to the fact that there must be two
computing networks, one for F and the other for Finv. Furthermore, the
parameters of Finv must always correspond to the parameters in F;
this method depends on their being always the same. I was saying that
I find it difficult to see how, neurologically, the two networks could
come to have the required inverse relationship, and how the parameters --
synaptic weightings -- in the F network could be transferred continually
and accurately to the corresponding parameters in the Finv network.

Following this, I talked about the fact that we really need a perceptual
function between y and the environment, and that this, too, has to be included
in the world-model F (and its inverse), because then x would be the output
of the perceptual function of the plant output.

Does this make more sense of my comments?
-----------------------------------------------------------------------
Best,

Bill P.

[Martin Taylor 960828 16:15]

Bill Powers (960927.1330 MDT)

Martin Taylor 960827 13:20

When the perceptual signal is not just a simple copy of the external
controlled variable, the internal world-model must not only reflect the
properties of the environment, but the properties of the perceptual
function, too.

   I'm not clear whether you are talking in general here, about MCT, or
   about PCT.

I see from your discussion that I didn't make myself clear. Here is a
diagram of the MCT model...

Does this make more sense of my comments?

Yes, much.

I'll have to leave it to Hans to judge whether the comments themselves make
sense. But I'm a bit confused about where an inverse function (a matrix
operator, presumably) would or could have parameters that correspond one
to one with the parameters of its direct function. Maybe I just don't
understand it well enough.

But then, Hans has said that the quasi-inverse need only approximate an
inverse, and it seems fairly straightforward for a neural net to use the
difference between x' and y to train the inverse in the same way it trains
the direct function.

I guess I should shut up, because I'm obviously misunderstanding either
the problem or the solution.

Until next week.

Martin

[From Bill Powers (960828.2130 MDT)]

Martin Taylor 960828 16:15 --

I'm a bit confused about where an inverse function (a matrix
operator, presumably) would or could have parameters that correspond one
to one with the parameters of its direct function.

In Han's challenge model, the external feedback function, the plant, was
represented as

x = k*u + d (forward model)

The inverse of that function, used to transform the reference signal r into
the driving variable u, was found by substituting r for x and solving for u
as follows:

r = k*u + d

r - d = k*u

u = (r - d)/k (inverse model)

You will see that the parameter k in the inverse is the same parameter that
is in the forward version. In more complex world-models, the same principle
is applied. The value of r is substituted for x, and the equation is solved
for u. The same parameters necessarily appear in both forms of the equation.
The Kalman Filter method adjusts the parameters in the forward form of the
world-model. Because, in a computer program, those are simply stored values,
the same values are used when the same symbol is encountered in the inverse.
Easy to do in the computer program; more difficult in a nervous system.

···

--------------------------------------------------------------------------
Best,l

Bill P.

[From Rick Marken (960903.0830)]

Hans Blom (960902) --

one can demonstrate that the gain (or "slowing function") in the output
path (of a PCT controller) must have properties very much like a ("dynamic")
inverse if control is to be of good quality.

Demonstate away.

How will the controller behave when the sign of the loop gain changes?

See "Mind Readings" pp. 109 - 132

By the way, Bill sent me a copy of your MCT model that includes a change in
the sign of the "loop gain" in the middle of a run. Unlike the human
controller, the MCT controller is completely unphased (so to speak;-)) by the
sign change. While the human response to such a sign change is a 1/2 second
exponential runaway (see the "Mind Readings" reference above), the MCT model
shows virtually not response at all. The gross mismatch between MCT model and
human behavior is rather convincing evidence that the MCT model is not the
appropriate model of human controlling.

Hans Blom (960903) --

The philosophical difference seems to be that PCT accepts a priori
that part of reality is incomprehensible and cannot and need not be
modeled.

No. PCT accepts (based on observation, not a priori) that all we (modelers
and modeled alike) can ever know of reality is our perceptions thereof.

PCT therefore seems not to attempt to create a better understanding of what
we call the environmental function.

Physics is about creating better models of "the environmental function". PCT
does provide, however, a model of how the physicist develops and tests models
of this function.

Hans Blom (960903d) --

Collecting only GOOD data implies to me that you only want to collect data
that confirm a model that you already have. That is bad science.

Yes, that IS bad science. Collecting GOOD data actually means collecting data
under conditions that _approximate_ as best as possible the conditions
assumed by the model under test.

There is nothing to be ashamed of if a model has limitations.

On this point you are surely an example to us all.

Best

Rick

[Hans Blom, 960904]

(Rick Marken (960903.0830))

one can demonstrate that the gain (or "slowing function") in the
output path (of a PCT controller) must have properties very much
like a ("dynamic") inverse if control is to be of good quality.

Demonstate away.

Mind if I don't and quote an authority? Probably not satisfactory to
you ;-). But I quote anyway, from Tsypkin's "Adaptation and Learning
in Automatic Systems (Academic Press, 1971). In his introduction to
the chapter on control, he warns the reader "This chapter is the most
complicated and therefore the most interesting one in this book",
also saying that "The reader will find that the theory of sensitivity
plays an important role in the realization of adaptive control
systems ...".

Rather than explaining the theory of sensitivity (which Tsypkin does
in earlier chapters), I just quote this from his introduction to the
chapter:

"It was noticed long ago, first in radio engineering and then in
automatic control, that an increase of the ... gain coefficients in
the closed loops can reduce the influence of the external
uncontrollable disturbances and variations in the characteristics of
the controlled object (plant characteristics). Many results of
control theory are related to this fact." ...

"Why then cannot such an effect of strong negative feedback sometimes
remove the influence of external disturbances and plant variations?
The answer to this question has been known for a long time. The
remarkable properties of strong negative feedback can only be used
when the stability of the closed loop systems is guaranteed
simultaneously with the increase of the gain coefficients. However,
this cannot always be achieved. The presence of time delays,
inertias, and occasionally of nonlinearities [either in the "plant"
or in the controller; JAB], does not permit an unlimited increase of
gain coefficients since that can destroy the stability of the
system."

So, in practice, the gain has an upper bound, due to stability
problems. It also has a lower bound: if the loop gain becomes too
low, control becomes ineffective. Between upper bound and lower
bound, there is an optimum somewhere. If the performance curve, whose
extremum is the optimum, is not a sharp peak, then we are lucky: a
wide range of gains will work. If not, then not.

Practice shows that the performance curve is usually asymmetrical: it
gradually slopes upward if the gain is increased (therefore a high
gain is nice) but it decreases abruptly where instability starts to
occur (a too high gain is catastrophic).

This has a practical consequence: pick a "safe" gain, relatively far
from the optimum, and control performance will not change much if the
plant characteristics change. But if you pick a gain close to the
optimum, the safety margin is much reduced. This explains why, unless
the plant can be characterized very well, optimal control usually is
adaptive: the peak of the shifting performance curve must be tracked
by the controller.

A test is simple. Just introduce a time delay into the environmental
function of any controller that you have lying around. Manipulate the
controller's gain, from low to high, and check whether control is
stable and, if it is, how good it is, at every control gain setting.
If you compute the sum of squares of the errors, this will give you
the shape of the performance curve and the optimal gain.

Now lower the gain of the environmental function by some factor and
repeat the above: the optimal control gain will have gone up by the
same factor. If this simple demo does not convince you, nothing will,
I guess ;-).

But Tsypkin continues: "Without an increase in a priori information
[about the "plant" to be controlled], it is not possible to increase
the gain coefficients and not affect the stability of the system at
the same time. For some reason, this is not often obvious. However,
with the larger gain coefficients, certain parameters and
nonlinearities which were not considered earlier in the analysis of
the system could become more apparent. But if they are unknown, then
the boundaries of stability could easily be crossed." This is often
not apparent in a computer simulation, where these small "plant"
imperfections are usually disregarded. Going from a nicely working
simulated arm to a real one may require more work than the initial
design.

Regrettably, Tsypkin continues, the required a priori information is
often not available. And he proceeds to show a number of algorithms
that collect that information _while control goes on_. That is,
basically, what adaptive control is.

How will the controller behave when the sign of the loop gain
changes?

See "Mind Readings" pp. 109 - 132

I have thus far not been able to lay hands on this PCT classic. Even
our library cannot find it. Is it -- or the article -- available on
the net somewhere?

By the way, Bill sent me a copy of your MCT model that includes a
change in the sign of the "loop gain" in the middle of a run. Unlike
the human controller, the MCT controller is completely unphased (so
to speak;-)) by the sign change. While the human response to such a
sign change is a 1/2 second exponential runaway (see the "Mind
Readings" reference above), the MCT model shows virtually not
response [error? JAB] at all. The gross mismatch between MCT model
and human behavior is rather convincing evidence that the MCT model
is not the appropriate model of human controlling.

That may be a scaling artifact. If you look more closely, you see
that the MCT controller initially controls in the wrong direction as
well. But after a short number of observations it is back to par. I
wouldn't know how to put a time scale along the "abstract" x-axis of
my demo.

The philosophical difference seems to be that PCT accepts a priori
that part of reality is incomprehensible and cannot and need not be
modeled.

No. PCT accepts (based on observation, not a priori) that all we
(modelers and modeled alike) can ever know of reality is our
perceptions thereof.

And our "memories" (however processed before storage) of those
perceptions, maybe?

Greetings,

Hans

[Hans Blom, 960904]
RE: change of feedback sign in Hans' model:

That may be a scaling artifact. If you look more closely, you see
that the MCT controller initially controls in the wrong direction as
well. But after a short number of observations it is back to par. I
wouldn't know how to put a time scale along the "abstract" x-axis of
my demo.

Actually, the MCT controller reverses the sign of k immediately, before even
one more iteration is run. I put in the sign reversal in the middle of the
run, and then stepped through the instructions one by one starting at the
instruction that reverses the sign. The reversal occurs immediately because
the sign of the error, (y - x), suddenly reverses and the error becomes
large. This results in a large correction to both k and d in the two
instructions immediately following the point where the reversal was done.
The MCT model never makes a move in the wrong direction, although following
the reversal there are a few minor corrections in successive iterations.

This is in contrast to a human subject, who starts a runaway which follows a
positive exponential curve quite closely for about 0.4 seconds (sampled 25
times per second, I believe).

The change I made was in about line 134:

if i < (maxtable div 2) then
x := kk*u + d1[i]
else
x := -(kk*u + d1[i]);

···

----------------------------------
Your discussion of loop gain and stability was interesting; it might be new
to some readers on this net. In Tom Bourbon's E. coli adaptive system, the
adaptation adjusted the output gain to keep the mean squared error as small
as possible (on the average). If the gain was too high (there was a
perceptual delay in the model, I believe), the system would tend to become
unstable and the gain would be reduced; if too low, disturbances would begin
to have a larger effect on the error signal, and the gain would be raised.
The result was that approximately the optimum gain was maintained by the E.
coli type adaptation.

Best,

Bill P.
---------------------------------------------------------------------------

[Hans Blom, 960910d]

(Bill Powers, 960904)

... the MCT controller reverses the sign of k immediately, before
even one more iteration is run. I put in the sign reversal in the
middle of the run, and then stepped through the instructions one by
one starting at the instruction that reverses the sign. The reversal
occurs immediately because the sign of the error, (y - x), suddenly
reverses and the error becomes large. ... The MCT model never makes
a move in the wrong direction, although following the reversal there
are a few minor corrections in successive iterations.

This is in contrast to a human subject, who starts a runaway which
follows a positive exponential curve quite closely for about 0.4
seconds (sampled 25 times per second, I believe).

It might still be a (time) scale artifact. The human controller has a
built-in time delay of about 0.5 seconds, I guess, the MCT controller
doesn't. This can be tested: to a human, offer the feedback
information at, say, 1 second intervals; this would make the delay
time negligible to the time within which a reaction is required.
Alternatively: give the MCT controller a delay of 0.5 seconds and see
how the behavior changes. I'll try to find time to do the latter.

Your discussion of loop gain and stability was interesting; it might
be new to some readers on this net. In Tom Bourbon's E. coli
adaptive system, the adaptation adjusted the output gain to keep the
mean squared error as small as possible (on the average). If the
gain was too high (there was a perceptual delay in the model, I
believe), the system would tend to become unstable and the gain
would be reduced; if too low, disturbances would begin to have a
larger effect on the error signal, and the gain would be raised.
The result was that approximately the optimum gain was maintained by
the E. coli type adaptation.

I hope that this makes clear why I say that (in a certain sense) a
controller must be the inverse of the world in which it lives. Even a
PCT controller...

Greetings,

Hans