Two architectures

[From Bill Powers (970129.1950 MST)]

In a post earlier today I described the MCT model as using a closed-loop
control system to adjust an internal model so as to make the controlled
variable x match the specification, r. This is not correct. The only closed
loop in the MCT model is the one that adjusts the parameters of the internal
world-model so its behavior is the same as the behavior of the external
plant, under the same driving signal u. The reference signal r is never
compared with the result x, so there is nothing to assure a match.
The result of the actual process is to make the forward world model's
parameters correct. Those same parameters are then used in the inverse of
the world-model, to transform r into u. So the only closed loop process ends
up assuring that the parameters in the inverse function match those of the
external plant as well as possible. There is nothing in this model that
checks to see whether x actually follows r. The relationship between r and x
is strictly open loop. If the inverse of the world model is calculated
incorrectly or inexactly, by a small amount or a large amount, x will fail
to match r, and no corrective action will be taken. The system has no way of
knowing that there is a mismatch; it works by blind reliance on the ability
to compute a correct inverse of the plant function, with no way (of its own)
to check the correctness. Only an external observer could detect that the
system is not working properly. And the properties of the external observer
are not part of this model.

In the PCT model, the basis for action is a direct comparison of x (as
perceived, but that holds for both models) with r. The system, which can be
adaptive, continues its action until x matches r; the error signal indicates
at all times the degree of mismatch, and serves as the basis for reducing
the mismatch.

It's obvious to me now that MCT and PCT are completely different
architectures, so different as to be mutually exclusive. As models of living
systems, they represent opposing interpretations. They can't both be right.

I don't care whether 2 or 2000 theoreticians and engineers think MCT is the
right way to go, as a way of controlling anything. I could be right and all
of them could be wrong. So I am going to stick to PCT.

Best,

Bill P.

[Hans Blom, 970130b]

(Bill Powers (970129.1950 MST))

In a post earlier today I described the MCT model as using a
closed-loop control system to adjust an internal model so as to
make the controlled variable x match the specification, r. This is
not correct.

Going up a level: Does this mental switching back and forth indicate
that you're still in the learning phase, where you think you know but
suddenly realize that you don't?

The following shows that you don't understand MCT yet. You have a
good grasp of the parts by now, but no idea yet what their _combined_
function does. You do thought experiments, not real experiments which
would immediately falsify your words. I compliment you on your
rhetoric -- as do others -- but your words are thoroughly misleading.
You still haven't understood the intimate coupling between building
the internal model on the one hand, and control on the other. These
two are fully intertwined in a way that apparently still evades you.
This is how they interact:

1: Control is bad if the error (called the prediction error: where I
want to be compared to where I find myself after taking a step; in
PCT jargon: reference level minus perception) is large. The model is
constantly automatically retuned so that it improves -- until, of
course, no further improvement in the quality of control (measured in
terms of the sum of squares of the prediction errors) is possible
(within the constraints of the model). The goal of (re)tuning is
minimization of the error, i.e. good control -- in fact, the best
possible control.

2. If the model is bad, control will be bad. Continue at 1. Result:
the model will be improved, and its parameters will settle at exactly
those values where the control quality is optimal.

1 and 2 together can be seen as an overall (control or minimization)
process, whose result (after model convergence) is twofold:

a) control is as good as it could possibly be (within the constraints
of the model), and

b) the best possible (with respect to this particular control task)
internal model of the environment is available.

This you understand, partly:

The result of the actual process is to make the forward world
model's parameters correct.

This ought to be qualified (what exactly is "correct" if we cannot
know the "real world"?), but is basically correct. Let me point out
your mistakes:

There is nothing in this model that checks to see whether x actually
follows r.

That you don't _see_ it doesn't mean it's not there. Let's say its
presence is implicit ;-). Anyway, x actually does follow r. The how
is explained in one of yesterday's posts.

The relationship between r and x is strictly open loop.

No. Not as long as perceptions are available. Only when the feedback
loop is cut -- when walking in the dark.

If the inverse of the world model is calculated incorrectly or
inexactly, by a small amount or a large amount, x will fail to match
r, and no corrective action will be taken.

Incorrect. Corrective action _will_ be taken. It seems to me that you
invert the relative priorities. Remember that the ultimate goal of an
adaptive controller is to _control_: to minimize the difference
between setpoint and perception. It might even be that the model is
incorrect ("biased") in the objective sense: its parameters do not
match those of the "real world"; this is sometimes found in
simulations where the real world _is_ known. It is not, however,
incorrect in the subjective (control) sense: it picks its parameters
such that it controls best. [On rereading this, it seems as if I'm
talking psychology!]

The system has no way of knowing that there is a mismatch; it works
by blind reliance on the ability to compute a correct inverse of the
plant function, with no way (of its own) to check the correctness.

What is correctness? If the "real world" cannot be known, how can we
know about a mismatch [between the world and its model; or between
the inverses of both]? The system's overall correctness check --
actually, its goal -- is to process the prediction error (setpoint
minus perception; where we want to be compared to where we are) in
such a way that this error is minimized. [I can think of no better
operational definition of correctness in the face of the fact that
the "real world" is unknown]. The "mere fallout" of this minimization
(control) process is an internal model.

Only an external observer could detect that the system is not
working properly.

Wrong. "Working properly" is, for a controller, identical to having a
minimal prediction error.

Bill, you ought to do some _real_ experiments. They are so much more
reliable than thought experiments...

Greetings,

Hans

[From Bill Powers (970130.0500 MST)]

Hans Blom, 970130b --

Going up a level: Does this mental switching back and forth indicate
that you're still in the learning phase, where you think you know but
suddenly realize that you don't?

Always. Frequently, anyway.

the internal model on the one hand, and control on the other. These
two are fully intertwined in a way that apparently still evades you.
This is how they interact:

1: Control is bad if the error (called the prediction error: where I
want to be compared to where I find myself after taking a step; in
PCT jargon: reference level minus perception) is large. The model is
constantly automatically retuned so that it improves -- until, of
course, no further improvement in the quality of control (measured in
terms of the sum of squares of the prediction errors) is possible
(within the constraints of the model). The goal of (re)tuning is
minimization of the error, i.e. good control -- in fact, the best
possible control.

The "error" of which you speak, as I understand it from your programs and
posts, is x' - x, not r - x, where x' is the output of the world-model and x
is the output of the plant as sensed. In translating from PCT to MCT, you
have misidentified the error signal. In PCT the error signal is always r - x
(or the perceptual equivalent of x).

The world-model is tuned so it responds to u in the same way that the real
plant responds, as nearly as possible. Is that correct? This could be done
even if u were being supplied by a test-signal generator: the EKF process
doesn't know where u is coming from. If u contains all the frequency
components that it will contain during actual operation, any test-waveform
will do. That is because the criterion of fit is not that x shall match r,
but that x' shall match x.

2. If the model is bad, control will be bad. Continue at 1. Result:
the model will be improved, and its parameters will settle at exactly
those values where the control quality is optimal.

The world-model can be good but control can still be bad if the inverse is
not correctly computed. You have said that "the inverse is no problem." From
the mathematical point of view, perhaps not (except where it doesn't exist
or is near a singularity). If the forward world-model is x = W(u), then the
inverse is just u = W^-1 (x), defined so that W * W^-1 = I, the identity
transform. Then we will have x = r * W^-1 * W = r. In terms of symbolic
computations, that is necessarily true.

But in a real system, the inverse must somehow be derived from the forward
form of the world-model and must be computed in a physically-separate place
(using different neurons in a brain model, different program steps in a
computer program). The forward model's parameters may indeed settle at
exactly the right values, but these values are "right" only in terms of
making x' match x. If there are difficulties in constructing the inverse, it
does not follow that x = r. Since the model that you presented contains no
provision for checking that x actually does equal r, and therefore no way of
correcting the difference if it exists, the model as a whole will not
necessarily settle to the state where x = r, but only to the state where x'
= x. Please correct me if I'm wrong about that, and show me the mistake in
my reasoning. Where is the guarantee that the _calculated_ inverse will be
the same as the _actual_ inverse?

You attempt to address this question:

There is nothing in this model that checks to see whether x actually
follows r.

That you don't _see_ it doesn't mean it's not there. Let's say its
presence is implicit ;-). Anyway, x actually does follow r. The how
is explained in one of yesterday's posts.

But this doesn't address the problem I have raised, which is that of
assuring that the calculation of the inverse produces the actual inverse of
the world model. The inverse may in fact _exist_ in the sense that
mathematically you can show that it involves no singularities. But it
doesn't follow that it is computed correctly. There is no "EKF" process like
that used to correct the forward model. You're speaking as a mathematician,
not as an engineer.

I understand how x follows r if the inverse is correctly calculated. But it
seems to me that you're taking that for granted, even though you recognize
that the _forward_ model requires some process to adjust it to fit the
behavior of the plant. Why do you not then need a comparable process to
assure that the putative inverse calculation actually produces the correct
inverse function? By your mode of argument ("let's say it is implicit"), you
could equally well say that the correct forward model of the plant is
implicit, and do away with the tedious EKF process for adjusting its
parameters. Why not just assume the correct parameters to begin with?

The problem is that you can't find the inverse just by running a signal
backward through the network that computes the forward model. You need a
separate network, differently organized, to compute the inverse. That that
raises the question of how the system can know that the second network is
actually doing a computation that is the inverse of what the first network does.

The relationship between r and x is strictly open loop.

No. Not as long as perceptions are available. Only when the feedback
loop is cut -- when walking in the dark.

If the inverse of the world model is calculated incorrectly or
inexactly, by a small amount or a large amount, x will fail to match
r, and no corrective action will be taken.

Incorrect. Corrective action _will_ be taken.

I disagree. Even with an incorrect inverse calculation, _some_ values of u
will be generated as r varies, and the EKF process will make x' match x. All
that the adaptive process can do is to make sure that given u(t), whatever
it is, the world-model responds the same way as the real plant does.

It seems to me that you
invert the relative priorities. Remember that the ultimate goal of an
adaptive controller is to _control_: to minimize the difference
between setpoint and perception.

That's irrelevant. You're only describing what you _hope_ the model will do.
It may not in fact do that. You're confusing the abstract _existence_ of a
mathematical inverse with the successful computation of that inverse. The
operation of the system you describe will make x = r ONLY if that
computation is correct. But there is no check on its correctness.

It might even be that the model is
incorrect ("biased") in the objective sense: its parameters do not
match those of the "real world"; this is sometimes found in
simulations where the real world _is_ known. It is not, however,
incorrect in the subjective (control) sense: it picks its parameters
such that it controls best. [On rereading this, it seems as if I'm
talking psychology!]

You see? You're still assuming that if an adequate forward world-model
exists, the inverse must necessarily be computed correctly. Of course when
you devise computer models, you make sure that the inverse equation is in
fact the correct inverse. If you made a mistake, you would soon see that x
is not following r correctly, and you would fix the mistake. But the model
does not directly compare x with r; it cannot know that a mistake was made,
so it can't correct the mistake. The EKF process can't correct the mistake,
because it doesn't know the origin of u; it will simply make x' match x no
matter how u behaves. If the inverse is not computed correctly, then x' will
match x but neither one will match r.

The system has no way of knowing that there is a mismatch; it works
by blind reliance on the ability to compute a correct inverse of the
plant function, with no way (of its own) to check the correctness.

What is correctness?

Correctness is whatever form will result in x = r.

If the "real world" cannot be known, how can we
know about a mismatch [between the world and its model; or between
the inverses of both]?

Easily. You simply use a comparator that compares the (sensed) value of x
with the value of r. Since that is the match you wish to accomplish, a
mismatch will be indicated by r - x being different from zero.

The system's overall correctness check --
actually, its goal -- is to process the prediction error (setpoint
minus perception; where we want to be compared to where we are) in
such a way that this error is minimized. [I can think of no better
operational definition of correctness in the face of the fact that
the "real world" is unknown]. The "mere fallout" of this minimization
(control) process is an internal model.

Yes, exactly, but your model has no way of detecting THAT "prediction
error." It never compares x directly against r. It only compares x' against
x. It is possible for x' to match x exactly, yet for both to differ from r.

Only an external observer could detect that the system is not
working properly.

Wrong. "Working properly" is, for a controller, identical to having a
minimal prediction error.

Bill, you ought to do some _real_ experiments. They are so much more
reliable than thought experiments...

Wow. I hope there was an implied smiley after that.

The basic problem I am raising is this: how does the _real_ system know that
the inverse it is calculating is in fact the correct inverse of the
world-model? This is completely separate from the problem of whether the
world model exactly represents the plant. The _only_ indication would be a
match of r and x. But this comparison is not made in your model. If you were
thinking in terms of _building_ a real system, you would see this as a real,
practical, problem that must be addressed. It's evident to me that you have
not considere this problem, and that by implication, neither has the MCT
community.

Best,

Bill P.

[Hans Blom, 970131]

(Bill Powers (970130.0500 MST))

I think I now -- finally? -- understand the problem that you pose. It
is a new problem that I've never considered, and it took me a while
to understand what you actually meant. I never thought that this
_could_ be a problem; I haven't come across it in engineering ;-).
But this last post of yours is especially clear. You state the
problem as follows, and it is, indeed, an interesting problem:

The basic problem I am raising is this: how does the _real_
system know that the inverse it is calculating is in fact the
correct inverse of the world-model? This is completely separate
from the problem of whether the world model exactly represents
the plant.

Let me rephrase the problem, and correct me if I still don't get it:

A model-based controller builds a perfect model, but makes a
significant error in its computation of the inverse of this model. A
psychological analogy could be an intact person with one major brain
injury: everything is in perfect working order except one function or
"computation". I'll treat the problem based on a simple example.

Assume that the environment function is x[k+1] = b*u[k] with b=3, and
that this function has been modeled perfectly. Assume also that we
are at time k and at time k+1 we want x to be 6. We then plug in and
get 6 = 3*u[k] and find that u[k] must be 6/3 = 2. We apply this u
and find that, indeed, x goes to where it ought to go in one step.
Thus far, everything is perfect.

Now assume that the inverse is computed incorrectly. Say that b is
estimated as 6 rather than 3, and that as a result u[k] = 1 rather
than u[k] = 2 is delivered by the controller. In fact, all outputs
will only be half of what they ought to be due to the defective
"inverter". In that case, x[k+1] will get the value 3*u[k] = 3 -- the
environment function has not changed -- rather than the desired 6.
This is bad control, obviously. Will the controller discover that it
controls badly? That depends on what the controller _believes_ it is
doing. We still assume that x and u are measured perfectly. To
simplify, there are two cases:

1. It picks up u _after_ the faulty inverse circuitry and thus knows
that it delivered u[k] = 1 (rather than 2, as its value ought to be;
but the controller doesn't know that). In its model adjustment phase,
it discovers that on u[k]=1 it perceives x[k+1]=3 (rather than the
desired xopt[k+1]=6, which it _does_ know about). Thus dx/du=3, the
old model is still correct and it will not be updated. Control will
remain bad, and this will not be discovered, despite the fact that
the controller could, in principle, discover that it controls badly.
The control law u[k] = xopt[k+1]/b(estimated) is so dumb that it just
doesn't use that information.

2. It picks up u _before_ the faulty inverse circuitry and thus
believes (erroneously) that u[k] = 2, whereas actually 1 is output.
Now, however, it discovers that on this supposed u=2 it perceives
x=3. Thus dx/du=1.5, the old model is incorrect and will be updated
to x[k+1] = 1.5*u[k], and accurate control will be restored. We have
the strange paradox that the model is greatly in error but control is
fine.

How can we interpret these findings? (Warning: speculation mode full
on!) In situation 1, the undiscoverable defect is "internal" in the
control system. In situation 2, it is "external" -- it is as if it
creates a new environment function -- and is corrected; correctly, in
fact, because the apparent "external environment" has indeed changed.
But situation 1 appears to be far more physiological, especially in
an adaptive control system where we assume that the controller knows
which output it really delivers at the controller/world interface. In
situation 1 we have an internal problem that control cannot correct.

Do we know psychological situations where situation 1 might apply?
(Warning: speculation mode still on!). Not at the lower levels of the
hierarchy, probably. I do not know of cases where patients attempt to
grasp something, don't, yet assume they do. But maybe at the higher
levels. We all know people who consistently overreact (don't we ;-),
and also people who habitually underreact. Could that be due to
"defective inverse circuitry" (don't take this too literally!)? Could
we find cases where apparently someone's internal model is correct,
yet behaviors are consistently over- or underreactions? Or would
those inappropriate reactions more likely be due to an inaccurate
model? These are questions. I have no answers.

Are there likely solutions to the problem of situation 1? Sure: a
different control law that _does_ somehow use the information that
control is bad even though the internal model is fine. But finding
such a control law is a problem that requires a lot more analysis. If
it is a "real" problem. Which it could be. After all, in an organism
almost anything can go wrong. Remains the question: are there, in the
psychological literature, cases which could be explained by the
assumptions of situation 1, a good model yet a bad inverse? How would
we test that?

Can we think of similar internal uncorrectable problems in a PCT
controller? Sure. Imagine that the subtraction in the comparator is
inaccurate and shows, for instance, a constant bias. Or that the
integrator in the output path accidentally becomes a differentiator.
Here, too, we would find pathological, uncorrected behavior. Could we
find cases in the psychological literature that could be explained by
such types of internal errors? Interesting questions.

Thanks, Bill, for posing an interesting problem that I have never
seen mentioned in the engineering literature, where it is always
assumed that it is the model-building that poses the problems and the
inversion is correct. You've given me much food for thought. Thanks.

Greetings,

Hans

[From Bill Powers (970131.0613 MST)]

Hans Blom, 970131--

Your post has restored my faith in you as a thinking person. I am sorry that
it took me so long to bring my idea into a coherent form that could express
what bothered me about the MCT model. My only excuse is that my thinking
seems to develop not in an orderly way from premise to conclusion, but more
as if I'm watching a latent photographic image emerge on printing paper that
is immersed in the developer under a dim safe-light. Is that a bird? An
airplane? No, by golly, it's Aunt Minnie!

Before we go on to draw any important conclusions, let's solidify the
progress we have made. My basic understanding of the concept behind MCT is
as follows:

First let's put aside temporarily the adaptive aspects of this model and
look at what the adaptation is supposed to produce as an end result.

  r(t) --->[Box 1] --> u(t) ----> [Box 2] ---> x(t)

The overall purpose of this arrangement is for r(t) to produce a
corresponding x(t), so that x(t) always equals r(t), as nearly as possible.

Box 2 contains the "plant," which in general we can observe and characterize
with an equation. This will be a matrix equation, with u and x being vectors
and Box 2 containing the matrix operator B. So

x(t) =B*u(t), the multiplication sign meaning matrix multiplication.

What is needed is to fill in another matrix operation A in Box 1, such that
r(t) will produce just that vector u(t) which, when transformed by B, will
produce the correct vector r(t). At this point it is we, the observers of
this system, who know whether r(t) = x(t).

From basic principles we know that if the square matrix A is the inverse of

the square matrix B, the desired relationship will be achieved. This is
equivalent to saying that the number of unknowns in the corresponding
individual equations is equal to the number of equations, so a unique
solution is possible, barring singularities.

In a little more detail, the inverse B^-1 is defined by the identity B*B^-1
= I, where I is the unit matrix with ones in the diagonal and zeros
elsewhere. If the connection between r and x is the unit matrix, then by
definition x(t) = r(t), the desired result. Therefore the matrix A must be
equal to B^-1 to get this result.

So now the problem has been reduced to finding the inverse of the plant
function.

The strategy for finding the _correct_ inverse, in the MCT model that you
have presented, is first to construct a forward simulation C of the plant
function B, using the Extended Kalman Filter or EKF method to adjust the
parameters. When this adjustment is complete, the simulation matrix C will
produce a variable x' which is at all times equal to the output x of the
plant, given u(t) as an input to both. Once the parameters have been
determined, the inverse can be obtained, and used in Box 1, above. The
reason for using this approach is that the controlling system has no direct
way of determining the plant function. It must deduce the parameters of the
plant function, and the EKF method is a way of doing that.

In the background of this process there is also "system identification,"
which is required to provide the skeleton of the simulation C in which there
are adjustable parameters. In general, system identification gives us a
matrix of polynomials, or some other basic form, which has the same
structure as the matrix of polynomials describing the plant -- without,
however, knowledge of the specific parameter values. Given that basic form,
the EKF method can then adjust the parameters on the basis of comparing the
simulation output x' with the actual output x of the plant. Ultimately, I
suppose, system identification can be combined with the EKF method to
construct the correct simulation by a single more complex process.

Once the simulation is in the correct form, all that remains is to take its
inverse to put into Box 1 as the matrix A. Taking the inverse involves
computing the transpose of the adjoint of the C matrix (describing the
simulation) and dividing it by the determinant of C (I hope I have that
right, but the actual rule doesn't matter here). Each entry in the A matrix
will be, in general, a mathematical combination of several coefficients in
the C or simulation matrix, obtained in the process of computing the
inverse. As the EKF process modifies the coefficients in the simulation
matrix, the inversion calculations must be done to update the coefficients
in the A matrix before r is used to calculate the next value of u. This
running calculation of the inverse is not generally identified as a specific
operation in the MCT model, although it must appear as an explicit operation
in any physical implementation of this model.

So the complete MCT model can be sketched in this way:

                      plant
  r -->[A] ---> u ---> [B} ----> x
        ^ | |
        > v (param adj) |
   [inverse]<--[C]<------ |
    [taker] | | |
                x'---->{EKF}<----
            simulation
               of x

That's the model as I have come to understand it.

If we imagine that the various matrix operations are being carried out by
neural networks, then each matrix -- A and C inside the controller -- must
be embodied in a physically different network: neural networks don't work
backward. The "inverse taker" process must translate the form of neural
network C into a form of neural network A that constitutes the inverse of C.
This means that the basic form of A must match the inverse of the basic form
of C, and that the coefficients in A (however they are embodied in the
neural network) must be the correct functions of the coefficients embodied
in the neural network representing C. These inverse calculations must be
done, in general, continuously, or in a computer model, on every iteration.
This is the only way in which variations in the properties of the plant can
be properly converted into the appropriate changes in the matrix A.

Finally, it is clear that there is no function in this model that can detect
directly a difference between r(t) and x(t). For such a function to be part
of the model, we would have to have, somewhere,

  r(t) --
          \
       [comparison] ---> difference information
          /
  x(t) --

The difference information would then have to be used in some way by other
parts of the model. Clearly, there is no such provision in the model as it
stands.

Is this development agreeable to you so far, Hans?

Best,

Bill P.

[Hans Blom, 960204]

(Bill Powers (970131.0613 MST))

Your post has restored my faith in you as a thinking person. I am
sorry that it took me so long to bring my idea into a coherent form
that could express what bothered me about the MCT model.

I could understand the first sentence, by itself. Or the second. But
the combination? Do you have your attributions crossed? ;-). I agree
with your basic sentiment, however: wouldn't it be nice if others
understood us even before we had a clear understanding ourselves?
Let's ask Rick about mind-reading ;-).

Before we go on to draw any important conclusions, let's solidify
the progress we have made. My basic understanding of the concept
behind MCT is as follows:

First let's put aside temporarily the adaptive aspects of this model
and look at what the adaptation is supposed to produce as an end
result.

r(t) --->[Box 1] --> u(t) ----> [Box 2] ---> x(t)

Let's go very slowly. I'll derive my version of the diagram using the
example that I used previously. I'll first add another box, which I
need to make things hang together, but which you leave out. As a last
step, I'll discard this box again, as per your instructions ;-).

Assume that the (unknown!) environment function is x(k+1) = 3*u(k).
Assume further that we are at time instant k, and that we have to
compute "action" u(k) in order that "perception" x(k+1) is equal to
"reference" r(k+1). Note that this implies a prediction. We are now
at time instant k, yet we specify a condition that is to hold at time
instant k+1. The accuracy of a prediction depends, of course, on how
well we can extrapolate the past into the future. If not at all, no
control is possible. If somewhat, we can control somewhat. That's the
basic philosophy of control. But back to more mundane matters.

The task of my first box is to model the environment, that is to
discover/estimate the environment function, i.e. to discover -- in my
example environment function -- that (for any k) x(k+1) is directly
related to u(k), and with a "gain" of 3, but to nothing else, i.e.
not to, say, u(k-3) or to x(k-6). So the inputs of this box are x and
u (the histories of x and u are also somehow saved or "summarized" in
this box), and its output is an estimate of the relationship x(k+1) =
3*u(k), which, if perfect, is exactly the same as the environment
function. I'll not go into the details of _how_ this box estimates
the environment function.

My second box is the controller. Its inputs are the model (from the
first box), and x and r, and its function is to compute such a u(k)
that x(k+1) will be equal to r(k+1); something else (a higher level?)
defines r(k+1). So the controller knows and works with

(1) x(k+1) = 3*u(k) [the model] and
(2) x(k+1) = r(k+1) [the control requirement]

and thus solves

(3) r(k+1) = 3*u(k) [eliminating x(k+1) from (1) and (2)]

from which the result is obtained:

(4) u(k) = r(k+1)/3

My third box is the environment function itself, with input u and
output x, in the example the function x(k+1) = 3*u(k).

So my block diagram, which isn't nearly as nice as yours (and where
time indices do matter!), would look like

                   > r
                  \|/
  -------- m -------- u -------- x
  > box1 |---->| box2 |----->| box3 |----->
  -------- -------- --------
   ^ ^ ^ ^
   >x |u |x |u

where m is the model.

I do not, in general, think that such a diagram is very helpful. The
formulas are much clearer to me, but that may be, of course, a matter
of taste. In order to _understand_ a block diagram, I do need to know
the formulas that define what each block does. That's why a diagram
is kind of superfluous to me. And I assume that the above diagram
doesn't look very agreeable to you, either.

I would slightly reformulate your conclusion:

The overall purpose of this arrangement is for r(t) to produce a
corresponding x(t), so that x(t) always equals r(t), as nearly as
possible.

into

The overall purpose of this [control] arrangement is to have the
controller compute a u(t) to produce a corresponding x(t+1), so that
x(t+1) always equals r(t+1); exactly and in one step if the model is
perfect and if there is no noise and if the controller is powerful
enough to deliver this output; as nearly as possible if any of these
conditions is not fulfulled.

This shows that MCT is also about "control of perception" ;-). This
also shows, by explicit use of time indices, that in MCT prediction
is involved: we do something _now_ in the hope to see some condition
fulfilled _in the future_. In the very near future, maybe, but in the
future nevertheless.

It is now easy to adapt my diagram to yours, but now with the
additional understanding that your box1 also has access to the model.
The difference between PCT and MCT is now (merely?) how u is
computed. In the MCT controller, the computation of u is based on the
two equations

(1) x(k+1) = 3*u(k) [the model] and
(2) x(k+1) = r(k+1) [the control requirement]

In other words, in the model (1) x(k+1) must be replaced by r(k+1) in
order for a u(k) to be found such that the control requirement (2) is
fulfilled.

Before going into inverses, I want to check back with you whether we
can agree on the above.

Note, by the way, that an _explicit_ inverse can be avoided. We know
that the controller must solve

(3) r(k+1) = 3*u(k)

A solution could also be achieved by some kind of internal "knob
twiddling" of u(k) such that (3) becomes fulfilled. If such "knob
twiddling" would be done by some kind of gradient method which
minimizes the difference between r(k+1) and 3*u(k), we would be back
to something closely approximating the PCT method, I think. That may
be the link we're looking for. But maybe I'm looking too far ahead
now...

Greetings,

Hans

[From Bill Powers (970204.0540 MST)]

Hans Blom, 960204 --

Let's go very slowly. I'll derive my version of the diagram using the
example that I used previously. I'll first add another box, which I
need to make things hang together, but which you leave out. As a last
step, I'll discard this box again, as per your instructions ;-).

Fine. No point in going past the last place where we agree.

Assume that the (unknown!) environment function is x(k+1) = 3*u(k).
Assume further that we are at time instant k, and that we have to
compute "action" u(k) in order that "perception" x(k+1) is equal to
"reference" r(k+1). Note that this implies a prediction. We are now
at time instant k, yet we specify a condition that is to hold at time
instant k+1. The accuracy of a prediction depends, of course, on how
well we can extrapolate the past into the future. If not at all, no
control is possible. If somewhat, we can control somewhat. That's the
basic philosophy of control. But back to more mundane matters.

The task of my first box is to model the environment, that is to
discover/estimate the environment function, i.e. to discover -- in my
example environment function -- that (for any k) x(k+1) is directly
related to u(k), and with a "gain" of 3, but to nothing else, i.e.
not to, say, u(k-3) or to x(k-6). So the inputs of this box are x and
u (the histories of x and u are also somehow saved or "summarized" in
this box), and its output is an estimate of the relationship x(k+1) =
3*u(k), which, if perfect, is exactly the same as the environment
function. I'll not go into the details of _how_ this box estimates
the environment function.

This is the first place we disagree. I am not willing to accept the
existence of a magic box which simply provides the information that you need
-- not when it is so critical to everything that follows. This is just to
note that I already begin to question your model at this point.

My second box is the controller. Its inputs are the model (from the
first box), and x and r, and its function is to compute such a u(k)
that x(k+1) will be equal to r(k+1); something else (a higher level?)
defines r(k+1). So the controller knows and works with

(1) x(k+1) = 3*u(k) [the model] and
(2) x(k+1) = r(k+1) [the control requirement]

Again, something just gets done with no indication of how. How can the model
be an input to the second box? We are talking about a physical system here,
not an abstract compution done in your mind. How does the first box pass the
model to the second box? Does it write it down on a sheet of paper and drop
it through a slot in the second box? And can the second box read what is on
that sheet of paper and understand it? In my final diagram, I included a
step called the "inverse taker." This was to indicate the place where some
circuit in the brain actually carries out the computation that transforms
the present form of the forward simulation into the inverse process in
another box. This doesn't just happen: there must be an actual physical
process that carries out the implied operation, and that process has to be
part of the model being proposed.

You say that it's easier for you to skip the block diagram and just work
directly with the equations. Of course it is! When you're not required to
show explictly every operation that is carried out as a part of the model,
you can supply those operations yourself, not distinguishing between what
you, the modeler, are doing, and what the model itself, as a physical
system, is doing. If you're seriously proposing a model, then you NEED a
block diagram, because it is an essential step in designing the system that
is actually going to carry out all the processes you describe -- ALL of
them, including the steps you now carry out in your head. Before you can get
your model to work -- before you can actually build an example of it -- you
are going to have to supply circuitry that will carry out every operation in
every box. You can leave NOTHING as an operation in your own head, because
the model is going to have to work without your help or interpretation. I
will grant that we can't always supply those details; that we often have to
posit an input-output function that just gets done. But the more we have to
posit, the weaker the model gets. You must at least recognize that there IS
a process, and that some physical part of the system is going to have to
carry it out, all by itself. This tells you what you are actually demanding
of the physical system you're modeling.

Drawing a block diagram is the best way I know to reveal places in a model
where there are holes. In your model, the hole now under discussion is the
process of passing a model from one functional box to another. In what
physical form does this model exist in your Box 1? Is it a neural network?
Does this neural network get physically transported into the part of the
system represented by box 2? Is the information carried by physical signals
of some kind? What effects do those physical signals have when they enter
box 2? I don't mean to suggest that your answers to each question would lead
to the particular question I ask next -- I'm only illustrating how the query
would go.

I suspect that to you, Box 1 and the arrow carrying the model into Box 2 are
not really physical things; they are immaterial concepts. They are stages in
a mathematical derivation or steps in a computer program, having no
particular physical existence. They're imaginary, not constrained by the
real world.

So we must resolve this issue, which boils down to two questions: What is
the physical form of the model in your Box 1, and what is carried from Box 1
to Box to Box 2 in the place where you show the model as being passed from
one box to another?

If you find this question baffling, then you do not yet understand the point
of my objections.

Best,

Bill P.

[Hans Blom, 970204d]

(Bill Powers (970204.0540 MST))

What is the physical form of the model in your Box 1, and what is
carried from Box 1 to Box 2 in the place where you show the model as
being passed from one box to another?

We seem to be on very different wavelengths: I talk of what kind of
functionality is required, you of what kind of material objects can
do the job. Anyway, think of the model as a vector, an ordered set of
parameter values. Each parameter could be carried from Box 1 to Box 2
as a "nerve current". Or it could be a synaptic strength that is set
by Box 1 and (also) used by Box 2. Or ...

There's usually a variety of ways in which one can implement a
certain concept. And I'm not even sure whether the actual
implementation in the brain -- if there is such a thing -- is
reconcilable with your notion of nerve currents, as you seem to
assume. For all that I know it could be something like phase-locked
loop-like frequency division or something else that works more
directly -- and more efficiently -- with the existing hardware that
uses action potentials.

I assume that you are fully aware of the fact that certain properties
of the nervous system are not accounted for in PCT. One particular
property of the nervous system that has not yet been exploited in any
PCT model -- as far as I'm aware of -- is, for instance, the fact
that our sensors seem to have a logarithmic conversion function
built-in. We don't see the world, it seems, we see its logarithm ;-).
This might also imply that, when we look into the functioning of the
nervous wetware and think we see addition and subtraction (of "nerve
currents"), we should more properly call them multiplication and
division. We are far from having exhausted all the possibilities of
what our wetware is capable of, and our models are still extremely
primitive. That is why I am far more comfortable with talking at the
abstract level of functionalities than at the level of mapping those
functions to wetware operations.

On second thought, I've always thought that PCT took the same stance.

Greetings,

Hans

[From Bill Powers (970204.0955 MST)]

Hans Blom, 970204d --

We seem to be on very different wavelengths: I talk of what kind of
functionality is required, you of what kind of material objects can
do the job. Anyway, think of the model as a vector, an ordered set of
parameter values. Each parameter could be carried from Box 1 to Box 2
as a "nerve current". Or it could be a synaptic strength that is set
by Box 1 and (also) used by Box 2. Or ...

All modeling requires a certain amount of arm-waving, because we can't guess
how some things -- a lot of things -- are accomplished in the real system.
But your approach consists ENTIRELY of arm-waving, and I simply can't buy
it. The fact that you can make your system work mathematically is no
indication WHATSOEVER that it could be implemented in a real nervous system.
Every practical concern I have raised, you have simply dismissed with a wave
of the hand. I don't foresee any change in this reaction to my comments. You
are willing to assume things that I am not willing to assume. That leaves us
at an impasse.

I assume that you are fully aware of the fact that certain properties
of the nervous system are not accounted for in PCT. One particular
property of the nervous system that has not yet been exploited in any
PCT model -- as far as I'm aware of -- is, for instance, the fact
that our sensors seem to have a logarithmic conversion function
built-in. We don't see the world, it seems, we see its logarithm ;-).

Yes, this has been considered. I have modeled control systems with
logarithmic (and power-law) input functions; the main effect is that the
loop gain drops at lower magnitudes of the perceptual signal. The primary
effect of this logarithmic response is only at the lower levels of
perception, however. When you get to configurations, shapes and orientations
are not affected; we do not, for example, perceive angles logarithmically
when they are involved in perceiving the rotational position of an object. A
rotation of 360 degrees is exactly equal to two rotations of 180 degrees.

In a modality like audition, we hear chords as being independent of pitch,
because the ratio of frequencies remains unchanged as the pitch changes.
Here the logarithmic form of perception is essential.

To some extent the logarithmic form of perception is compensated, at the
lower levels, by the square-law response of muscles. The compensation is not
exact, of course, but the two forms are roughly complementary, the result
being that there is a reasonably large linear portion of the overall system
curve. Also, with respect to the stretch reflex tonic signal, it is known
that recruitment effects end up producing a _very_ linear overall
relationship between signal magnitude and actual elongation, even though the
response in individual neurons is nonlinear.

This might also imply that, when we look into the functioning of the
nervous wetware and think we see addition and subtraction (of "nerve
currents"), we should more properly call them multiplication and
division.

This is from B:CP:

The nervous system, no matter what kind of computer to which we liken it,
does not operate according to the forms we find convenient to represent
mathematically. At best, mathematical descriptions can only provide a form
that is reasonably close to reality. The "summation" of neural currents is
only roughy linear -- 2 + 2 may equal 4, but 4 + 4 may equal a little less
than 7; 7 + 7 may equal 8, and 1/2 + 1/2 may equal zero. This nonlinearity
means that everything we would like to call "summation" also contains a
little of what we would call "multiplication," and vice versa.

But that was on page 23, the section on "Neural analog computers." I don't
suppose you have read that far.

We are far from having exhausted all the possibilities of
what our wetware is capable of, and our models are still extremely
primitive. That is why I am far more comfortable with talking at the
abstract level of functionalities than at the level of mapping those
functions to wetware operations.

It's one thing to admit that our knowledge of neural processes is
incomplete. It's another to use that fact as a garbage can into which we
throw all considerations of realism. There is, I think, an obligation to use
what we DO know of neural systems, so that at least our models fit what is
known. If you ignore what is known just because there's a lot that isn't
known, you're just giving yourself license to indulge in fantasies without
any constraints. I don't consider that modeling.

On second thought, I've always thought that PCT took the same stance.

Then you have misunderstood PCT.

Best,

Bill P.

[Hans Blom, 970205]

(Bill Powers (970204.0955 MST))

All modeling requires a certain amount of arm-waving, because we
can't guess how some things -- a lot of things -- are accomplished
in the real system.

Because it is impossible to do an in-depth analysis of the kind of
physical systems that we are concerned with, down to elementary
particles, we have to be satisfied with working in terms of (much)
higher level notions. You work in terms of notions that appeal to
you, and so does everyone. Sometimes a branch of science is more or
less unified in that most people use the same notions. It is like
using the same language. It helps greatly in understanding each
other.

But your approach consists ENTIRELY of arm-waving, and I simply
can't buy it. The fact that you can make your system work
mathematically is no indication WHATSOEVER that it could be
implemented in a real nervous system.

Could be? Tell me why it _can_ be implemented in transistors but not
in neurons.

Every practical concern I have raised, you have simply dismissed
with a wave of the hand.

I simply don't know enough about neuroanatomy and -physiology to even
understand what are "practical concerns" if we were to build complex
circuitry as collections of real neurons. I do have the understanding
that a real neuron is a "universal computing element" with which we
could, in principle, build any type of computation. We can build any
type of circuit with relays, radio tubes, transistors, and neurons.
You yourself gave a (partial) demonstration of that in B:CP. So I
keep wondering why you object so strenuously that things that can be
realized in transistors cannot be realized in neurons. Are computers
so infinitely more powerful than brains? No. They are equivalent in
the sense that both are Turing Machines that could, in principle, do
anything.

I don't foresee any change in this reaction to my comments. You are
willing to assume things that I am not willing to assume.

And the other way around.

That leaves us at an impasse.

Does it? We do speak (slightly) different languages, but does that
prohibit understanding?

We don't see the world, it seems, we see its logarithm ;-).

Yes, this has been considered.

Have you also considered how logarithmic functions can be _exploited_
rather than just considered a nuisance, as you seem to say:

To some extent the logarithmic form of perception is compensated, at
the lower levels, by the square-law response of muscles. The
compensation is not exact, of course, but the two forms are roughly
complementary, the result being that there is a reasonably large
linear portion of the overall system curve.

We are trained in the analysis of _linear_ functions. But the CNS is
hardly linear. Might that not offer thusfar unexplored possibilities?

It's one thing to admit that our knowledge of neural processes is
incomplete. It's another to use that fact as a garbage can into
which we throw all considerations of realism.

Realism? Your type of realism? You open a can of worms...

There is, I think, an obligation to use what we DO know of neural
systems, so that at least our models fit what is known.

To what extent does PCT do that?

If you ignore what is known just because there's a lot that isn't
known, you're just giving yourself license to indulge in fantasies
without any constraints. I don't consider that modeling.

Fantasies are also called hypotheses. And modeling comes in flavors:
modeling of functionality is one type. In the final analysis, we
never rise beyond it. So I guess I don't understand you. What _is_
modeling to you?

On second thought, I've always thought that PCT took the same
stance.

Then you have misunderstood PCT.

Bill, don't be naive. As soon as you let go of action potentials and
introduce "nerve currents", for instance, you do functional modeling
as well.

Greetings,

Hans

[From Bill Powers (970205.0925 MST)]

Hans Blom, 970205]

But your approach consists ENTIRELY of arm-waving, and I simply
can't buy it. The fact that you can make your system work
mathematically is no indication WHATSOEVER that it could be
implemented in a real nervous system.

Could be? Tell me why it _can_ be implemented in transistors but not
in neurons.

In my just-previous post I showed why it can't be implemented in transistors
OR neurons. When you say "transistors" I assume you're talking about digital
computers. In your computer program, you have always made the "inverse" part
of the model anlytically exact, and you have assumed that an output u can be
calculated which affects the real environment EXACTLY as it affects the
internal simulation. These assumptions are the Achilles Heel of MCT.

Every practical concern I have raised, you have simply dismissed
with a wave of the hand.

I simply don't know enough about neuroanatomy and -physiology to even
understand what are "practical concerns" if we were to build complex
circuitry as collections of real neurons.

Then let me state what I mean plainly. The primary practical concern is
accuracy of computation, coupled with the fact that the real world involves
multiple consecutive integrations and unpredictable disturbances. Your
symbolic calculations are infinitely accurate; no real system, other than a
symbol-handling system of exactly the kind that permits you to do
mathematics, can be infinitely accurate. And even if a symbolic calculation
is done, it has to be turned into analog variables in order to be translated
into action.

I do have the understanding
that a real neuron is a "universal computing element" with which we
could, in principle, build any type of computation. We can build any
type of circuit with relays, radio tubes, transistors, and neurons.
You yourself gave a (partial) demonstration of that in B:CP. So I
keep wondering why you object so strenuously that things that can be
realized in transistors cannot be realized in neurons. Are computers
so infinitely more powerful than brains? No. They are equivalent in
the sense that both are Turing Machines that could, in principle, do
anything.

All that is true only if you think categorically. The real computations that
you can do with neurons (or transistors) are analog, involving continuous
variables like chemical concentrations (or voltages and currents). Yes, you
can do any conceivable computation, but you can't do it _accurately_. The
MCT model is basically an open-loop model, and it relies completely on
accurate computation in the path between r and x -- no, not just "accurate";
_perfect_. As I illustrated with my example of steering a car under ideal
conditions, the effects of even microscopic errors are magnified in the real
world and become significant with alarming rapidity. The only kind of system
that can maintain good control in this kind of world is the PCT model, not
the MCT model.

Have you also considered how logarithmic functions can be _exploited_
rather than just considered a nuisance, as you seem to say:

Yes, and I gave an example: perceiving chords. A major third is perceived as
the same chord regardless of its pitch, and this requires a logarithmic
scale for perceiving pitch.

We are trained in the analysis of _linear_ functions. But the CNS is
hardly linear. Might that not offer thus far unexplored possibilities?

Of course. So what? That is not the issue between us.

Realism? Your type of realism? You open a can of worms...

There is, I think, an obligation to use what we DO know of neural

systems, so that at least our models fit what is known.

To what extent does PCT do that?

Read the chapter in B:CP called "premises." Also, look at the Little Man
Version 2, which contains a fairly literal rendition of what is known about
the muscles and the stretch and tendon reflexes, as well as a consideration
of how the brain converts retinal information into depth and position
signals (the neuromorphic engineers are building analog VLSI chips which
actually can produce depth and position signals from binocular retinal
images). The old digital view of neural functions has been pretty thoroughly
discredited by modern research into modeling neurons -- but you will find,
in the chapter on Premises, a suggestion for how a digital computing element
can be constructed with an analog neuron. I think that PCT is consistent
with what is known about the capacities -- and accuracy -- of neural
computing elements.

Bill, don't be naive. As soon as you let go of action potentials and
introduce "nerve currents", for instance, you do functional modeling
as well.

Nerve currents are as consistent with action potentials as electrical
currents are with the dynamics of electron and hole movement in solids; in
fact the relationship is of the same kind. Nerve currents can be calculated
directly from any model of a neuron, or from direct measurements. The
connection is clear and explicit. No arm-waving is involved.

Best,

Bill P.

[Hans Blom, 970206b]

(Bill Powers (970205.0925 MST))

In my just-previous post I showed why it [model-based control] can't
be implemented in transistors OR neurons.

Bill, let's distinguish between two things:

1. Model building. Have I convinced you that an on-line real-time
system can form an accurate model of its environmental function? If
not, we need to go into this in more detail.

2. Control using the information in that model. You have convinced me
that the control law that I used in my explanations and demos is not
adequate if no good inverse (of the environment function) can be
computed by the system. In the past, I have frequently stressed that
I was concerned with model building, and that the control law that I
used was an "afterthought" -- not the important issue. It is, in
fact, the simplest one imaginable. You have identified a new problem
that I did not consider before and demonstrated, in effect, that
having a good model is no good if that model cannot be or is
inadequately _used_. I am willing to consider this problem, i.e. how
to control using the knowledge embodied in the model given a bad
inverse computation, but then we must agree on the first point:
_that_ a model can be built. We can then proceed and try to find
control laws -- e.g. ones that do not need an inverse -- that are
robust in the face of (large) errors in computations of inverses.

How's that?

Hans

[From Bruce Gregory (970206.0945 EST)]

Bill Powers (970205.0925 MST)

Nerve currents are as consistent with action potentials as electrical
currents are with the dynamics of electron and hole movement in solids; in
fact the relationship is of the same kind. Nerve currents can be calculated
directly from any model of a neuron, or from direct measurements. The
connection is clear and explicit. No arm-waving is involved.

Except that done by Little Man.

Bruce G.

[From Bill Powers (970209.0615 MST)]

Hans Blom, 970206b--

Bill, let's distinguish between two things:

1. Model building. Have I convinced you that an on-line real-time
system can form an accurate model of its environmental function? If
not, we need to go into this in more detail.

All right, let's go into it in more detail. Our discussion has led me to an
understanding -- I think -- of the basic limitation of the kind of model you
propose, so perhaps we could focus on the specific situation where I think
your model has a problem.

Suppose the task is simply to control the azimuth angle of a theodolite,
using a motor to turn it to point in any direction around the horizon. This
is a simple one-dimensional task. To make the task as easy as possible,
let's say that the theodolite is mounted on a highly damped bearing, so that
applying a torque to it makes it swing at an angular velocity proportional
to the torque. We can neglect the inertia of the theodolite, assuming it
will coast to a stop almost instantly when the torque is removed.

The plant equation, in differential-equation terms, is

dx/dt = A*u(t)

Where x is the horizontal angle, and u is the torque applied by the motor.
We want to be able to turn the theodolite to point in any specified
direction given by a signal r. If r changes, we want the theodolite pointing
direction to follow r. We can confine the movement range to 359 degrees, to
avoid problems of turning through more than one full revolution.

The motor we are using is not a stepping motor; it has an armature that is
driven by a spinning magnetic field, so the torque applied to the theodolite
is simply proportional to the current in the stator windings.

I know how the PCT model of this pointing system would be designed. We would
make the motor torque proportional to (r - x). After a step-change in r, the
position x would approach r asymptotically, along an
exponentially-decreasing curve of velocity. This method depends on
continuously comparing the controlled variable x with the reference signal r.

So how would this control system be designed using the MCT approach?

Best,

Bill

[Hans Blom, 970211b]

(Bill Powers (970209.0615 MST))

Bill, let's distinguish between two things:

1. Model building. Have I convinced you that an on-line real-time
system can form an accurate model of its environmental function? If
not, we need to go into this in more detail.

All right, let's go into it in more detail. Our discussion has led
me to an understanding -- I think -- of the basic limitation of the
kind of model you propose, so perhaps we could focus on the specific
situation where I think your model has a problem.

Suppose the task is simply to control the azimuth angle of a ...

Are we talking at cross-purposes again? I propose to distinguish
between modeling and control, as is customary in MCT; you start with
control immediately, before a good model is available. The problem
may not be in the model (I doubt it), it may be in the control law
that I used and its explicit use of an inverse.

Moreover, instead of introducing something new altogether, I'd prefer
to refer you to some examples of the MCT approach, to be found at

  http://ankeborg.isy.liu.se/cgi-bin/reports

Paper 1251 treats the problem of channel equalization in telephony
(identification of the unknown characteristics of the channel and
providing a correction) and offers a solution which avoids an
inverse, and does not even seek an approximate one. See also paper
1724.

Paper 1519 treats the problem of MCT control of hydraulic crane, as
does the longer version of paper 1520.

Interesting are, I think, also papers 1622 and 1742, which relate
systems identification with neural networks, paper 1664 which deals
with tracking time-varying parameters, and 1910, in which prof. Ljung
tells us his view on the history of this branch of engineering, which
seems to be far more popular in Europe than in the Americas.

See whether your objections still hold after reading, e.g., paper
1519 or 1520.

Greetings,

Hans

[From Bill Powers (970212.1815 PST)]

Hans Blom, 970211b--

Are we talking at cross-purposes again? I propose to distinguish
between modeling and control, as is customary in MCT; you start with
control immediately, before a good model is available. The problem
may not be in the model (I doubt it), it may be in the control law
that I used and its explicit use of an inverse.

I have no problem with modeling; it's usually possible to come up with a
reasonable model of a physical system, as long as you don't try to use the
model to predict a long distance into the future. The issue here is control:
using any means to make the output of some physical process follow a
preselected path.

Moreover, instead of introducing something new altogether, I'd prefer
to refer you to some examples of the MCT approach, to be found at

http://ankeborg.isy.liu.se/cgi-bin/reports

Paper 1251 treats the problem of channel equalization in telephony
(identification of the unknown characteristics of the channel and
providing a correction) and offers a solution which avoids an
inverse, and does not even seek an approximate one. See also paper
1724.

Paper 1519 treats the problem of MCT control of hydraulic crane, as
does the longer version of paper 1520.

Interesting are, I think, also papers 1622 and 1742, which relate
systems identification with neural networks, paper 1664 which deals
with tracking time-varying parameters, and 1910, in which prof. Ljung
tells us his view on the history of this branch of engineering, which
seems to be far more popular in Europe than in the Americas.

See whether your objections still hold after reading, e.g., paper
1519 or 1520.

When I get home I will look up these papers; it's not practical now, since
I'm getting my mail through long distance telephone calls.

However, it seems to me that the problem I proposed is simple enough: to
sketch in how the MCT model would be used to control the value of x for a
plant in which

dx/dt = A*u(t).

I have selected this simple plant because it involves one integration: that
is, x = integral( A*u(t)*dt). I believe that any system with the
architecture you are proposing will have difficulty controlling this plant,
because it will be unable to correct for integration errors in the model.
However, if you are able to show that it doesn't have this problem, I will
have learned something of value, so I hope you'll address yourself to laying
out the MCT controller for this plant.

Best,

Bill P.

[Hans Blom, 970213]

(Bill Powers (970212.1815 PST))

I have no problem with modeling; it's usually possible to come up
with a reasonable model of a physical system, as long as you don't
try to use the model to predict a long distance into the future. The
issue here is control: using any means to make the output of some
physical process follow a preselected path.

However, it seems to me that the problem I proposed is simple
enough: to sketch in how the MCT model would be used to control
the value of x for a plant in which

dx/dt = A*u(t).

I have selected this simple plant because it involves one
integration: that is, x = integral( A*u(t)*dt). I believe that any
system with the architecture you are proposing will have difficulty
controlling this plant, because it will be unable to correct for
integration errors in the model.
However, if you are able to show that it doesn't have this problem,
I will have learned something of value, so I hope you'll address
yourself to laying out the MCT controller for this plant.

Will do. Reasonable enough. And I'll spell it out so clearly that
even Rick will be able to follow my argument ;-).

In view of your statement that you have no trouble with modeling,
let's first assume that a perfect model is available. A perfect model
is, of course, identical to the "system to be controlled":

dx/dt = a*u(t)

or

dx = a*dt*u(t)

[In a more formal derivation, I would need to use different symbols
for the real system and for its model; since we assume a perfect
model for now, the two coincide and we can dispense with this formal
nicety].

In order to avoid integrals, I'll reformulate this as a difference
equation

x(k+1)-x(k) = a*dt*u(k)

where dt is the sampling/control interval. Rewrite this to

x(k+1) = x(k) + a*dt*u(k)

and we have the standard difference equation form. Note that this
equation has the character of a prediction: at sample instant k, and
knowing only x(k) and u(k) [and, of course, a (given by the model)
and dt (any small enough value will do)], we also know x(k+1), the
value of x one sample time ahead.

Now, in the control law that I have used thus far, we desire the
following relationship

x(k+1) = r(k+1)

where r is the prescribed reference level, i.e. the value that x is
to have in the immediate future. It would make no sense to specify a
reference level r(k) for x(k) at sample instant k, where x(k) has a
known value, is fixed, and thus cannot be manipulated anymore. This
formula shows that control is all about manipulating the _future_.

By combining both formulas we get

r(k+1) = x(k) + a*dt*u(k)

and, solving for u, we have

u(k) = [r(k+1)-x(k)] / (a*dt)

This is the computation of u in case of a perfect model. The dt in
the denominator, by the way, tells us that this control law is
differentiator-like.

If we insert the control law into the model equation, we get

x(k+1) = x(k) + a*dt*u(k) =
x(k+1) = x(k) + a*dt*[r(k+1)-x(k)]/(a*dt)
       = x(k) - x(k) + r(k+1)
       = r(k+1)

That is, the controlled variable goes to the specified reference
value (1) in one step and (2) without error and (3) even if the
reference value changes over time in whichever way (assuming no
limitations on u).

Let us now analyze what happens if we have an imperfect model. In
that case, we must select a suitable model structure, i.e some
(regression) relationship between x(k+1) and previous values of x
and/or u that "explains" the data sufficiently accurately. Choosing
such a relationship belongs to the domain of modeling. Let's assume
that the modeling is well done and comes up with the simple structure
that best resembles reality:

x(k+1) = x(k) + a*dt*u(k) + e(k)

where the difference equation is of the same form but an additional
term e, the modeling error (also called "noise" or "disturbance"),
represents all deviations between model and reality. In the imperfect
model case, the model's a will also not coincide exactly with the
real a (this makes the analysis slightly more complex, so I'll skip
it for now). We compute u from the above expression and the control
goal x(k+1) = r(k+1):

r(k+1) = x(k) + a*dt*u(k) + e(k)

We do not know e(k) and assume it to be zero. Why? Well, if we don't
know anything about e, it could have any value at all. However,
modeling assures that e will be (close to) zero _on average_; in
facts, modeling has as its primary goal to pick such a model that e
is as small as possible, on average, in the data that were analyzed
when constructing the model.

Thus, solving for u, we get the same (on average "best") control law
as above:

u(k) = [r(k+1)-x(k)] / (a*dt)

See how good control is, i.e. what actually happens to x(k+1), if we
use this control law in the noisy/imperfect model case:

x(k+1) = x(k) + a*dt*u(k) + e(k) =
       = x(k) + r(k+1) - x(k) + e(k)
       = r(k+1) + e(k)

or

x(k+1) - r(k+1) = e(k)

Thus the control error at every sample/control instant is equal to
the modeling error. That is why we want good models...

Note that, in the above, x must be measurable at all (sample) times
(i.e. there is no "walking in the dark" in this derivation). Note
also that the derivation is valid for any and all sample instants.
Thus, in particular, the control error does not increase over time,
as seems to be your worry.

As the control law formula shows, the computation of u(k) depends on
knowledge of r(k+1), x(k), a (or rather its estimate by the model)
and dt (and on accurate subtraction, multiplication and division, of
course!).

All right so far?

Greetings,

Hans

[From Rick Marken (970214.0820)]

Bill Powers (970212.1815 PST)--

it seems to me that the problem I proposed is simple enough: to sketch
in how the MCT model would be used to control the value of x for a
plant in which

dx/dt = A*u(t).

Hans Blom (970213) --

Will do. Reasonable enough. And I'll spell it out so clearly that
even Rick will be able to follow my argument ;-).

I can follow it quite well, thanks. You managed to propose an MCT model
that is exactly equivalent to a PCT model. You managed to do this by
failing to distinguish properly between the functions and variables
involved in the environment and system equations in your analysis.
What I can't figure out is whether you hit on this canard by accident
(my currect hypothesis) or whether you were clever enough to do it on
purpose.

You start by creating the following difference equation:

(1) x(k+1) = x(k) + a*dt*u(k)

This is the _environment_ equation; it describes the relationship
between system output, u(k), and controlled input, x(k+1).

Next, you "derive" the system equation:

(2) u(k) = [r(k+1)-x(k)] / (a*dt)

describing how system output, u(k) depends on system input, x(k).
This is just a simple proportional control system: output is
proportional, by the factor 1/(a*dt), to error -- the difference
between system input, x(k) and a reference specification for
that input, r(k+1).

Then you say:

This is the computation of u in case of a perfect model.

But this is silly because there IS NO MODEL in your system equation
(equation 2). It took me a while to figure out your trick here. In the
environment equation (1) you were apparently thinking of u() as the
environmental function that relates system output to system input.
But u() IS NOT A FUNCTION; IT IS A VARIABLE (the argument k is just the
index of the variable). If u() were the environmental function, then the
argument of u() would be the system output!! But it's clear from your
equations (and a diagram of your system -- see below) that u() IS THE
SYSTEM OUTPUT VARIABLE, NOT THE ENVIRONMENT FUNCTION TO BE MODELED.
Pretty clever, Hans;-)

So there is no model-based control here. The system model (equation 2)
is a regular perceptual control system, controlling the perceptual
representation of x(k).

Here is a diagram to help you understand what you did:

       System | Environment
r(k+1)-->C---> | --> u(k)
         ^ | |
         > > v
         ><--- | <--x(k+1)

Note that your equation (1) describes the environmental connection
between u(k) and x(k+1). Note also that you have not explicitly shown
the conversion of the environmental variable, x(k+1) into the system
(perceptual) variable that is actually compared to r(k+1) [an
expectable lapse in someone who sees MCT as an advance over PCT;-))].
Your equation (2) describes the system comparator function,
C = [r(k+1)-x(k)] / (a*dt). Of course, this equation should be
r(k+1)-p(k)] / (a*dt) where p(k+1) = f(x(k+1)), but that's just a
little detail that only a fanatic PCTer would care about.

All right so far?

Sure. If your goal was to describe an arcane method of coming up with a
PCT model of control that you could call an MCT model. I'd still like
to see what the MCT model of this controller looks like.

Best

Rick

From Bill Powers (970215.0500 MST)]

Hans Blom (970214) --

I proposed the plant function

dx/dt = A*u(t), or

x(t) = integral(A*u(t)*dt)

and as expected, you converted it to the discrete equation

x(k+1) = x(k) + A*u[k]*dt.

You then solved for u(k):

u(k) = [x(k+1) - x(k)]/(A*dt)

and with the assumption that x(k+1) = r(k+1), you obtained

u(k) = (r[k+1] - x[k])/(A*dt)

as I rather suspected you might do.

Unfortunately, this equation doesn't reduce to the continuous form as dt
approaches zero. Instead, you end up with an infinite value of u(k). So
when you put this expression for u(k) together with the original one to
show how x depends on r, you end up (in the limit as dt approaches zero)
with a form containing infinity times zero, which is indeterminate. This
fact is hidden by your method of calculation.

The problem is that you are mixing an ordinal index, k, with a measure
of physical time, t. When you compute the value of x(t) at two different
times, the time difference must be equal to dt, not 1. If you change the
equation above from

u(k) = [x(k+1) - x(k)]/(A*dt)

to

u(t) = [x(t+dt) - x(t)]/(A*dt)

you have the correct form (in terms of the fundamental theorem of the
calculus). Now you have [x(t+dt) - x(t)]/dt; in the limit, as dt
approaches zero, the expression becomes exactly dx/dt. But when you use
k and k+1 in the numerator, this is no longer true: as dt shrinks,
nothing happens to the value of x(k+1) - x(k), so the value of u(k) goes
to infinity. This means that your conversion of the continuous equation
to discrete form is incorrect.

Another problem is that your derivation implies a physical
impossibility. Remember that I used a physical example, a theodolite
being moved from one horizontal angle to another. Suppose that the
original position was +60 degrees and the final position was -60 degrees
from midscale. My equation says that the rate of change of angle, dx/dt,
is A times the applied force, u. Let A = 10 units, and let dt = 0.01
second. We thus have

dx/dt (degrees/second) = 10*0.01*u = 0.1*u.

Clearly, if the theodolite is to move 120 degrees in dt seconds, the
force required would be u = 1200 units. If dt were set to 0.0001 second,
u would have to be 120000 units. Lacking any explicit relationship
between k+1 and t + dt, we have no way of computing the actual output
that would be required.

Suppose we start with x = 60 and r = 60, with dt = 0.001: the actual
position matches the reference signal r. Now we set r to -60. By your
derivation,

u(k) = (r(k+1) - x(k))/(A*dt), or

u(k) = (-60 - 60)/10*0.001 = -1200 units.

We now apply this u(t) to the plant, represented as

x(k+1) = x(k) + A*u(k) * dt, from which we get

x(k+1) = 60 + 10* (-1200)*0.01, or

x(k+1) = 60 - 120 = -60

So apparently, the theodolite has moved from 60 to -60 in a single
iteration, just as you claim.

However, we will get this same result for any size of dt. Suppose dt is
0.000001 sec. Now we get u = 1200000 units, and applying this to the
plant we find that x(k+1) is still -60 -- again, the move takes place in
a single iteration even through 1000 times the output is now required.
I'm sure you can see that as we let dt decrease without limit, to get
the continuous case, we always end up with x(k+1) having the correct
value of -60, but the magnitude of the output rises toward infinity. So
this is what I meant in the paragraph of this post near the beginning.
Mathematics says your solution will work, but physics says it won't.

···

-------------------
Now to the subject of modeling errors.

In the imperfect
model case, the model's a will also not coincide exactly with the
real a (this makes the analysis slightly more complex, so I'll >skip it for now). We compute u from the above expression and the >control goal x(k+1) = r(k+1):

r(k+1) = x(k) + a*dt*u(k) + e(k)

We do not know e(k) and assume it to be zero. Why? Well, if we >don't know anything about e, it could have any value at all. >However, modeling assures that e will be (close to) zero _on >average_; in fact, modeling has as its primary goal to pick such >a model that e is as small as possible, on average, in the data >that were analyzed when constructing the model.

This isn't the right way to handle modeling error. If there is a
modeling error, something about the assumed equation will differ from
the "right" equation. This difference, which we can call e, lies within
some small range epsilon of zero. It is true that if you made many
determinations of the parameters, the average value of e over many
trials might average close to zero (also, it might not). But on any one
run of the model, e will have just one value somewhere in the range plus
or minus epsilon, and it is extremely unlikely that this value will be
zero. This value will be integrated by the plant to produce a
continually increasing departure of x from r. Even if the parameters are
repeatedly re-evaluated and contain no systematic errors, integrating
the errors will lead not to averaging them out, but to a random walk.

In my example of the automobile trip, I showed that even a microscopic
error in computing the angle of the steering wheel or the steering
effort would soon have very large effects on the position of the car in
its lane. This is what is wrong with any open-loop control method.

Best,

Bill P.

[Hans Blom, 960217]

(Bill Powers (970215.0500 MST))

u(k) = [x(k+1) - x(k)]/(A*dt)

and with the assumption that x(k+1) = r(k+1), you obtained

u(k) = (r[k+1] - x[k])/(A*dt)

as I rather suspected you might do.

Unfortunately, this equation doesn't reduce to the continuous form
as dt approaches zero. Instead, you end up with an infinite value of
u(k).

You misunderstand my argument, and you are wrong. Remember that I
discretized time into intervals of duration dt. It is alright to let
dt go to zero, as you propose, but in that case you also have to
remember that the time interval between x[k+1] and x[k], and between
r[k+1] and x[k], goes to zero as well. It might help if you replaced
k+1 by k+dt, wherever it occurs. The expression for u then becomes

u(k) = (r[k+dt] - x[k])/(A*dt)

Here you clearly recognize a time increment of dt in both numerator
and denominator.

So when you put this expression for u(k) together with the original
one to show how x depends on r, you end up (in the limit as dt
approaches zero) with a form containing infinity times zero, which
is indeterminate. This fact is hidden by your method of calculation.

What _is_ hidden in the formula above is, that if u is limited, one
cannot arbitrarily pick any difference r[k+dt] - x[k] and expect it
to be brought to zero by an "inadequate" u. In such cases, the output
will saturate and u[k] cannot take x[k+dt] all the way to r[k+dt].
That is, indeed, a real design constraint; it can be combatted by
either increasing the motor's power and/or by decreasing one's
expectations of what a small motor can do.

The problem is that you are mixing an ordinal index, k, with a
measure of physical time, t. When you compute the value of x(t) at
two different times, the time difference must be equal to dt, not 1.

I assumed you would understand that I count in increments of dt, as
you do in PCT equations as well. Or don't you? But as I said, replace
1 by dt and see if you understand the derivation then.

Another problem is that your derivation implies a physical
impossibility.

The "physical impossibility" that you refer to has to do with the
limited output power that is available in the theodolith example that
you refer to. Clearly, if one's means are limited, one cannot fulfill
any (unreasonable) goal. But that is an additional constraint (on u),
that I did not take into consideration in my formulas, although I
mentioned it in my post. What will happen in practice is that the
reference level cannot be adequately met, and that the perception x
will lag behind what we want it to be, r. As would be quite normal in
a PCT controller as well.

This isn't the right way to handle modeling error. If there is a
modeling error, something about the assumed equation will differ
from the "right" equation.

Yes. I assumed in my discussion of the influence of modeling error
that the formula that we started with did not describe an actual
theodolith precisely, but was only an approximation, a "model". In
that case, e is the approximation or modeling error, the difference
between reality and its model. In real life, we _never_ have access
to the real thing, we _always_ have to be satisfied with a model. It
is only in simulations that we have the luxury of knowing the
characteristics of the "real" (?) thing.

But on any one run of the model, e will have just one value
somewhere in the range plus or minus epsilon, and it is extremely
unlikely that this value will be zero. This value will be integrated
by the plant to produce a continually increasing departure of x from
r.

Yes, I know you think so. But did you try it out in a simulation? You
might make an error in your "thought experiment", you know...

In my example of the automobile trip, I showed that even a
microscopic error in computing the angle of the steering wheel or
the steering effort would soon have very large effects on the
position of the car in its lane. This is what is wrong with any
open-loop control method.

What I described in my post, on which I presume you comment here, is
NOT open-loop control; I explicitly stated that x is available at all
(sample) times. Try the formulas out in a simulation and check
whether there is an "integration error" -- the formulas are even
easier than their PCT counterparts...

By the way: I cannot make sense of Rick's post. He accuses me of
describing a controller, it seems. I admit that I do. And that I
describe a PCT controller, it seems. Which I do not. I am glad,
however, that you recognize the similarities, Rick! We're both
talking about control, you know ;-).

Greetings,

Hans