new MCT controller

[HANS_BLOM2]

[Hans Blom, 970321]

I've completed the analysis of the new MCT controller, based on the
motion equations that we now agree on. But not programmed it yet.
That will be straightforward, however. Expect the program some time
after the weekend. Here is the analysis.

The equations that govern our idealized theodolite's motion are:

a(t) = (u(t) + d(t)) / J (1)
v(t+dt) = v(t) + a(t) * dt (2)
x(t+dt) = x(t) + v(t) * dt + 0.5 * a(t) * dt * dt (3)

The objective of the MCT controller is:

x(t+dt) = r(d+dt) (4)

I assume that both x(t) and v(t) are observed at every iteration
(other variants are possible, but this assumption may give the best
comparison with the PCT controller). Start by combining (1), (3) and
(4):

r(t+dt) = x(t) + v(t) * dt + 0.5 * (u(t) + d(t)) * dt * dt / J (5)

For the moment, we assume that d(t) is known (it isn't, of course,
but its estimate is; for the estimation of the true disturbance see
below). Therefore the only unknown in (5) is u(t). We derive an
explicit solution for u(t). From

0.5 * (u(t) + d(t)) * dt * dt / J = r(t+dt) - x(t) - v(t) * dt (6)

we get

u(t) = 2.0*J * (r(t+dt) - x(t) - v(t) * dt) / (dt*dt)) - d(t) (7)

This is the equation that the controller uses to compute its output.
Note that if d(t) is exactly equal to the true disturbance and if
u(t) is not limited by saturation of the controller's output, the
control objective (4) is realized exactly (and in a single step!).

We want, however, to make the controller robust against output
saturation: maybe the actual u(t) given is different from what we
computed in (7). In either case, the next predicted observation is,
given the value of u(t) that is the true output:

x(t+dt) = x(t) + v(t) * dt + 0.5 * (u(t) + d(t)) * dt * dt / J (8)

where we have combined (1) and (3). Note its resemblance to (5).

In (8), the only unknown is d(t); we have used its estimate, not its
true value. If there is a difference between both, (8) will not give
a correct prediction. Let us call the true perception at time t+dt by
the name of x'(t+dt) and the true disturbance at time t by the name
of d'(t). The formula that "explains" the true perception is then:

x'(t+dt) = x(t) + v(t) * dt + 0.5 * (u(t) + d'(t)) * dt * dt/J (9)

Subtracting (8) from (9) gives:

x'(t+dt) - x(t+dt) = 0.5 * (d'(t) - d(t)) * dt * dt / J (10)

Now we can reconstruct d'(t):

d'(t) = d(t) + 2.0 * J * (x'(t+dt) - x(t+dt)) / (dt*dt) (11)

Note that the correct value of the true disturbance d'(t) can only be
known at time d+dt, when we observe x'(t+dt). Thus its estimate
always runs one sample late. As a result, we can predict that the
controller will perform badly when the disturbance consistently
varies significantly between samples. The assumption about the
disturbance (the disturbance model) is, indeed, that it is constant:

d(t+dt) = d(t) (12)

Since we have redundant measurements, the estimation of d'(t) could
be done using equation (2) as well, or any (arbitrarily weighted)
combination of both the expressions for x(t+dt) and v(t+dt). The same
applies to the computation of u(t); it too could be based on equation
(2), but that would be control of _velocity_. That, by itself, would
introduce what Bill calls "integration error", accumulation of small
computational inaccuracies, so it ought to be combined with control
for x. But those are unimportant niceties for now.

This completes the analysis of the MCT controller -- unless I've made
errors in my computations, of course...

Greetings,

Hans

[Martin_Taylor6]

[Martin Taylor 970321 10:05]

[Hans Blom, 970321]

I've completed the analysis of the new MCT controller, based on the
motion equations that we now agree on. But not programmed it yet.
That will be straightforward, however. Expect the program some time
after the weekend. Here is the analysis.

The equations that govern our idealized theodolite's motion are:

a(t) = (u(t) + d(t)) / J (1)
v(t+dt) = v(t) + a(t) * dt (2)
x(t+dt) = x(t) + v(t) * dt + 0.5 * a(t) * dt * dt (3)

The objective of the MCT controller is:

x(t+dt) = r(d+dt) (4)

I assume that both x(t) and v(t) are observed at every iteration

By whom? By the analysis program, I hope. Not by the controller,
I hope. The controller should observer only every Dt >> dt, shouldn't
it?

(other variants are possible, but this assumption may give the best
comparison with the PCT controller). Start by combining (1), (3) and
(4):

r(t+dt) = x(t) + v(t) * dt + 0.5 * (u(t) + d(t)) * dt * dt / J (5)

For the moment, we assume that d(t) is known (it isn't, of course,
but its estimate is; for the estimation of the true disturbance see
below).

Those "dt"s should be "Dt"s. Otherwise no experimenter will know how the
theodolite moves between the moments when the controller observes it, and
even if the experimenter could know it, the computed motion of the theodolite
would be what the controller thinks it is rather than approximating
what it would be in a continuous world.

Therefore the only unknown in (5) is u(t). We derive an
explicit solution for u(t). From

0.5 * (u(t) + d(t)) * dt * dt / J = r(t+dt) - x(t) - v(t) * dt (6)

we get

u(t) = 2.0*J * (r(t+dt) - x(t) - v(t) * dt) / (dt*dt)) - d(t) (7)

This is the equation that the controller uses to compute its output.

As written, you are saying that the controller changes its output every
time the experimenter looks at the simulated world. How does the
controller know that the experimenter is looking? You haven't included
a sensor in the controller for "I'm being looked at," so far as I can
see. Explain how the controller knows to observe the theodolite only
at moments the experimenter observes it, and every time the experimenter
observes it, and you may convert me.

Note that if d(t) is exactly equal to the true disturbance and if
u(t) is not limited by saturation of the controller's output, the
control objective (4) is realized exactly (and in a single step!).

A single experimenter observation interval, not a single step, at least
so far as we can tell from all this. The experimenter observation
interval determines how realistically the theodolite motion emulates
the motion of a theodolite in the continuous world. You'll convince
me of this "single step" claim when you allow the theodolite motion
to be observed at a nanosecond time scale, if the experimenter so
chooses, while the controller is observing every millisecond or
every second.

...
This completes the analysis of the MCT controller -- unless I've made
errors in my computations, of course...

Can't say about that. But unless you separate the concepts of dt, the
experimenter's observation interval onto the simulated world, and Dt,
the controller's observation interval onto the theodolite, you won't
have said anything relevant to "single step" correction, or indeed
about anything relevant to what MCT controllers with sampled
observations do in the real world.

You have often said that in practice, MCT controllers perform well in
the real world. Why won't you give us a chance to see that a simulated
MCT controller would work as claimed, in a world we are allowed to
observe as closely as we wish?

Martin

[Bill_Powers1]

[From Bill Powers (970322.1344 PST)]
Data connection in phone in room works just fine.

Hans Blom, 970321--

Those "dt"s should be "Dt"s. Otherwise no experimenter will know how the
theodolite moves between the moments when the controller observes it, and
even if the experimenter could know it, the computed motion of the
theodolite would be what the controller thinks it is rather than
approximating what it would be in a continuous world.

The point you're raising here is important and valid _in general_, but I've
tried to set up this particular problem so that dt and DT can be the same.
This is possible only because u(t) is known to be a constant between steps.
The equations of motion to which Hans has agreed are correct for any size of
dt (actually, DT) -- even, for example, DT = 10 seconds. This is a
convenience for the programmer, because it's not necessary to set up a
separate, special, calculation of the theodolite equations in which many
small steps are taken during each interval DT, which would be necessary if
u(t) were a general function of time. I emphasize that if u(t) were NOT
constant between steps, the more complicated way of doing the theodolite
equations would be needed, and the difference between dt and DT would be
critical.

Best,

Bill P.

[Bill_Powers1]

[From Bill Powers (970322.1355 PST)]

Hans Blom, 970321--

I have a question about one of your steps:

When you solve for the value of r(t+dt) you use the equation that predicts
the next value of x:

x(t+dt) = x(t) + v(t) * dt + 0.5 * a(t) * dt * dt.

Note that this prediction is based on the _current_ values of x, v, and a,
where a(t) = (u(t) + d(t))/J. By our convention, these are the values that
exist at the _beginning_ of the interval designated by t.

To compute a new value of u(t)-- that is, u(t+dt) -- you use the same
equation, substituting r(t+t) for x(t+dt). This is the step I am asking about.

r(t+dt) = x(t) + v(t) * dt + 0.5 * (u(t) + d(t)) * dt * dt / J (5)

For the moment, we assume that d(t) is known (it isn't, of course,
but its estimate is; for the estimation of the true disturbance see
below). Therefore the only unknown in (5) is u(t). We derive an
explicit solution for u(t). From

0.5 * (u(t) + d(t)) * dt * dt / J = r(t+dt) - x(t) - v(t) * dt (6)

we get

u(t) = 2.0*J * (r(t+dt) - x(t) - v(t) * dt) / (dt*dt)) - d(t) (7)

Since we have not yet advanced the time index, the time t still refers to
the values at the beginning of the current interval. While equation (7) is
true, it can't be solved for a NEW value of u(t), since the value of u(t)
was established at the beginning of this interval. The value of u(t) is
given; it's not an unknown in equation 7. If you do calculate it as implied,
you're substituting a different value of u(t) for the value that has already
occurred. In other words, you're solving for the value of u(t) which, if
that had actually been its value, would have made the next value of x equal
to r. But that was NOT its value. Its value was the result of the _previous_
iteration, which started at t-dt and gave rise to u(t).

It might be a good idea to lay out a time-line and establish clearly when
the various calculations are done, and to what values of the variables the
time index refers. This question becomes critical right during these
calculations.

Best,

Bill P.

[HANS_BLOM2]

[Hans Blom, 970324c]

(Martin Taylor 970321 10:05)

I assume that both x(t) and v(t) are observed at every iteration

By whom? By the analysis program, I hope. Not by the controller,
I hope. The controller should observer only every Dt >> dt,
shouldn't it?

No, I'm building a model for the controller to use. The observer can
check, with a microscope or oscilloscope if you want, how the
controller functions.

Those "dt"s should be "Dt"s. Otherwise no experimenter will know how
the theodolite moves between the moments when the controller
observes it, and even if the experimenter could know it, the
computed motion of the theodolite would be what the controller
thinks it is rather than approximating what it would be in a
continuous world.

The experimenter can chose any observation intervals or moments as
he/she wants. If the controller is well designed, there will be no
"surprises" between the moments the controller changes its output.
One standard requirement is that control is "fast enough" to handle
the dynamics of the thing it is to control, and thus that the
controller's dt is small enough. The assumed smoothness or inertia of
the physical world will result in certain smooth closed loop behavior
which is, in every detail, open to inspection by an outside analyst,
but _not_ by the rather dumb controller which relies on the knowledge
imparted it by the designer ;-).

As written, you are saying that the controller changes its output
every time the experimenter looks at the simulated world.

That is a funny interpretation. The controller "looks at its world"
and "acts on its world" every dt seconds. Or every two dt seconds. Or
whatever. Depends on the design. Once in action, it operates
according to its software and fully independently of whatever an
observer may do. I'm not telling you something new, am I?

How does the controller know that the experimenter is looking?

Martin, you're playing with words, not contributing to understanding.

Greetings,

Hans

[HANS_BLOM2]

[Hans Blom, 970324]

(Bill Powers (970322.1355 PST))

x(t+dt) = x(t) + v(t) * dt + 0.5 * a(t) * dt * dt.

Note that this prediction is based on the _current_ values of x, v,
and a, where a(t) = (u(t) + d(t))/J. By our convention, these are
the values that exist at the _beginning_ of the interval designated
by t.

Huh? An _interval_ designated by t? My assumption here is that u can
be computed "immediately" after x and v have been observed. In
practice, that is a very good approximation. It only requires that
the time required for the computation of u is very short compared to
dt. If this is not true, the situation becomes more complex because
we have an extra delay in the feedback loop. Would you want to
incorporate an extra feedback delay in both the MCT and the PCT
controllers?

To compute a new value of u(t)-- that is, u(t+dt) -- you use the
same equation, substituting r(t+t) for x(t+dt). This is the step I
am asking about.

I assume, indeed, that at time t we know r(t+dt). Adding time indices
to the PCT controller expressions may yield some insight...

u(t) = 2.0*J * (r(t+dt) - x(t) - v(t) * dt) / (dt*dt)) - d(t) (7)

Since we have not yet advanced the time index, the time t still
refers to the values at the beginning of the current interval. While
equation (7) is true, it can't be solved for a NEW value of u(t),
since the value of u(t) was established at the beginning of this
interval. The value of u(t) is given; it's not an unknown in
equation 7.

What do you mean by "equation (7) is true"? What is it used for?
Which is the dependent variable? Or is there none?

It might be a good idea to lay out a time-line and establish clearly
when the various calculations are done, and to what values of the
variables the time index refers. This question becomes critical
right during these calculations.

Will do. Soon.

Greetings,

Hans

[Martin_Taylor6]

[Martin Taylor 970324 09:50]

[Hans Blom, 970324c]

There's not much point in going on saying "you don't get the point"
to each other, I guess.

The experimenter can chose any observation intervals or moments as
he/she wants.

Good. Put that into your code, so that the experimenter using your
code can do so.

If the controller is well designed, there will be no
"surprises" between the moments the controller changes its output.

An assertion that can't be tested until your code separates Dt from dt.

One standard requirement is that control is "fast enough" to handle
the dynamics of the thing it is to control, and thus that the
controller's dt is small enough. The assumed smoothness or inertia of
the physical world will result in certain smooth closed loop behavior
which is, in every detail, open to inspection by an outside analyst,
but _not_ by the rather dumb controller which relies on the knowledge
imparted it by the designer ;-).

Yes, that's the point I've been trying to get you to put into the code.

As written, you are saying that the controller changes its output
every time the experimenter looks at the simulated world.

That is a funny interpretation. The controller "looks at its world"
and "acts on its world" every dt seconds.

Exactly the moments when the experimenter has a chance to look at the
world, too. Not, I think, a funny interpretation at all.

Or every two dt seconds. Or
whatever. Depends on the design. Once in action, it operates
according to its software and fully independently of whatever an
observer may do. I'm not telling you something new, am I?

No. But maybe you are telling yourself something that you refuse to
see. The theodolite itself should be a simulation that the experimenter
can view whenever the experimenter wants, not at only those moments
when the controller is looking. Put it that way round, if you like--the
experimenter can look only when the controller says "OK, you can look
now, because this is one of those moments when I'm sure the theodolite
is doing what I tell it to do."

How does the controller know that the experimenter is looking?

Martin, you're playing with words, not contributing to understanding.

I hate "playing with words" in a technical discussion, for all that I
love to do so for fun. No, I'm just trying to get you to submit code that
allows the experimenter to test the assertion you make with zero support,
that the theodolite comes to its desired position at time T = 0+dt and
never deviates from that position at T = 0+dt+epsilon, where epsilon may
not be an integer multiple of dt.

In a related post, you seemed to be playing with words that amount to an
assertion that the simulated theodolite DOES NOT EXIST except when the
controller is looking at it (actually in several posts including today).
You obfuscate with irrelevancies such as:

I don't want to model the theodolite's heat
exchange with its environment or the way it reflects light.

which is, I suppose, a response to my suggestion that you would like to
be simulating a controller of a theodolite that reacts to torque in the
way a real theodolite (or an idealized one without friction) would in the
real world.

Let's try one last time, with a very specific question. Where would the
experimenter see the theodolite at time T = 0+0.2*dt, 0+0.4*dt, 0+0.6*dt,
0+0.8*dt, 0+1.2*dt, and 0+1.4*dt (we know where it will be seen at 0+dt).

Martin

[HANS_BLOM2]

[Hans Blom, 970324f]

(Martin Taylor 970324 09:50)

There's not much point in going on saying "you don't get the point"
to each other, I guess.

Not much, no. But I like your specific question:

Where would the experimenter see the theodolite at time T =
0+0.2*dt, 0+0.4*dt, 0+0.6*dt, 0+0.8*dt, 0+1.2*dt, and 0+1.4*dt (we
know where it will be seen at 0+dt).

The following code -- to be found in the program -- will tell you
where the experimenter could observe the _simulated theodolite_ at T
= 0+0.2*dt, 0+0.4*dt, 0+0.6*dt, and 0+0.8*dt.

  specify dt = 0.2, 0.4, 0.6 or 0.8;
  aa := (u + disturbance) / J;
  xx := xx + vv*dt + 0.5*aa*dt*dt;
  vv := vv + aa*dt;

where u, disturbance and thus also aa are assumed to have gotten
their value at T=0 and to remain constant during one full dt. That is
what we agreed the model ought to do. Let's not quarrel whether that
is realistic. Note that aa remains constant during dt, vv increases
linearly and xx quadratically with dt.

To observe the _simulated theodolite_ at T = 0+1.2*dt and 0+1.4*dt,
you will have to wait till the controller has performed one iteration
around the loop, subtract dt, and use the above code with T = 0 +
0.2*dt and 0 + 0.4*dt. The reason that you have to wait is that at T
= 0 + 1.0*dt the controller will deliver a new output that will
change the value of aa and the trajectories of xx and vv. Unless you
want to simulate the simulation and also calculate what the next
value of u will be. In theory, that is possible. In practice, small
computational errors will introduce deviations from the theoretical
solution.

Good enough?

Greetings,

Hans

[Martin_Taylor3]

[Martin Taylor 970324 13:45]

Hans Blom, 970324f]

(Martin Taylor 970324 09:50)

>Where would the experimenter see the theodolite at time T =
>0+0.2*dt, 0+0.4*dt, 0+0.6*dt, 0+0.8*dt, 0+1.2*dt, and 0+1.4*dt (we
>know where it will be seen at 0+dt).

The following code -- to be found in the program -- will tell you
where the experimenter could observe the _simulated theodolite_ at T
= 0+0.2*dt, 0+0.4*dt, 0+0.6*dt, and 0+0.8*dt.

  specify dt = 0.2, 0.4, 0.6 or 0.8;
  aa := (u + disturbance) / J;
  xx := xx + vv*dt + 0.5*aa*dt*dt;
  vv := vv + aa*dt;

where u, disturbance and thus also aa are assumed to have gotten
their value at T=0 and to remain constant during one full dt. That is
what we agreed the model ought to do. Let's not quarrel whether that
is realistic. Note that aa remains constant during dt, vv increases
linearly and xx quadratically with dt.

To observe the _simulated theodolite_ at T = 0+1.2*dt and 0+1.4*dt,
you will have to wait till the controller has performed one iteration
around the loop, subtract dt, and use the above code with T = 0 +
0.2*dt and 0 + 0.4*dt. The reason that you have to wait is that at T
= 0 + 1.0*dt the controller will deliver a new output that will
change the value of aa and the trajectories of xx and vv. Unless you
want to simulate the simulation and also calculate what the next
value of u will be. In theory, that is possible. In practice, small
computational errors will introduce deviations from the theoretical
solution.

Good enough?

I guess I'm terminally stupid. In the code as of [Hans Blom, 970324] I see
a loop as follows:

···

-------------------

begin
  dt := 0.1; {or any other value}
  t := 0.0; {start at t=0}
  r := 0.0; {initially zero}
  x := 0.0; {initially zero}
  v := 0.0; {initially zero}
  d := 0.0; {estimate of disturbance}
  repeat
    writeln (t:12:3, r:12:6, x:12:6, v:12:6, d:12:6, u:12:6);
    r := reference (t+dt); {specify desired x}
    control; {controller computes u}
                                  {predict next observation}
    p := x + v*dt + 0.5 * (u + d) * (dt*dt) / J;
                                  {time shifts by dt here}
    observe; {observe x (and v?)}
                                  {update disturbance estimate}
    d := d + 2.0 * J * (x-p) / (dt*dt)
  until t > 9.0
end.

----------------

It STILL looks to me as if there is one execution of control for every
execution of observe, and that happens at each increment of dt. Could you
explain how this lets me see what is happening to the theodolite at times
0+0.2dt, 0+0.4dt..., 0+1.2dt, 0+1.4dt (in the example, those are at T=0.02,
T=0.04,...,T=0.12, T= 0.14? I don't see how the experimenter backs up time
after T=0+dt to see the historical values for T=0.2dt etc. as you seem to
suggest should be done.

If I read the code correctly, it still doesn't answer the question, so it
is not "Good Enough."

I'd be quite happy to allow Dt to be an integer number of dt increments,
so maybe an amendment like the following might work (I'm not a Pascal
programmer, so forgive syntax errors):

In the list of variables at the beginning of the loop, add M=10 (or any
other value above 2). Where it says "control" put
  if (t % (M*dt) == 0) then control;

If I got that right, the controller will observe, and therefore will change u
only every Mth time the experimenter observes. If your claim for the controller
is correct, the error should be zero for all values of t equal to or
greater than M*dt after a sudden change of the reference level to a new
steady value.

Fair enough? (Allowing from the bastard mixing that I am sure must be there,
between C and Pascal syntax).

Martin

[HANS_BLOM2]

[Hans Blom, 970325g]

(Martin Taylor 970324 13:45)

If I read the code correctly, it still doesn't answer the question,
so it is not "Good Enough."

In the code below, you can specify any observation time at all, in
variable tobs. This is in an open-loop theodolite, but the exact same
code can be used when the theodolite is controlled. How's that?

Greetings,

Hans

program theodolite_observer;

const
  J = 1.0; {theodolite characteristic}

var
  tobs, {extra observation at this time}
  x, {perceived position}
  v, {perceived velocity}
  u, {controller's output}
  d, {disturbance}
  r, {reference level}
  dt, {time increment; var so it can be changed}
  t: real; {time; for display/loop control purposes}
  c: integer; {counter}

procedure observe;
{this procedure implements the theodolite motion equations; it
generates the NEXT observation(s) x(t+dt) and v(t+dt), given u(t);
it also shifts the time by dt}
const
  aa: real = 0.0;
  vv: real = 0.0;
  xx: real = 0.0;
begin
  aa := (u+d) / J;
  xx := xx + vv*dt + 0.5*aa*dt*dt;
  vv := vv + aa*dt;
  t := t + dt;
  x := xx; v := vv {the controller may use these}
end;

procedure observex (tt: real);
{simulate an extra observation at time t+tt, any time}
var
  aa, vv, xx: real;
begin
  aa := (u+d) / J;
  xx := x + v*tt + 0.5*aa*tt*tt;
  writeln ('extra observed position follows:');
  writeln (tobs:12:6, xx:36:6)
end;

begin
  c := 0; {counter}
  dt := 0.001; {or any other value}
  t := 0.0; {start at t=0}
  x := 0.0; {initially zero}
  v := 0.0; {initially zero}
  u := 1.0; {constant acceleration}
  d := 0.0; {no disturbance}
  tobs := 1.999999; {specify extra observation time}
  repeat
    if (tobs > t) and (tobs < t+dt) then {right iteration}
      observex (tobs - t);
    if (c mod 100) = 0 then {show results}
      writeln (t:12:6, v:12:6, x:12:6, 0.5*u*t*t/J:12:6);
                                  {time shifts by dt here}
    observe; {observe x (and v?)}
    inc (c)
  until t > 2.001
end.

[Martin_Taylor3]

[Martin Taylor 970325 13:45]

Hans Blom, 970325g]

(Martin Taylor 970324 13:45)

>If I read the code correctly, it still doesn't answer the question,
>so it is not "Good Enough."

In the code below, you can specify any observation time at all, in
variable tobs. This is in an open-loop theodolite, but the exact same
code can be used when the theodolite is controlled. How's that?

Well. it goes partway. Now if you include something like this in your
code, where tobs = {0.2dt, 0.4dt....N*02.dt}, we might have something.
(And that applies to looking at Bill Powers' code, too).

···

----------------------
+Hans Blom, 970325b

+That is, which time
+index should rv (the reference for velocity) get? v(t) is already
+known -- and thus fixed -- at this time, so it makes no sense to
+specify a reference rv(t) for it. Doing so would be like saying "I
+want this value, which is at 0, to be 1". That is as unreasonable as
+saying that 0 = 1.

What is unreasonable in saying "At this moment I want to see v=1, and
at this moment I actually see v = 0, so at this moment there is a
discrepancy of 1 unit between what I want to see and do see"? You
seem to be asserting that I cannot want anything to be different from
the way I currently see it.

+And _when_ would we like v
+to have its new value? At t+dt, preferably, if at all possible. So
+the interpretation of (3) is
+
+ rv(t+dt) := 0.0

How on earth is the controller going to guess what the reference value
will be at a time dt in the future? That value is entirely in the hands
of the Gods who send the reference value down from on high (the experimenter
in this case).

As for your question "when would we like v to have its new value?" the
answer clearly is "Now." Of course we can't always have what we want,
but, being control systems, we act to bring it about, nevertheless.

If, at time t+dt we still happen to have the same reference value, 1, for
v, and also our actions have served to bring v to this value, then we will
be happy--there will be no error then. But v may have come nicely to the
value v=1, and by then the Gods may have told the controller that its reference
is now (at t+dt) rv=-20. And that will BE the reference value at t+dt, and
the error will be +21 despite v having (in one step) come to the value we
used to want, at time t.

+rx(?) := 0.0; (4)
+
+>>> Same thing: What is the interpretation of this statement? x(t) is
+known at this time, so it makes no sense to specify a reference rx(t)
+for it. Doing so would be like saying "I want this x(t), which I know
+is at 3 right now, to be at 4". At 4 _when_?

Now. As you say, "same thing." I cannot know what my reference will be for
x at time (t+dt).

Of course, I can easily (now) hold reference values for what I want x to
be at t+k*dt. But they are not necessarily the values of the reference
for x at time t+k*dt I will have at time t+k*dt.

In other words, r(t,(x+dt)) is a different variable from
r(t+dt, x(t+dt)).

I can want anything within my capacity to perceive, regardless of what I
currently perceive to be the case. I can act so as to bring my perceptions
closer to my wants, but I cannot know what my wants will be when the
time comes that all the perceptions have arrived where I initially wanted
them to be.

+If we make this explicit, the interpretation of (4) is
+
+ rx(t+dt) := 0.0 {or any other value};
+
+Strange enough, this interpretation (and which other interpretation
+is reasonable?) ...

Well, not that one. What is reasonable is that I now (time = t) have a
reference value, and that I now (time = t) have an actual value.

+...tells us that the PCT controller strives for "single
+step" error correction as well!

PCT doesn't have the concept of a "step." PCT operates in an analogue
world with continuous time. PCT is simulated in a discrete world of digital
computation, using sampling interval dt small enough that making it smaller
makes no discernable difference to the result (whether or not the output
force saturates). I, a control system, may "strive for" instantaneous
correction of all perceptual errors, but I don't get it because the world
has mass, transport lags, and so on. That doesn't invalidate PCT. What
would invalidate PCT is if the simulation models characteristically behaved
in a way unlike the way the biological control systems turn out to behave
when they are observed.

+There are two ways, I think, to compare the PCT and MCT controllers.
+The first is a mathematical analysis, such as I've given here for the
+PCT controller

Wrongly, I'm afraid.

+The second way is to put the _programs_ side by side and check that
+the _sequence of statements_ is identical and that the only real
+difference is in the way u is computed

That sounds reasonable to me. Both should allow the experimenter to observe
on a time-scale independent of the time-scale used by the controller.
Since the PCT controller observes _all the time_ while the MCT controller
observes only at intervals Dt, there will be a difference there. But it
could be resolved by a two-stage process; let the controllers both observe
on a time-scale Dt, which is reduced until changing it no longer affects
the PCT result. Use that as the time-scale of observation for both. Then
use an experimenter observation interval on what is happening of dt=0.1*Dt.

Would that be fair?

Martin

[Martin_Taylor3]

[Martin Taylor 970325 14:45]

Martin Taylor 970325 13:45]

My message of an hour ago just arrived--rather more quickly than the message
of six weeks ago:-)

But it has an ambiguity I'd like to clear up.

+Hans Blom, 970325b

+rx(?) := 0.0; (4)
+
+>>> Same thing: What is the interpretation of this statement? x(t) is
+known at this time, so it makes no sense to specify a reference rx(t)
+for it. Doing so would be like saying "I want this x(t), which I know
+is at 3 right now, to be at 4". At 4 _when_?

Now. As you say, "same thing." I cannot know what my reference will be for
x at time (t+dt).

Of course, I can easily (now) hold reference values for what I want x to
be at t+k*dt. But they are not necessarily the values of the reference
for x at time t+k*dt I will have at time t+k*dt.

In other words, r(t,(x+dt)) is a different variable from
r(t+dt, x(t+dt)).

I think this would be more intelligble if I said: r(tau,x(tau+dt)) is a
different variable from r(tau, x(tau)), whether tau = t or tau = t+dt or
tau = t + k*dt.

(Apart from the obvious typo in the original).

Sorry about that.

Martin

[HANS_BLOM2]

[Hans Blom, 970326c]

In the code below, you can specify any observation time at all, in
variable tobs. This is in an open-loop theodolite, but the exact
same code can be used when the theodolite is controlled. How's
that?

Well. it goes partway. Now if you include something like this in
your code, where tobs = {0.2dt, 0.4dt....N*02.dt}, we might have
something.

That would generate a terrific amount of output to sort through,
Martin. I've given the general idea. What you suggest is entirely
possible, but specifying a _list_ of times at which to present
additional observations might be more suitable. The experiment is,
after all, perfectly reproducible. But I leave that up to you.

What is unreasonable in saying "At this moment I want to see v=1,
and at this moment I actually see v = 0, so at this moment there is
a discrepancy of 1 unit between what I want to see and do see"? You
seem to be asserting that I cannot want anything to be different
from the way I currently see it.

No, I do not assert that. You can want anything your heart desires.
I'm talking about _control_. You may have such a desire, but you
cannot control for it: there is no way in which actions may result in
the fulfilment of that particular, unfulfillable desire. And I'm very
reluctant to design a controller that is so completely ineffectual.

How on earth is the controller going to guess what the reference
value will be at a time dt in the future? That value is entirely in
the hands of the Gods who send the reference value down from on high
(the experimenter in this case).

Martin, where does a reference level come from? From inside the
organism, we usually suppose. And often we find that the organism is
an effective control system. So not all its goals are unrealistic...

As for your question "when would we like v to have its new value?"
the answer clearly is "Now." Of course we can't always have what we
want, but, being control systems, we act to bring it about,
nevertheless.

Sorry, but no. Not even a controller can achieve the unachievable.

Since the PCT controller observes _all the time_

Which PCT controller do you have in mind here?

Greetings,

Hans

[Martin_Taylor6]

[Martin Taylor 970326 15:50]

[Hans Blom, 970326c]

In the code below, you can specify any observation time at all, in
variable tobs. This is in an open-loop theodolite, but the exact
same code can be used when the theodolite is controlled. How's
that?

Well. it goes partway. Now if you include something like this in
your code, where tobs = {0.2dt, 0.4dt....N*02.dt}, we might have
something.

That would generate a terrific amount of output to sort through,
Martin.

Who would do that? I presume it would ordinarily be graphed, like
the output of most of Bill's programs.

How on earth is the controller going to guess what the reference
value will be at a time dt in the future? That value is entirely in
the hands of the Gods who send the reference value down from on high
(the experimenter in this case).

Martin, where does a reference level come from? From inside the
organism, we usually suppose. And often we find that the organism is
an effective control system. So not all its goals are unrealistic...

That doesn't come close to answering the question of how the control
system at time t can know/guess/model what its reference value for
a perception will be at time t+dt in the future. I have no qualms at
all about statements that at time t one can have a reference value
for what the perception will be at time t+dt. But the value of the
reference is its value NOW. And even if NOW I can't affect the value
of the perception is have NOW, nevertheless I can NOW want the perception
to have its reference value, and act to make it so, without specifying
any particular moment when it should take on that reference value. All
I need say is "as soon as I can, and thereafter, unless my reference
value changes in the interim."

You need a different notation to label "the reference value at time
tau for signal x" and "the reference value for (signal x at time tau)."
In my previous post I used (without explanation) r(tau,x(t)) to signify
the value at time tau of the reference signal for (x at time t). Your
analysis referred to the value at time t+dt of the reference for signals
x and v, whereas you should have dealt with the value of the reference
at time t for the signals at time t+dt (if their values at that future
time--and only at that moment--was what interested your controller).
In my notation, you dealt with r(t+dt, x(t+dt)) (which you said in
words was ridiculous) rather than with r(t, x(t+dt)) (which in words
was what you said you were interested in, for reasons that remain
obscure).

Moving on one step. Assuming that the reference value is 1.0, why does
your model controller limit itself only to r(t, x(t+dt)) = 1.0? Why does
it not also have r(t, x(t+2*dt)) = 1.0, r(t, x(t+3*dt)) = 1.0 ...?

As for your question "when would we like v to have its new value?"
the answer clearly is "Now." Of course we can't always have what we
want, but, being control systems, we act to bring it about,
nevertheless.

Sorry, but no. Not even a controller can achieve the unachievable.

Do you always argue in an alternation of tautology and irrelevance?
You don't do it in person. Why in e-mail?

Since the PCT controller observes _all the time_

Which PCT controller do you have in mind here?

The one being simulated in the discrete, computer-based, simulation.
The simplest is the one that has a filter function on its input that
produces a _continuously varying_ perceptual signal, which is compared
with a _continuously varying_ reference signal to provide a _continuously
varying_ error signal, which serves as an input to an output function
that is a continuous function of time (typically an integrator) and
which produces as output a _continuously varying_ signal that through
various transducers (which are continuous functions of time) has effects
on the inputs to the filter function mentioned first, in conjunction
with the effects of a _continuously varying_ independent signal called
a disturbance.

The fact that a digital computer can't make continuously varying signals
or filter functions in no way alters the fact that the PCT controller
being simulated observes its world all the time.

Martin

[HANS_BLOM2]

[Hans Blom, 970327c]

(Martin Taylor 970326 15:50)

Martin, where does a reference level come from? From inside the
organism, we usually suppose. And often we find that the organism
is an effective control system. So not all its goals are
unrealistic...

That doesn't come close to answering the question of how the control
system at time t can know/guess/model what its reference value for a
perception will be at time t+dt in the future.

Yes, that seems to be one of the central questions. I will, however,
demonstrate -- in due time; I have to go slowly -- that a PCT
controller ALSO controls for _future_ values of the perception to be
at certain reference values. That is simple logic, it seems to me: an
observed value cannot be changed anymore. Yet, more than logic seems
to be needed. And that I simply cannot understand.

I have no qualms at all about statements that at time t one can have
a reference value for what the perception will be at time t+dt. But
the value of the reference is its value NOW.

Yes. NOW I have to know (decide, compute, control for) where I WANT
TO go. You might call that "planning", in our case over a possibly
infinetisimally small time period dt. You are right: the reference r
for x at time t+dt must be known at time t. That is, the comparator
compares r(t+dt) and x(t), and the controller's output is driven by
the difference. And that in the PCT controller as well! Surprise?

And even if NOW I can't affect the value of the perception is have
NOW, nevertheless I can NOW want the perception to have its
reference value, and act to make it so, without specifying any
particular moment when it should take on that reference value.

How would you want to model that, i.e. express it in mathematics or
in computer code? I need more than words, Martin...

All I need say is "as soon as I can, and thereafter, unless my
reference value changes in the interim."

In my MCT theodolite controller, the "as soon as I can" translates
into "after a time increment dt", which proves to be possible (unless
the controller's output saturates). We also stipulated that during dt
the reference value is constant. So I actually seem to do what you
suggest...

You need a different notation to label "the reference value at time
tau for signal x" and "the reference value for (signal x at time
tau)."

I think (hope!) I understand you. I am saying that the reference for
x(t+dt) should already be known at time t. Yet I call it r(t+dt).
That may be confusing. The alternative would be to call it r(t), but
with the understanding that it is the reference for x(t+dt). That
seems only a change in names. I'm used to (and the literature always
uses) my notation; I'm hardly original ;-). I hesitate to start a
sect that uses a name different from the rest of the world for no
good reasons.

Or do you have something deeper in mind?

Moving on one step. Assuming that the reference value is 1.0, why
does your model controller limit itself only to r(t, x(t+dt)) = 1.0?
Why does it not also have r(t, x(t+2*dt)) = 1.0, r(t, x(t+3*dt)) =
1.0 ...?

For reasons of simplicity. If 1) the measurements are noise-free and
2) the controller output does not saturate, there is no need to do
so, and my formulation is mathematically equivalent and simpler. If
either of these conditions is not fulfilled, your approach may be
needed -- and you will indeed find it in the literature. It requires
that a "plan" is available (a reference _trajectory_ over some finite
or infinite future, e.g. a specification of what the position is to
be at a certain time in the future), and it results in computation of
a control _trajectory_, a sequence of values of u, i.e. u(t),
u(t+dt), u(t+2*dt), etc. If, however, observations take place at
every N*dt, only the first u of this sequence, that is u(t), is
output. The reason is that the new measurement provides additional
information which allows a refinement or update of the model, and
thus a refined control sequence to be computed. In my imagination I
see people doing this all the time ;-).

So your suggestion is a good one. But it isn't required in our simple
example. And in practice it requires quite some computing time and/or
hefty hardware, so you'll encounter it only infrequently and only in
the most advanced applications.

Sorry, but no. Not even a controller can achieve the unachievable.

Do you always argue in an alternation of tautology and irrelevance?

Only when needed, Martin ;-).

Since the PCT controller observes _all the time_

Which PCT controller do you have in mind here?

The one being simulated in the discrete, computer-based, simulation.

That makes it an imaginary one. Imaginary controllers are hard to
analyze, unless you provide a full and explicit description. I was
talking about the "discrete, computer-based, simulation". That is a
PCT controller, too, I guess. And _it_ observes only at discrete time
instants separated by a time period dt.

The fact that a digital computer can't make continuously varying
signals or filter functions in no way alters the fact that the PCT
controller being simulated observes its world all the time.

Am I also allowed to say: "The fact that a digital computer can't
make continuously varying signals or filter functions in no way
alters the fact that the MCT controller BEING SIMULATED (my emphasis)
observes its world all the time"? If not, why not? The only
limitation of an MCT controller that runs in a discrete (sampled)
physical system (a computer) is that "all the time" is an infinite
number of moments which cannot be handled; a _large_ number of
moments _can_ be handled, however. Usually (in bandwidth limited
systems) to good advantage. But you know all that...

Greetings,

Hans

[HANS_BLOM2]

[Hans Blom, 970327d]

(Martin Taylor 970326 15:500

I forgot to add this:

How on earth is the controller going to guess what the reference
value will be at a time dt in the future?

At least _frequently_ it happens that a reference _trajectory_ is
prescribed. In our theodolite example, for instance, we already know
at time t=0 what the reference position will have to be during the
full 9 seconds that the controller is to operate.

Greetings,

Hans

[Martin_Taylor3]

[Martin Taylor 970327 14:30]

Hans Blom, 970327d]

(Martin Taylor 970326 15:500

I forgot to add this:

>How on earth is the controller going to guess what the reference
>value will be at a time dt in the future?

At least _frequently_ it happens that a reference _trajectory_ is
prescribed. In our theodolite example, for instance, we already know
at time t=0 what the reference position will have to be during the
full 9 seconds that the controller is to operate.

_We_ know. Does your controller know? If so, it knows more than is
being allowed to the PCT controller. The PCT controller _could_ know
it if it were a two-level hierarchy, the upper level being a sequence
perception controller with that (known-to-the-experimenter) sequence
as its reference.

Martin

[HANS_BLOM2]

[Hans Blom, 970407d]

(Martin Taylor 970327 14:30)

How on earth is the controller going to guess what the reference
value will be at a time dt in the future?

At least _frequently_ it happens that a reference _trajectory_ is
prescribed. In our theodolite example, for instance, we already
know at time t=0 what the reference position will have to be during
the full 9 seconds that the controller is to operate.

_We_ know. Does your controller know? If so, it knows more than is
being allowed to the PCT controller.

No. The MCT controller as I implemented it uses no more knowledge
about the reference level's time trajectory than the PCT controller.
A different -- more complex -- MCT controller might do so, however.
Its error criterium would not be the simple minimization of xref - x,
but might be the more complex minimization of

      t_end
    INTEGRAL (xref(tau) - x(tau)) dtau
tau = t_start

or something like that.

The PCT controller _could_ know it if it were a two-level hierarchy,
the upper level being a sequence perception controller with that
(known-to-the-experimenter) sequence as its reference.

I've wondered how one could implement that. Can you provide an
example?

Greetings,

Hans

[Martin_Taylor3]

[Martin Taylor 970408 11:45]

Hans Blom, 970407d]

(Martin Taylor 970327 14:30)

>>>How on earth is the controller going to guess what the reference
>>>value will be at a time dt in the future?

>>At least _frequently_ it happens that a reference _trajectory_ is
>>prescribed. In our theodolite example, for instance, we already
>>know at time t=0 what the reference position will have to be during
>>the full 9 seconds that the controller is to operate.

>_We_ know. Does your controller know? If so, it knows more than is
>being allowed to the PCT controller.

No. The MCT controller as I implemented it uses no more knowledge
about the reference level's time trajectory than the PCT controller.

I find this series of questions and answers more than a little confusing.

My initial question had to do with a notational convention that seemed to
indicate the following: at iteration n, the reference value that was to
be used for the controlled variable was the reference value that would
exist at iteration n+1.

Your answer was that the reference values were known in advance for the
whole time the control simulation would run.

My question was whether the theodolite controller knows in advance this
fact, because if it does not, then the initial answer is wholly irrelevant
to the question.

Your answer is that it does not, making the initial answer wholly irrelevant.

Are you surprised that I am sometimes confused by your explanations and
(apparent non-)answers to my questions? (To avoid confusion, this is a new
question, not a rephrasing of the one quoted at the head of this message).

A different -- more complex -- MCT controller might do so, however.
Its error criterium would not be the simple minimization of xref - x,
but might be the more complex minimization of

      t_end
    INTEGRAL (xref(tau) - x(tau)) dtau
tau = t_start

or something like that.

I don't see the future values of the reference signal represented in this
formulation.

>The PCT controller _could_ know it if it were a two-level hierarchy,
>the upper level being a sequence perception controller with that
>(known-to-the-experimenter) sequence as its reference.

I've wondered how one could implement that. Can you provide an
example?

In what form? Schematically, it's pretty trivial, though I probably should
defer to Bill Powers for the details and code. The sequence is itself
a perception, for which there can be a reference. The input to the
sequence controller is the current value of the perception. The perception
of sequence clearly involves historic time, so there has to be the moral
equivalent of a shift register (there needn't be an actual shift register,
of course, but there has to be something that either evaluates in the
comparator the ongoing sequence deviation, or something in the perceptual
function that generates a representation of the ongoing input as a
current perception of a building sequence). A shift register provides at
its parallel outputs a pattern that is, at that point, indistinguishable
from any kind of configuration. The output function can provide output
that serves as a reference value for the position control level. After
learning, and we assume a properly reorganized system in the same way
you assume a well-trained Kalman filter, the reference level demanded for
a position to be assumed at time t can be provided at time t-k, where
k corresponds roughly to the delay of the environmental feedback function.

The point is that if the desired sequence is known, the sequence control
level can feed references to the position control level for positions to
be taken in the future, so that by the time the position is desired, it will
have been attained. A one-level system needs a model to be able to do this.
A multi-level system doesn't.

Don't hold me to any detail of the sequence-level controller, though. It
isn't something I've much thought about, other than to acknowledge the
sequence level as one of the levels of control. The provision of predictive
reference levels follows from that. The mechanism may be different from
what I suggest--off the top of my head--above. My hat! What that implies

Martin

[HANS_BLOM2]

[Hans Blom, 970410]

(Martin Taylor 970408 11:45)

I find this series of questions and answers more than a little
confusing.

And so do I. We do seem to have very different mental models ;-).

My initial question had to do with a notational convention that
seemed to indicate the following: at iteration n, the reference
value that was to be used for the controlled variable was the
reference value that would exist at iteration n+1.

Your answer was that the reference values were known in advance for
the whole time the control simulation would run.

Yes and no. Let me explain.

The _program_ contains code which (necessarily, of course!) specifies
the reference value for all times (from t=0 to t=9 seconds): the
theodolite is to go from position 0 to position 1 starting at time t1
and stopping at time t2 at constant velocity.

However, the _code section that implements the MCT controller_
currently _uses_, in the iteration that belongs to time t, only the
reference for time t+dt. The same goes for the PCT controller,
although in that case it is not made _explicit_ that the reference
xref in the iteration that belongs to time t prescribes what x(t+dt)
is to be -- even though it does, as I demonstrated in the "bare
bones" controller.

There is no reason, however, why either controller should not use
reference values for other future times -- if they are of any use to
the controller.

My question was whether the theodolite controller knows in advance
this fact, because if it does not, then the initial answer is wholly
irrelevant to the question.

Both the MCT and the PCT theodolite controllers "know" in advance
that at time t only the reference for time t+dt is available. This
knowledge is embedded in the controller's code, because the code only
refers to one reference value (the one for t+dt) at time t, _not_ to
reference values even farther in the future. Different controllers
might, however.

Are you surprised that I am sometimes confused by your explanations
and (apparent non-)answers to my questions?

Yes and no. I'm surprised because I think the answer is clearly
visible -- in the program code that I supplied: I cannot be much
clearer than that -- together with an analysis. No, because even
working back from program code to "meaning" is a hard job, especially
so if someone suggests that extremely familiar code that you have
always interpreted in a certain way has a subtly different meaning.

Its error criterium would not be the simple minimization of xref -
x, but might be the more complex minimization of

      t_end
    INTEGRAL (xref(tau) - x(tau)) dtau
tau = t_start

I don't see the future values of the reference signal represented in
this formulation.

Sorry, change of variables. And an error, which I must correct. I
meant to say that the controller's _current_ time, which we sofar
called t, is the "start" time of a planning episode. Also, the
minimization is that of _squared_ errors. So I'd better change the
above formula into

                  t_end
goal = minimum ( INTEGRAL (xref(tau) - x(tau))^2 dtau )
                  tau = t

This formula describes the controller's overall goal: to minimize the
integral squared error ("sum of squared errors" in a sampled model)
starting from the current time t until some future time t_end, which
might be infinite (or "until death do us part"). If we base the
control law on such a criterium, we might for instance tolerate
errors at particular times if these are more than compensated for by
smaller control errors over longer time periods at other occasions.
This can be called anticipation.

A controller whose control law is based on a formula like the one
above implements "planning". But a necessary requirement for planning
is _that we are able to specify the reference values for future times
right now_. If not, no planning is possible.

Am I being clear now?

The PCT controller _could_ know it [a reference value trajectory]
if it were a two-level hierarchy, the upper level being a sequence
perception controller with that (known-to-the-experimenter)
sequence as its reference.

I've wondered how one could implement that [in a PCT model]. Can
you provide an example?

In what form? Schematically, it's pretty trivial, though I probably
should defer to Bill Powers for the details and code.

Thought experiments are often pretty trivial ;-). I'd like an
implementation as a demonstration that it can be done -- and how. But
I appreciate your thoughts.

The sequence is itself a perception, for which there can be a
reference.

Like "I want to be in LA the day after tomorrow at 3 pm" and a
description of how to get there?

The input to the sequence controller is the current value of the
perception.

Like "I'm not there now"?

The point is that if the desired sequence is known, the sequence
control level can feed references to the position control level for
positions to be taken in the future, so that by the time the
position is desired, it will have been attained. A one-level system
needs a model to be able to do this. A multi-level system doesn't.

There's more. In the theodolite controller, the desired sequence is
fully known beforehand, yet it isn't needed because knowledge of only
the next reference value is sufficient (mathematically equivalent).
Unless _constraints_ are present, such as output saturation or
knowledge that the output will be forced to zero during a certain
time interval (being unable to do something youself, because e.g. the
airplane pilot decides how the trajectory unfolds). Only if such
things are known _beforehand_ will the controller be able to include
their effects in its control strategy. In the case of output
saturation, the controller could consider its task to be to deliver
only such output values that are lower than the saturation limit.
Much harder, because planning becomes necessary, yet doable.

Don't hold me to any detail of the sequence-level controller,
though. It isn't something I've much thought about, other than to
acknowledge the sequence level as one of the levels of control. The
provision of predictive reference levels follows from that.

All right. I've sketched a brief outline of how MCT approaches this.
Now let's see a PCT implementation...

Greetings,

Hans