[From Bill Powers (950531.0845 MDT)]
Hans Blom (950530) --
Something finally sank in. You have mentioned several times computing u
so that the error should be corrected "in a single iteration." I have
finally figured out what you mean.
In the world-model with no observation noise or external disturbance, we
have
(1) x = c + a*x + b*u
The value of x that we want is xopt, so we have
xopt = c + a*x + b*u, and solving for u we get
(2) u = (xopt - c - a*x)/b
Substituting (2) unto (1) for u, we get
x = xopt, in one iteration.
How simple!
My way of modeling a simple control system must have puzzled you
greatly, because customarily I have represented the environmental
feedback connection as a simple constant, without in any way indicating
what approximation is hidden by this representation. As far as my
equations are concerned, if the output attains a value o, then the
controlled quantity (in the absence of a disturbance) must instantly
attain a value Kf*o, where Kf is the proportionality factor representing
the feedback connection. This is going to take some explaining.
I'm going to be talking here about a simple control system with no
internal world model; or alternatively, I will be taking about how your
model controls its internal world-model without considering the real
system outside. The following is a general discussion, Hans, and I fully
expect that you are familiar with the principles involved.
···
-----------------------------------------------------------------------
Suppose we have a controlled variable qi and an output variable o.
The "plant" is represented by the proportionality factor Kf:
(3) qi = Kf*o (Here qi corresponds to your xt, and o to your u)
Suppose we wish qi to come to some value qopt. Then
qopt = Kf*o, or
(4) o = qopt/Kf
Substituting o from (4) into (3), we get
qi := Kf*(qopt/Kf) = qopt.
So qi comes to the value qopt in one iteration. This is the same
approach you have been using.
Note that the loop gain of this system is Kf/Kf = 1.0. If a disturbance
acted on it, it could oppose only half of the effect on qi, which would
be very poor control indeed.
----------------------------------------------
Now suppose that we arbitrarily add a disturbance to the value of qi and
see how the same approach works.
qi = Kf*o + d
The value of qi that we want, qopt, is
qopt = Kf*o + d
Solving for o, we have
o = (qopt - d)/Kf
Once again, we find that after one iteration we have
qi = Kf*(qopt - d)/Kf + d = qopt. So the disturbance is cancelled after
one iteration.
---------------------------------------
There is only one problem: we must know d in advance in order to compute
the correct o to cancel it, and build that value of d into the equation
of the controller. If d is a function of time, d(t), then we must build
d(t) into the controller and time it so it is synchronized with the
external d(t) acting on qi.
So we might easily come to the conclusion that a control system can't
resist disturbances that are not represented in the equation of the
control system. This seems to be your claim, Hans, and what your
professor from long ago taught you.
This explains a peculiarity in your program, the way the function for
computing the random noise variable is organized. This function returns
a new value of the random noise only on every second call; the
intervening calls return the stored value from the previous call. In a
given iteration of the program, the "normal" function is called once in
computing the external system model, then again in computing the world-
model. This arrangement of the function "normal" assures that the
internal (modeled) disturbance and the external one are identical,
iteration by iteration. This means that they will cancel exactly when
you solve the equations, in the same way that d cancels in the example
just above. Without that provision, the noise in the world-model could
be adjusted to have the same frequency distribution and variance, but
the actual values would not be the same as the noise in the external
system and there would be noise in the model that was not the same as
the noise in the external system.
This, of course, is a serious problem in modeling any real system,
because we have to ask how the adaptive part of the system can know the
details of the external noise, instant by instant. In general, it can't.
------------------------------------------
Now let's ask how the control system might be organized to make qi match
qopt without any knowledge of the external disturbance d.
Let's start with the same "plant", including the disturbance:
(5) qi = Kf*o + d
We compare qi with a reference signal qopt to produce an error signal e:
e = qopt - qi
We then generate the output o in an integrator which sums the error
signal times a scaling factor Ko:
o = o + Ko*e
You notice that we come up with a different form for the controller,
which contains no reference to the disturbance d:
(6) o = o + Ko*(qopt - qi)
Notice that as long as qi is different from qopt, o will continue to
change. The net result is that for any constant value of d, qopt - qi
will become smaller and smaller in an exponential decay. When it reaches
zero, we will have
Kf*qo = qopt - d, and
qi = qopt
So this system compensates for the disturbance without the controller
containing any representation of the disturbance.
The speed with which this compensation occurs depends on 1/(Ko*Kf). The
approach of qi to qopt will follow a curve approximated as
exp(-Ko*Kf*t),
so the larger Ko*Kf is, the faster the approach will be. In the digital
representation, this continuous approximation begins to break down, with
the limiting case being where an error is just corrected in one
iteration. At that point the exponential representation is about 30 to
40 percent off (the digital representation then reaches equilibrium
faster than the exponential one does). Further increases in Ko*Kf do not
result in faster operation in the digital calculation; rather,
computational artifacts begin to appear, eventually leading to a runaway
oscillation as Ko*Kf increases further. This makes sense because an
error can't be corrected in less than one iteration: the digital
controller begins overcorrecting as Ko*Kf increases.
-----------------------------------
OK, this is the way the "canonical" PCT model is usually presented. If
the environmental feedback function is something other than a constant
of proportionality, then we can't necessarily use the same form of the
controller function, because of the system dynamics.
Suppose we use your form of the real system, which is also the
representation of the internal world-model:
x := c + a*x + b*u + d.
While it isn't obvious, this equation, when iterated, amounts to a leaky
integration. If c and d are constants, we have equilibrium when
x = (c + b*u + d)/(1-a)
One integration in a feedback loop is all that is required to stabilize
it (any added integrations, in fact, would tend to destabilize it). So
the controller we use can be a simple proportional controller:
u = G*(xopt - x)
The value of G is limited, in the digital computation, to the magnitude
where the error will be corrected in a single iteration. This controller
form contains no reference to d, yet it will produce variations in the
output u that are just right for counteracting the effects of d (and c)
on x. The total steady-state loop gain is G/(1-a).
However, in order for this control loop to serve as a generator for
values of u that will keep the real system output xt matching xopt, it
is necessary that the model contain a representation of the disturbance,
and that this representation covary exactly with any real disturbance
acting on the real system. The question is now how the adaptive model
appears able to do this, when it contains no representation of a low-
frequency disturbance that influences xt.
------------------------------------------
In the Kalman filter, there are steps where the constants are adjusted
to achieve the adaptation. These steps occur in the routine called
"process_observation".
t := (y - x) / (pxx + pvv);
x := x + pxx * t;
c := c + pcx * t;
a := a + pax * t;
b := b + pbx * t;
When there is no observation noise, the value of pvv, a constant in the
program, is initialized to a very small value, about 1e-6. The Kalman
filter adjusts pxx until it is nearly equal to pvv. As a result,
pxx/(pxx + pvv) becomes nearly 0.5 This means that
x := x + 0.5*(y - x), or 0.5*(x+y);
Without observation noise, y = xt.
So each time that "process_observation" is executed, the value of the
world-model's output, x, becomes the average of x and xt, giving the
real controlled variable the same amount of effect on x as the world-
model has. This step is immediately followed by the steps that pick the
next value of xopt, and compute the value of u -- which depends as much
on xt as on x because of the operation in "process_observation."
So when there is no observation noise, the world-model no longer
determines the value of x; the loop is closed partly through the
external system. This is why the program seems to work like an ordinary
negative feedback control system. It IS operating that way.
------------------------------------------
I have wondered why that step,
x := x + pxx * t;
is necessary, since x, I thought, was supposed to be the output of the
world-model. This step puts in a change of x in addition to whatever
value is calculated by the world-model (in a previous step), so in truth
x is NOT strictly the output of the world-model. Considering that this
step greatly modifies the influence of the world model when there is no
observation noise, I wonder if it belongs there at all.
------------------------------------------
Finally, the form of the controller for the world-model.
The computations with which we began result in a form
u = (xopt - c - a*x)/b
for the controller. This, however, leaves us with a control system that
is less capable than it might be, with the capability depending
critically on the value of a.
The loop gain is computable as du/dx * dx/du. (partial derivatives)
Differentiating the world-model equation with respect to u we get
dx/du = a*dx/du + b, or
dx/du = b/(1-a)
From the controller model we get
du/dx = -a/b.
Thus the loop gain, the product of these two partials, is -a/(1-a) and
is unaffected by b.
When at is set to 0.9 as in the program, and when a = at, the loop gain
is -0.9/(1 - 0.9) or -9. So if somehow the real disturbance is
represented in the world-model, only 0.9 of its effect can be resisted.
When the program is operating in the real-time negative feedback mode,
as it does when there is no observation noise, only the same fraction of
the effect of the real disturbance on xt can be resisted.
This is near the lower limit of what is considered "good control". We
can get closer by changing the controller to a simple proportional
controller:
u = G*(xopt - x).
G can then be increased until the system is at the maximum loop gain
compatible with stability and avoidance of computational artifacts. When
this is done in the program, the Kalman filter adjusts itself so that x
becomes 0.499999*(xt+x), and resistance to disturbance is at least as
good as it is with the more complex way of computing u, and probably
better (I didn't measure it).
---------------------------------------
That is probably more fooling around with Hans' model than anyone wanted
to read.
-----------------------------------------------------------------------
Best,
Bill P.