[From Bill Powers (950625.0810 MDT)]
RE: Hans' model.
Despite the hiatus in the discussion with Hans Blom, I've been trying to
figure out what strikes me as so strange about his method of setting up
a control system equation. Hans took the equation by which the
controlled variable depends on the output, and solved it for the output
that would correct the error on the next iteration. Not only did this
method correct the error in one iteration, it corrected the error
_perfectly_, leaving no error at all from then on. PCTers who followed
the details of this discussion must have been wondering why OUR control
system models creep toward zero error and never actually get there, when
there seems to be a closed-loop model that could correct error instantly
and perfectly.
Hans starts with an equation purporting to be a model of a physical
system:
x[t+1] = a*x[t] + b*u[t] + disturbance terms.
We can drop the disturbance terms for now, so we just have
(1) x[t+1] = a*x[t] + b*u[t].
u is the driving function at the input to the world-model, x is the
output variable of the world-model.
Hans then sets up the condition
(2) x[t+1] = xopt, and solves (1) and (2) together:
xopt = a*x[t] + b*u[t].
From this we can compute the form of the adaptive model's output
function that produces u:
u[t] = (xopt - a*x[t])/b
Substituting this expression for u[t] into (1), we get
x[t+1] = xopt. The error is corrected on the next iteration!. In fact,
there is never any difference between x and xopt.
···
-------------------------------------------
Despite his vehement denials, Hans followed a very simple procedure
involving taking the inverse of the world-model:
x = f(u), from which we compute
u = (inverse f)(xopt)
Obviously, then, x = xopt for any form of the function f.
What he did was simply to compute that value of u that would make x be
exactly equal to the value of xopt at all times. So his world-model
"loop" can be represented as
xopt
>
[u = (inverse f)(xopt)]
>
v
f(u)
>
x = xopt
The world-model's output variable x is always identically equal to xopt,
regardless of the values of the parameters in the world model.
Thus when y, the perception of the true controlled variable xt, is
compared against the output of the model x, it is really always being
compared against xopt, because x is always equal to xopt regardless of
the operation of the Kalman filter. If we say that y is a true
representation of xt, then the actual input to the Kalman Filter
operation is (xopt - xt) -- the error signal!
The Kalman filter operations vary the values of the world-model
parameters until y = x, which means until xt = xopt. So in a control-
system model format we have
xopt
>------>------
> > e, error
-------->[comparator] ---o----->---------
> > > >
y v v Kalman
> (inv f)(e+xopt)<--- filter
[input func] | \
> u \
> > output function
> >
xt <----[f(u)]------------
Note that the world-model itself is a smokescreen, not being shown in
the above diagram. In the loop that runs from x back to x through the
world-model in Hans' diagram, the net effect is always to make x = xopt,
independently of what the parameters are. So the Kalman filter
operations have no effect on the world-model loop or on the value of the
output of the world model, which is always xopt. The world model itself
plays no part in the adaptation.
The real effects of the Kalman filter operations take place in the
_inverse_ of the world-model which is placed in the output part of the
system. By varying the parameters in the inverse function, the adaptive
filter causes xt to approach xopt; indeed, that is the error that keeps
the adaptation going. The Kalman filter makes cumulative corrections, so
the effect is that of a control system with an integral output function.
After enough time has passed (and of course in the absence of
disturbances), the error signal will become exactly zero.
---------------------------------------------
In the above diagram I have shown the reference signal entering both the
comparator and the output function. If the comparator loop were missing,
and the output function's parameters were adjusted so the output
function was the exact inverse of the real function f, then variations
of the reference signal alone would cause xt to follow xopt, open-loop.
In that case, reconnecting the comparator loop would make no difference
because the error signal would always be zero.
However, if an independent disturbance were to act on xt, the error
signal would become non-zero and add to the effects of xopt in the
output function. This would produce resistance to the effects of the
disturbance even though disturbances are not taken into account in the
output function. What this arrangement does is to set up a basic open-
loop connection between the reference signal and the controlled
variable, with a closed-loop effect that comes into play only when there
are disturbances, or when the output function is not the exact inverse
of the real environmental feedback function -- the two main conditions
that produce a nonzero error signal.
When such errors occur, there are two results. The first result is that
the effect of the error signal shows up as a variation in output opposed
to the effect of the disturbance, just as if the reference signal had
changed. The second result is that the Kalman filter operations receive
a nonzero input, and begin slowly altering the parameters of the output
function. If the Kalman filter is smart enough, it might even be able to
add regular disturbances to the output function that would oppose the
effects of regular external disturbances, restoring the error signal to
zero even in the presence of regular disturbances and permitting the
equivalent of open-loop operation. How well this would work would depend
on how accurately the external disturbance can be matched by an internal
generator, including synchronization. I would expect some rather severe
limits on the success of such attempts to anticipate disturbances other
than constant ones.
In some earlier explorations of arrangements like that in the above
diagram, I think I showed that the dual function of the reference signal
can be achieved with a normal comparator and a single reference signal,
just by assigning different weights at the two inputs of the comparator.
I would very much like someone else to verify this. If this is true,
then I think we would have something much closer than before to a merger
of the ordinary PCT control system and the adaptive control system
proposed by Hans.
-----------------------------------
The above diagram is in fact completely equivalent to Hans' model, even
though it doesn't seem to be because of the missing "forward" world-
model. I have shown that the forward world model is actually a dummy
operation having no effects on adaptation. Its only function is to
assure that x is always identically equal to xopt. What does have an
effect is the change in parameters in the _inverse_ of the world model
that is in the output function of the control system. In fact, the above
diagram is exactly the "alternative" use of the Kalman filter which I
had previously proposed to Hans, before realizing that it is
functionally identical to his model. The identity is not exactly self-
evident, so perhaps both Hans and I can be forgiven for not seeing it
immediately.
-------------------------------------------
Since the Kalman filter is only one possible method of reorganization, I
propose that the general adaptive control model substitute the label
"reorganization" for "Kalman filter" in the above diagram. The result is
a very nice package (if it really works), amenable to expansion into
multiple control systems and a hierarchy of systems with each system
containing this ability to adjust itself to minimize control errors.
There is even the ability to run for a while open-loop, when the source
of error signals is lost. We have considered models of this form before,
but never realized that they could also take care of limited open-loop
operations.
All that remains is to solve the problem of handling loss of input
signals. As several people have noted, a signal with a value of zero is
not conceptually the same as lack of any signal. But I am mostly
convinced that in real circuits, the two conditions must be treated as
the same. In other words, cutting off sensory inputs results in
perceptual signals having a value of zero. The receiving circuits are
still there, and the significance of a perceptual signal of zero
frequency remains what it was before.
This means that to make the above diagram work right, we need some way
to detect the condition of zero input signal and use it to cut off the
effect of the error signal. Hans' model did that just by letting the
user of the program do it; the user could make error signals ineffective
through manual setting of one system variable to a value that switched
off the error signal's effects. There may be other more automatic ways,
especially if we think in terms of pairs of balanced one-way comparators
(the norm in the nervous system). The absence of _both_ inputs to a
push-pull comparator would be an error signal of zero, which is what we
need.
-----------------------------------------
I hope there are at least one or two listeners out there who have been
following the details of this discussion. I think this analysis is a
major discovery concerning the model Hans presented; I don't know how
general its implications would be for Extended Kalman Filters, but in
this one application we can see that the real organization is very
different from the apparent one. If this is a general truth, it would
obviously have extended implications. Some independent checking of my
reasoning would be welcome.
-----------------------------------------------------------------------
Best,
Bill P.