Hans' new demo

[From Bill Powers (950530.1315 MDT)]

Hans Blom (950530)--

I have started looking into your new demo. At first glance it
behaves just as you say. The perceptual signal varies with the
disturbance, but the controlled variable xt does not (nor does
x).

     Frequency plays no role. In the code that follows I present
     a demo where the observation noise term vt contains both a
     low frequency AND a high frequency component (ft = 0). After
     convergence, both are effectively disregarded so that xt
     tracks xopt, whereas y does NOT track xopt.

I need a little help here. I increased the value of the
disturbance of y by modifying a program step as follows:

              >>>
              vvv
    y := xt + 2.0*vt * (sin (run / 50) + sin (run / 155))

The result was that x began to depart noticeably from xt. When I
plotted xt - x, with a range from -1 to 1, the final amplitude of
the difference was about 0.5 units (the maximum total disturbance
amplitude was 2 units).

This led me to look at the final values of the parameters a and
b. In the "true" external system, they are a = 0.9, b := 0.1.
When we progressively increase the multiplier of vt above, we get
the following:

disturbance a b
  0.0*vt 0.899 0.100
  0.5*vt 0.908 0.091
  1.0*vt 0.922 0.076
  2.0*vt 0.939 0.054
  4.0*vt 0.950 0.037

So that made me wonder what would happen if I put in an external
disturbance of xt. The result was that the system behaved as if
it were unstable, and the final difference (x - xt) was wandering
between more than +20 and less than -20 units, after 6000
iterations. Obviously, the requirement that the world-model
compensate for the input disturbance is incompatible with
compensating for a disturbance of the controlled variable -- at
least as I modified the program. I noticed quite a few
computational oscillations, too, during the adaptation.

I was wondering why it is necessary to specify in advance the
amplitude of the disturbances. Does the program need to know them
in advance?

I'm going to have to go back to the negative feedback control
system and see what would actually happen with a perceptual
disturbance. I thought I had it figured out, but I don't
understand the results your program gives.

Can you explain how the Kalman filter works in words of no more
than three syllables?

···

-----------------------------------------------------------------
------
Best,

Bill P.

[From Bill Powers (950530.1130 MDT)]

Hans Blom (950530) --

I have finally realized why the disturbance of the variable y has
so little effect. Cancel all my previous guesses. It is because
the difference x - xt has a very small effect on the world-model.
This effect is seen in the procedure called
"process_observation":

  t := (y - x) / (pxx + pvv);
  x := x + pxx * t;
  c := c + pcx * t;
  b := b + pbx * t;

The magnitudes of pxx/(pxx+pvv) and so on are very small: the
effect of (y-x) on x and c are on the order of 0.0003*(y-x) per
iteration, and on the other variables, a and b, on the order of
1e-5*(y-x) per iteration. During the first 20-50 iterations these
effects are much larger (as large as 0.3 per iteration), so we
see the disturbance showing up in x and xt. But soon the values
of the coefficients become very small, and then both y and
disturbances of y have only very tiny effects except over
thousands of iterations.

The effect of the disturbance of y, and of y itself, is therefore
being averaged over a large number of iterations, thousands. Even
after 6000 iterations, the parameters are still changing
slightly. Therefore the oscillating disturbance of the perception
is simply not perceived by the model-based control system. It is
averaged out. There is no effective disturbance of the
perception.

When a _constant_ disturbance of +1.0 is added, rather than an
oscillating disturbance with an average value of zero, the value
of xt drops negative by about 1.0 units. The value of y tracks
xopt, while xt follows along 1.0 unit lower than xopt and x. So
the perception, rather than the actual output of the real system,
is made to track the reference signal xopt.

So what seemed to me anomalous behavior is, after all, a matter
of the frequency of the disturbance. If you set the disturbance
of y so that

y = xt + 2.0*vt*(sin(run/1900)),

you will see xt gradually falling below xopt as we go through the
first quarter-wave of a sine wave during 6000 iterations. If you
slow the disturbance even more, you will see y approaching the
varying value of xopt and xt approaching xopt - disturbance.

···

a := a + pax * t;
----------------------------------
This makes it clear that "unmodeled dynamics" whether at the
input or the output are unopposed by this version of the adaptive
control system. The best way to think of the Kalman-filter
adaptive system is as a pattern generator which can be gradually
altered to drive an external system in a desired way, open-loop.

The behavior of this system does NOT control perceptions except
at exceedingly low frequencies. That is because on the time-scale
of the reference-signal variations, it is not a negative feedback
control system with respect to the controlled variable, xt. On
that time scale, xt is driven open-loop and the system has no
ability to resist disturbances of xt, or of y, the perception of
xt.

=====

Following the above investigations, I went back to a copy of your
original program, and tried adding a disturbance to y. I found
exactly the effect I had predicted: y was controlled, and xt
showed the effect of the disturbance while x did not. The
original program controls perceptions when the disturbance is
simply added to y without changing anything else in the program.

When I checked the value of pxx/(pxx+pvv) it came out 0.99,
instead of 0.0003 as in the "pseudo" demo. The reason was that
the observation noise was 1e-6 instead of 0.5. When I changed the
observation noise to 1.0 (using a disturbance with an amplitude
of 1.0), I was back to the same conditions (I think) as in the
new demo.

I have noticed something in your "normal" routine. What it does
is to give the same value of random noise twice in a row, then
change to the new value twice in a row, and so on. This means
that the noise disturbance applied to either xt or to y is
identical to the noise disturbance assumed in the world-model,
not just in variance and magnitude, but point for point.

Can you explain why it is necessary to tell the model what
magnitude of disturbance to expect? This is clearly an operation
from outside the model, information about the world that it could
not get without outside help. pvv is not calculated anywhere in
the program that I can see: it is initialized (as are several
other critical numbers). And can you explain why it was necessary
to make the "normal" noise in the world-model identical to the
same noise in the real system? You clearly went to some trouble
to assure that the same values would be used in both places.
There must have been a reason for doing this, even though it
makes no sense that the world-model would contain this kind of
information about noise in the external world.

-----------------------------------------------------------------
------
Best,

Bill P.