adaptive/predictive model-based control

[From Bill Powers (950503.1130 MDT)]

Hans Blom (950503) --

Got your program and it runs fine. It will take me some study to
understand how it all works, but I've tried a few things so far and come
up with one major problem. This control system doesn't seem to be able
to resist disturbances of its controlled variable, xt.

In the procedure step_system, I used an external disturbance that just
adds a sine wave to the value of xt, as shown below with the original
disturbance term commented out:

procedure step_system; {execute one step of system}
{instead of normal (ft) try also ft * sin (run/5)}
begin
  xt := ct + at * xt + bt * u + 2.0*sin(run/5); {normal(ft);}
end;

This seems to frustrate the system completely. The value of xt shows the
sine-wave disturbance undiminished. From my very brief examination of
the code, it seems that the disturbance you are using, normal(ft), is
both very small and very rapidly changing, so even though it does appear
undiminished in the controlled variable, it is not noticeable. When a
more normal disturbance is used, there appears to be no resistance to it
at all.

This would seem to be expected from the organization of the controlling
system. The internal model it is using is built up from experience with
the behavior of the controlled variable (as perceived), with certain
parameters being adjusted for the best average performance. That's all
very well, but this approach would make it impossible for the control
system to protect the controlled variable -- the real one -- from
arbitrary disturbances. The arbitrary disturbances alter the real
"controlled" variable but not the internal model; hence the system as a
whole is unable to cope with the disturbances.

The most important advantage of your approach is that it permits control
to continue for brief periods of loss of the input information, when the
external variable is not being disturbed. It is also quite good at
adjusting the model parameters to cope with changes in the external
system parameters. So I would say that you are definitely on the track
of some useful adaptive features of the control system. However, these
advantages come at the expense of losing one of the most important
properties of a good control system, the ability to maintain the
controlled variable at any specified reference setting in the presence
of unpredictable external disturbances.

I had suspected that the concept of a disturbance was treated
differently from the PCT way in the readings about adaptive control that
I have done; your model, plus some remarks you have made, confirms that
suspicion. You are thinking of disturbances as small added amounts of
zero-centered noise, not as large unsystematic perturbations. The amount
of noise you have put into the system is so small as to be negligible --
when I multiplied normal(ft) by 2000, it made no visible difference in
the behavior of xt. However, my addition of a disturbance with an
amplitude of 2 units completely disrupted the system, and there was no
sign of resistance to the disturbance.

The adaptive processes you use (which I will study) are probably worth
learning about. But the net result is not what I think of as a control
system, because it can't work in a real environment where significant
disturbances are always present.

···

----------------------------------------------------------------------
Best,

Bill P.

[Hans Blom, 950504]

(Bill Powers (950503.1130 MDT))

Got your program and it runs fine. It will take me some study to
understand how it all works, but I've tried a few things so far and come
up with one major problem. This control system doesn't seem to be able
to resist disturbances of its controlled variable, xt.

Just a short reply for now: an adaptive control system cannot resist two
types of disturbances: 1) completely unpredictable ones such as high-
frequency noise (ft * normal in the demo) -- these CANNOT be modelled -- and
2) disturbances that ARE not modelled (your ft * sin), for whatever
reason -- because the model is not sufficiently complex. Note that in the
model-based control literature, the preferred jargon is not "disturbance"
but "unmodeled dynamics". This has deep philosophical implications...

In the procedure step_system, I used an external disturbance that just
adds a sine wave to the value of xt ... This seems to frustrate the system
completely. The value of xt shows the sine-wave disturbance undiminished.

This is a beautiful example, which I had intended to add as one of the
"experiments", but forgot about. If the sine wave cannot be modelled -- in
my simple demo it is not, although in a more complex one it could -- it
cannot be resisted. This is not a bug, it is a feature! Think about it: if
things happen in the world that you cannot (mentally or intuitively) grasp,
you will be blind to those things and unable to control them. Do not blame
the demo for something that it cannot do; attempt to see it as a demonstrat-
ion of a real-world phenomenon. Yes, I know, this takes some getting used
to ...

                             The arbitrary disturbances alter the real
"controlled" variable but not the internal model; hence the system as a
whole is unable to cope with the disturbances.

Yes, things like that happen. If disturbances are truly arbitrary, they
cannot be coped with by ANY control system. That seems to be reality. If
they are not arbitrary but law-like/predictable, a more complex model will
do the job.

With the sine wave disturbance, I suggest the following experiment: MAKE the
model richer. Assume that, somehow, a much richer model could accurately
predict the sine wave. In my simple demo you can mimick that by adding the
same sine wave expression to the prediction of x. Yes, this is a sly trick,
I know. But the result is realistic if for the moment you disregard that
the sine wave model (three parameters: amplitude, frequency, phase) has to
be learned as well, and that it takes some time for an accurate sine wave
term to be modeled as part of the prediction of x.

The most important advantage of your approach is that it permits control
to continue for brief periods of loss of the input information, when the
external variable is not being disturbed.

Yes, that is one thing that I wanted to demonstrate. That was what the
"feedforward" discussion was all about.

                                         It is also quite good at
adjusting the model parameters to cope with changes in the external
system parameters.

The reason for this, I think, is the fact that "knowledge" (model parameter
values) is modelled as probability distributions, a very different approach
from your models. Martin: this is also where the link is with information
theory.

                So I would say that you are definitely on the track
of some useful adaptive features of the control system.

Actually, this program was a Pascal rewrite from FORTRAN sources that I
composed around 1972. Extended Kalman filters are even older than that. It
is just the display part that is so much easier (and prettier) now...

                                                    However, these
advantages come at the expense of losing one of the most important
properties of a good control system, the ability to maintain the
controlled variable at any specified reference setting in the presence
of unpredictable external disturbances.

That is where our interpretations diverge. Completely UNPREDICTABLE external
disturbances (a sudden flood or earthquake, a sudden death of a beloved one)
cannot be/are not modelled and hence are not defended against. We are only
as good as our models. However, PREDICTABLE external disturbances CAN be
modelled and hence taken into account when controlling: expand the model. It
would be relatively simple to add a model component for a sinewave disturb-
ance to the demo.

The adaptive processes you use (which I will study) are probably worth
learning about. But the net result is not what I think of as a control
system, because it can't work in a real environment where significant
disturbances are always present.

My interpretation is this: if in a real environment significant disturbances
are present that either cannot be modelled or are -- for whatever reason --
not modelled, they cannot be defended against.

For the moment, stick with this simple example and see what it can and
cannot do. Let's keep out of a discussion where you say "but your system
cannot do X" and I say "if you add Y to the model, it can". Such a dis-
cussion would be endless and less than fruitful. It is the CONCEPT of model-
based control that I wanted to demonstrate. And that is done best with a
very simple model and a very simple world, I think.

Greetings,

Hans

<[Bill Leach 950507.23:41 U.S. Eastern Time Zone]

[Hans Blom, 950504]

Yes, things like that happen. If disturbances are truly arbitrary, they
cannot be coped with by ANY control system. That seems to be reality.

Some of the rest of your posting tends to cause me to believe that you do
not mean this the way that it sounds to me. Control systems resist ALL
disturbances that change perception to the limits of system output and it
matters not if the disturbances are random, arbitrary or regular in any
pattern (such can certainly influence the success of control but not
whether the system will attempt to resist).

Am I 'missing' your point?

-bill