Information flow through control systems

[From Bill Powers (930312.2100)]

RE: information theory and control systems.

In my last post I came across an idea (control systems reject
information from disturbances) that has now led to a further
development that may (or may not) help resolve all these 'tis-so-
'taint-so squabbles.

In a very rough way, we can estimate the information capacity of
a channel simply by comparing the input waveform with the output
waveform. If the output bears a close resemblance to the input,
we might guess with some confidence that the information capacity
of the channel is at least as great as the amount of input
information, because apparently little information is being lost.
This is not an exact calculation, but a reasonable first
approximation.

Now apply this to the key relationships in the behavior of a
control system. There are two major channels we can observe or
infer: one from the disturbance that affects the controlled
variable to the action that opposes the disturbance, and one from
the reference signal to the state of the controlled variable.

If we apply a varying disturbance to the controlled variable, we
do not observe that the waveform of the disturbance is reproduced
in the controlled variable. Instead, it is reproduced (inverted,
but faithfully) in the action, the output, of the control system.
As we increase the bandwidth of the disturbing variations, we
observe faithful reproduction of the disturbance waveform in the
output actions, up to some limiting bandwidth. After that
bandwidth is exceeded, we find that high-frequency information in
the disturbance waveform begins to be lost, failing to be
reproduced in the output or action waveform. So we have an
estimate of the information-carrying capacity of that channel.

In the same way, we can apply a varying waveform to the
reference-signal input to the control system. Because this
waveform is transmitted through the comparator to the output of
the system, we would expect, perhaps, that the output waveform
would closely resemble the waveform applied to the reference
input -- but it does not ( in the presence of disturbances, at
least). It is the controlled variable that faithfully follows the
waveform applied to the reference input, whether or not
disturbances are present. By gradually increasing the bandwidth
of the waveform applied at the reference input, we could find the
bandwidth at which the waveform of the controlled variable begins
to differ from that applied to the reference input, and again
estimate the channel capacity.

We can therefore say that information impinging on the control
system from outside, in the form of a disturbance waveform, shows
up almost unattenuated in the waveform of the system's output,
while information impinging on it in the form of a reference
waveform shows up almost unattentuated in the variations of the
controlled quantity -- in both cases, provided that the bandwidth
of the driving waveform is not too great.

Information, therefore, is transmitted through a control system
in two ways: from disturbance to output, and from reference
signal to controlled variable. Within the channel capacities,
information is not destroyed but transformed: the control system
does not "reject" information as I proposed earlier, but routes
it to different places, depending on its origin.

The mechanics of this dual transmission of information are
contained in the inner workings of the control system. The
closed-loop relationships that actually exist are quite different
from the apparent input-output relationships we see in the two
"channels." Any information-theoretic analysis of such a system
must NOT use any simple straight-through causal calculations, but
must be done with the closed loop taken into account.

So far I have not seen any information-theoretic analysis that
takes the closed loop into account. The normal assumptions would
seem not to hold inside the control system, for there is no
simple path for either flow of information to follow. The
information content of a signal inside the closed loop is partly
determined by that signal itself; I have never seen this case
treated. Perhaps someone else has.

The important insight here is that information from both sources
is not lost, but shows up in another system variable.

ยทยทยท

------------------------------------------------------------
Best,

Bill P.