[From Bill Powers (930403.2100 MST)]
Martin Taylor (930403.1540) --
Bill originally pointed out that the information in a single
sample of the perceptual signal was blog P/r (I use blog for
log base 2, but for the rest of this posting I shall just use
"log", where you can take any base you want). The information
in a single sample of the disturbance was log D/r, at least
that is the most information that the PIF can provide. Now if
there is control, P > D, so clearly a single sample of the
perceptual signal cannot convey all the information in a single
sample of the disturbance.
I think you meant P < D.
If the perceptual signal's bandwidth is less than that of the
actual disturbing variable, all information in the disturbing
variable above that bandwidth is simply lost. The high-frequency
components in the disturbance can't be reconstructed at all from
the perceptual signal or any function of it -- they simply aren't
there. This is true whether there's any feedback control or not.
So the effective bandwidth of the disturbance is always equal to
or less than that of the perceptual signal: Bd <= Bp.
ยทยทยท
-----------------------------------------
A long time ago, it was proved that a continuous waveform can
be EXACTLY reconstructed from a finite set of samples, if the
waveform has finite bandwidth.
This is true, but it's an artificial case. No natural systems
have rectangular distributions; the frequency response simply
gets smaller and smaller with higher frequencies, eventually
becoming of no practical importance at some very high frequency.
Also, the manner of the dropoff makes a lot of difference. It's
possible to have a tail-off form such that the integral of all
contributions from any frequency to infinity is still finite: the
Fourier series doesn't necessarily converge, or if it does, the
contribution from "invisible" high frequencies is not negligible.
The Fourier series is usually cut off arbitrarily at some
harmonic, without much proof that there's no point in keeping 100
more harmonics. When a natural waveform isn't actually created by
a finite set of harmonically-related oscillators, one has to take
great care in applying a Fourier analysis. What looks like a
reasonable cutoff point may introduce large errors in the inverse
transform.
The result is that to sample a natural waveform sufficiently
often to reproduce it "perfectly," one must raise the sampling
frequency far above the nominal "cutoff" point. We have found
that to represent human tracking behavior well enough to model it
within, say, 5%, we must sample at least 20 times per second,
even though the nominal control-system bandwidth is only 2.5 Hz.
That measure of bandwidth is simply a traditional measure, the
point at which the output drops to sqrt(1/2) of the low-frequency
response. This means that the amplitude of the response at this
so-called cutoff point of 2.5 Hz is still 70 percent of the zero-
frequency amplitude. At 5 Hz is it 35%, and at 10 Hz it is 17.5%
To reduce the remaining components to under 5% of the zero-
frequency amplitude, you have to go clear out to 40 Hz or more.
This is why I was glad to get a super-VGA graphics card; I can
now sample at 86.8 Hz, at which the remaining high frequencies
will probably be truly negligible. I hope.
-------------------------------------------------------
In the situation at hand, we have
perceptual information per sample = log (P/r)
disturbance information per sample = log (D/r)
control ratio = D/P
I'm not sure what you mean by the control ratio or how it is
related to the discussion of bandwidth that just preceded it.
I'll assume you mean the ratio of the observed ranges of the
disturbance and the perceptual signal. As you have put it, it
will be much greater than 1 when control is good. In fact, D/P
will be approximately the loop gain of the system, for
frequencies within the system bandwidth.
Now, the perceptual signal can convey the information available
in the disturbance only if the bandwidth of the disturbance is
less than that of the PIF, in otherwords, several samples of
the perceptual signal are used to convey the information from
each sample of the disturbance signal.
Bp log (P/r) >= Bd log (D/r)
It took me a while to realize that this isn't a statement of
fact, but a statement of the condition required for the
perceptual signal to carry all the information in the
disturbance. As I understand it, both sides express the MAXIMUM
information rate that can be present. Under these conditions,
you're saying, the maximum ACTUAL information rate about the
disturbance will be equal to Bd log(D/r).
In my derivation of 930331, I said
So the information in the input quantity is -blog(R'/r), which
is equivalent to -blog(R/[(1+G)*r]).
In your terms, this would be -log(D/(r*(1+G))). Compare that
with your
Bp log (P/r) >= Bd log (D/r)
Substituting, this says that the condition is (ignoring the
negative sign)
Bp log(D/(r*(1+G))) >= Bd log (D/r).
The effective bandwidth of the perceptual signal then is equal to
that of the disturbance (Bp = Bd), so we simply have the
expression I derived. It's true that
blog(R/r*(1+G)))
--------------------
blog(R/r)
is necessarily less than 1; you can delete the equal sign in your
expression for all loop gains greater than zero.
So: I showed initially that the bandwidth of the effective
disturbance must always be equal to or less than the bandwidth of
the perceptual signal. I had previously shown how to calculate
the information in the perceptual signal per sample, which means
the relative information rates assuming that the bandwidths are
the same. If the actual bandwidth of the disturbance is less than
the maximum bandwidth of the perceptual signal, the information
rate in the perceptual signal without feedback is the same as
that in the disturbance. With the feedback, the information rate
in the perceptual signal is therefore the information rate in the
disturbance minus log(1+G).
We find, therefore, that a control system subtracts blog(1+G)
bits per second from the information entering via the effective
disturbance, over the bandwidth of the effective disturbance. I
believe that this forms the bridge between purely analogue
computations of the behavior of control systems and computations
in terms of the variables of interest to information theorists.
Note the one contribution unique to control theory, which comes
out of the closed-loop nature of the system: the effect of the
loop gain.
As a matter of curiousity, suppose that we have a control system
and a disturbance with a range of 1024 discriminable levels. We
want to reduce the effect on the perceptual signal to 1/64 of
that amount, or 16 discriminable levels. This means that the
incoming information contains 10 bits per sample, and the
remaining perceptual information is to contain 4 bits per sample.
We therefore want to lose, or "block", 6 bits per sample. This
implies that
blog(1+G) = 6, or G = 63.
The effect of a disturbance with magnitude D on a control system
is reduced to D/(1+G). In this case, the magnitude of D is 1024
units, so the remaining effect would be 1024/(1+63) or 16 units.
We therefore get the same numerical answer for the relationship
between loop gain and the range of the perceptual signal whether
we use the simple analogue computations or the information-
theoretic computations. The reason, as far as I can see, is that
the only difference is in doing the computations in logarithms or
linear quantities. In either case, we have to use the fact,
derivable only from control theory, that the effect of the
disturbance is reduced by a divisor of (1+G). Given that fact, we
can either compute the amplitude variations in the perceptual
signal as a function of the amplitude variations in the
disturbance, or compute the information in the perceptual signal
as a function of the information in the disturbance. There
doesn't seem to be any essential difference beyond the use of
logarithms.
------------------------------------------------------------
Best,
Bill P.