[Martin Taylor 930415 14:00]
(Bill Powers 930407.2000)
Bill comments on my information-theoretic analysis of the control loop.
I have a problem with the expository methods, apparently. When I try
to say things precisely, they are not understood, but when I try to
use simplifications, I am told that this is not the way things are.
For example:
If the power in the signal is finite, it has an equivalent
rectangular bandwidth, and can be reconstructed.
It can't be reconstructed if you use the equivalent rectangular
bandwidth. You have to use the actual frequency components as far
out as they contribute significantly to the waveform. Otherwise
you will leave out the high-frequency variations that are beyond
the arbitrary limit of the rectangular bandwidth.
Of course. That's obvious. But note the word "equivalent." I use
the general approach of Blackman and Tukey (The measurement of power
spectra, p24 in the 1958 Dover edition) in dealing with the problems
of fractional degrees of freedom.
I did not say that a signal is contained in a finite band. I said that
if it has finite power, it has an equivalent rectangular bandwidth. This
is normal in signal processing applications, which is what Bill likes to
think a control system is, judging by his emphasis on the continuity
of signals. (So do I, for the same reason).
Martin proposes a relationship between the number of distinct
values in the disturbance (D/r) and gain;
Once more unto the quantized breach, dear friends. Let's get it straight.
I do not now, and never have, belonged to the quantized party of America.
I don't plead the Fifth. I don't have friends who are quantized. They
all behave continuously, like good citizens.
Martin proposes a relationship between the number of distinct
values in the disturbance (D/r) and gain; namely, that the
maximum effective gain achievable in a control system is
G = (D/r)^(1-B)
And Bill points out that this leads to absurd conclusions. In mitigation,
I can only quote my comment that went along with this equation in my
930405 16:00 posting:
This is the maximum effective gain that can be achieved in any control
system, if my ever-reliable 
algebra is working.
It obviously wasn't working, and that's where the argument should have hit.
I wish I had someone to check my signs and brackets, etc. When I redid
the algebra, skipping fewer steps, I got
G = (D/r)^(1-1/B) - 1
My error was (for the second time in that posting) in the April 3 formula
P/r = (D/r)^(Bp/Bd), in which the ratio should have been Bd/Bp, as I should
have seen with a moment's reflection.
Now let's see if the absurdity persists. For large D/r, the Gain can be
very large. For large bandwidth ratio, and D/r > 1, the gain can be very
large (i.e. infinite gain at zero frequency disturbance). For a bandwidth
ratio approaching unity, the maximum useful gain approaches zero. All
correct so far (see below for the argument that the last statement is,
contradicting Bill).
The derivation was done on the basis that the
perceptual information per sample = log (P/r)
disturbance information per sample = log (D/r)
This is OK for large SNR, but for small SNR one has to use the continuous
information formula, in which P and D now become the RMS values rather than
the ranges, and r is the RMS uncertainty of the sample after the observation.
And before we get into "the observation is an exact number, whatever it is,"
remember that the uncertainty is that of the CEV given the perceptual signal,
not of the perceptual signal given the perceptual signal. (That's a problem
with the wording in this entire discussion--it's not the perceptual signal
that is represented by P, but the perceptual signal referred to the same
environmental context as the CEV, and as D. A small point, but one worth
remembering occasionally).
The valid formulae are
Perceptual information per sample = log ((P+r)/r)
Disturbance information per sample = log ((D+r)/r)
which makes the algebra more complicated. But I've done it three ways
on paper now, getting the same result each time, so I hope it's right
this time. (But I am, as you must now know, a very sloppy algebraist).
The large-disturbance approximation is, as above,
G <= (D/r)^(1 - 1/B) -1
The accurate (I hope) expression valid for indefinitely small D is
G <= (D/r)/(((D/r)+1)^(1/B)-1) -1
I don't propose to copy the algebra to the net, but if someone can
redo it and finds a different result, I may try. There are too many
fractions, exponents and brackets to make it convenient, but here's
one intermediate line that leads soon to the answer above:
D/((G+1)*r) = ((D/r)+1)^(1/B) - 1
For all values of D/r, when B = 1, G = 0. This is a natural result,
as one can see by simply looking at the action of the control system.
If the PIF has a bandwidth Bp, it has an intrinsic delay on the order
of 1/2Bp. By the time the error that relates to the disturbance at that
moment has resulted in output to counteract it, the disturbance has an
unrelated value (if B = 1, the disturbance has the same bandwidth
as the PIF, and therefore has an indeterminate value after a time 1/2Bp).
Any output generated by the ECS will add in quadrature with the new
value of the disturbance, and no control is possible.
This contradicts Bill's:
There is no
reason that the perceptual bandwidth has to be greater than the
disturbance bandwidth.
The reason is that intrinsic delay, if you look from a straightforward
analogue signal-processing viewpoint. It is the lack of information,
if you look at it either way, but since that's what you don't accept,
I can't use it as part of the argument.
Even if an imaginary sampling process is
inserted, the perceptual function that follows the sampling stage
doesn't have to have a wide enough bandwidth to reproduce the
sampling frequency. The normal practice is to smooth the samples
until only the envelope is visible. Of what use would a 40 KHz
signal be at the terminals of a loudspeaker?
Yes, I did try to make that point, but you said it was unnecessary,
so I dropped it.
There is no necessary relationship between the perceptual
function's bandwidth and that of the _objective_ disturbing
variable.
No, there isn't, unless you are talking about the possibility of control.
You have to determine, in designing the control system, what you want to
be able to control, and if you don't make the perceptual function of
wide enough bandwidth, you can't control.
There is, however, a necessary relationship to the
_effective_ bandwidth of the disturbing variable.
The effective disturbance is measured by its predicted effects on
the perceptual signal in the absence of any other effect. No
matter what the bandwidth of the external disturbing variable,
its effects on the perceptual signal are limited to the bandwidth
of the perceptual function. So the effective disturbance's
bandwidth can be no greater than that of the perceptual function.
True. But this is an S-R approach to the problem, something I have
been trying to avoid all along. This whole argument is based
on the control loop. Without the loop, acting effectively, I have
no argument. For all Rick's fulminations, I'm not talking S-R stuff,
on which your comments depend. The loop is IT.
It should be easy enough to set up a simulation to test the effect of
increasing gain up to and then beyond the limit that seems to be implied
by the information analysis. (I take no responsibility for the correctness
of the algebra; it is the principle that I stand behind.) Remember that
an integrator in the output is another filter, and the analysis assumes
no effects of that kind. I haven't attempted that analysis algebraically,
so I can't guarantee the result. Use a flat spectrum output gain with
zero delay. Use a PIF simulation that has a delay appropriate to the
bandwidth of a causal filter of the same bandwidth. Insert noise at the
input to the PIF, added to the sensory input, to simulate r.
I'm not going to do it, given my time constraints. Allan may choose to,
but I think the results would be more likely to be believed if done by
Bill or Rick. After all, Rick turned our proof that information from the
disturbance does get through the perceptual signal to the output signal
into a series of claims that he had proved it didn't. Just like the
Polish celebration of their great victory over the Mongols in the 14th
century, in which almost the entire Polish and allied nobility was killed
with little loss to the Mongols. Better do it yourselves and convince
yourselves one way or the other.
···
-------------
(Mary Powers 9304.15)
I have a nasty
suspicion that information theory, at bottom, still uses the
telephonic metaphor in talking about the human nervous system. A
message is coded, transmitted, and decoded, and there are all
kinds of theories and calculations having to do with noise in the
system, degradation of the message, etc. BUT...in a control
system, _the signal is never decoded_. It remains a signal, is
processed as a signal, interacts with other signals.
I think the telephonic metaphor (the absolutist, third-party view of
a fixed coder, channel, and decoder) is the reason information theory
has been much misused. Shannon did introduce and develop it in that
context, and much writing about it has been done as if that were its only
context. I have tried to disabuse CSG-L readers of this misapprehension.
Information must be considered in the context of the viewer. In the
analysis of the control system, as in a straightforward analog
engineering analysis, I take what Bob Clark calls the Engineer's
viewpoint, and look only at the physical capabilities of the system.
It is the same viewpoint that Avery took when thinking of the blob
that had variable patches of light on its surface. From the blob's
viewpoint, there was no light, no variability, no nothing. But the
outsider could see the potential for a blob to exist that could make
use of the variability of the light. Evolution could develop a blob
that sees, as the outsider could analyse. Likewise here. The ECS may not
use the available information, but the Engineer can see, from the outside
viewpoint, how much could be there to be used, and therefore can determine
the limits of performance of the ECS under a wide range of conditions.
We have
talked, I think mistakenly, about constructing reality out of
perceptual signals. This implies a little woman in my head who is
decoding signals and extracting information from them. It's hard
to imagine that this reality I see when I look around me is
simply the signals themselves, but what else can it be?
I don't know why this was in the same paragraph as the preceding. It
seems to go off on a completely irrelevant philosophical tangent. My
main complaint against the DME was this kind of recursion. Privately,
I proposed to Bob Clark that we have that discussion when I return in
mid-June, and he agreed. I have no answers to the phenomenon of
consciousness, and I haven't heard any serious claims that PCT has any
such answers, either.
And the neurophysiologists haven't dissected any little men or women
out of cadavers' heads, so far as I have heard, so they must run away
when we die. Logic?
Martin
algebra is working.