[From Bill Powers (930416.0800 MDT)]
Martin Taylor (930415.1400) --
Otherwise you will leave out the high-frequency variations that
are beyond the arbitrary limit of the rectangular bandwidth.
Of course. That's obvious. But note the word "equivalent." I
use the general approach of Blackman and Tukey (The measurement
of power spectra, p24 in the 1958 Dover edition) in dealing with
the problems of fractional degrees of freedom.
What does that have to do with reconstructing a waveform? A power
spectrum is of as little use as a rectangular distribution for
reconstructing a waveform. What you need are the specific
amplitude and phase values for every frequency out to the limit
where there is enough amplitude left to make a difference in the
waveform. A power spectrum throws away sign information and phase
information, both of which are essential for the reconstruction.
I did not say that a signal is contained in a finite band. I
said that if it has finite power, it has an equivalent
rectangular bandwidth.
That is true, but irrelevant to reconstructing a waveform. The
"equivalent" bandwidth is equivalent only in those respects you
preserve in constructing it -- which does not include amplitude,
phase, or frequency. The equivalent rectangular bandwidth is an
artifact constructed for convenience in doing certain kinds of
calculations. It is not equivalent to the actual situation in any
of the ways that would be helpful in reconstructing a waveform.
Once more unto the quantized breach, dear friends. Let's get it
straight. I do not now, and never have, belonged to the
quantized party of America. I don't plead the Fifth. I don't
have friends who are quantized. They all behave continuously,
like good citizens.
Then why do you keep talking about D/r? Why not just let r go to
the limit of zero (the continuous case) and the uncertainty to go
infinity (log(D/r), and use those continuous equations? It seems
to me that all the basic definitions require that D/r be a finite
whole number, and that the boundary between d and d+r be fixed,
with no values of d between d and d+r. To me that is quantization
-- the division of a continuous scale into fixed intervals.
Bill points out that this [derivation of a relationship between
bandwidth and D/r) leads to absurd conclusions. In mitigation,
I can only quote my comment that went along with this equation
in my 930405 16:00 posting:
This is the maximum effective gain that can be achieved in any
control system, if my ever-reliablealgebra is working.
algebra is working.It obviously wasn't working, and that's where the argument
should have hit.
And what about your patronizing exasperation about my not paying
attention to the relationships you had laid out and that I had
not duly memorized?
The new equation is
G = (D/r)^(1-1/B) - 1
And you say about it
For a bandwidth ratio approaching unity, the maximum useful gain
approaches zero. All correct so far ---
That is to say, if the bandwidth of the disturbance is 1 hz, and
the bandwidth of the perceptual function is 1 Hz, the maximum
usable loop gain in the associated control system is zero. This
is simply not true. We will shortly have a new version of Simcon
that allows creating a random disturbance with a one, two or
three stage filter to permit varying the bandwidth and the
frequency rolloff. So you can create a disturbance with a
bandwidth of 1 Hz. Using an amplifier with a time constant of 1
sec (or whatever the right number would be) you can give the
control system an input bandwidth of 1 Hz. I suggest that before
you carry this line of deduction any further, you set up a
control system and test your "all correct so far" conclusion. The
mathematics may be all correct so far, but the conclusion is
still false. There is obviously a problem with the premises.
You will find that the control system can be adjusted to keep the
effects of the disturbance on the perceptual signal very small,
with a loop gain very much larger than 0. If you don't want to do
it, I will do it for you.
The bandwidth of the disturbance has absolutely nothing to do
with the maximum usable loop gain in a control system. There is
something wrong with your premises, which renders all the
manipulations that follow from them spurious.
... for small SNR one has to use the continuous information
formula, in which P and D now become the RMS values rather than
the ranges, and r is the RMS uncertainty of the sample after the
observation. And before we get into "the observation is an exact
number, whatever it is," remember that the uncertainty is that
of the CEV given the perceptual signal, not of the perceptual
signal given the perceptual signal.
In passing, I'm curious why there is such rejection on the IT
side of concepts like variance and correlation, which involve
exactly the same kinds of derived measures: "RMS" variations,
which is just the calculation of sigma. Correlation is simply
sigma(xy)/[sigma(x)*sigma(y)]. These measures are obtained from
discrete samples, and I'm sure that by rearranging the
expressions through equivalence transformations and zinging in a
logarithm here and there, you could show that the expressions are
closely related to those by which uncertainty is calculated
(conditional uncertainty in the case of sigma(xy)). The basic
premises are essentially identical.
As to "the uncertainty is that of the CEV given the perceptual
signal", that uncertainty has to be zero, because the CEV is
defined as the external correlate of the perceptual signal. You
are arguing from the premise that the CEV has an objective
existence independent of the control system, so that the
perceptual signal will represent it more or less accurately. Thus
the CEV could be in one state, while the perceptual signal
represents it as being in a somewhat different state. But that
contradicts the definition of a CEV.
The perceptual signal is the only true measure of the CEV. The
bandwidth of the perceptual signal IS the bandwidth of the CEV.
The valid formulae are
Perceptual information per sample = log ((P+r)/r)
Disturbance information per sample = log ((D+r)/r)
This can't be the correct set of formulae. The implication is
that the perceptual information per sample is independent of the
(imaginary) sampling frequency. When you let the sampling
frequency go to infinity to approach the continuous case, the
information per sample must decrease accordingly, or else the
information rate will go to infinity. It will go to infinity-
squared, because r also approaches zero.
This is very much like the problem of representing a continuous
control system on a digital computer. If you just naively compute
your way around the loop, you will end up with oscillations at
the frequency of iteration for any loop gain greater than 1. To
make the computations independent of the iteration frequency, and
get the right behavior, you must put in a slowing factor that
introduces physical time. As the iteration rate increases, the
amount of change permitted per iteration decreases, so the system
converges to the correct behavior as iteration frequency
increases without limit.
Your formulae above do not take physical time into account, which
is why they lead to an absurd result as the sampling frequency is
raised. The information per sample must decrease as sampling
frequency is increased. But there is no variable in the formulae
representing sampling frequency as samples per unit of physical
time. Without an explicit variable linking your discrete
calculations to physical time, you can't make the transition from
the discrete representation to the continuous (physical)
representation -- correctly.
There is no
reason that the perceptual bandwidth has to be greater than the
disturbance bandwidth.
The reason is that intrinsic delay, if you look from a
straightforward analogue signal-processing viewpoint.
I think you're confusing "disturbance bandwidth" with "the
highest frequency component of the disturbance." And note that a
pure delay doesn't change the bandwidth at all. Only an integral
lag (or some such) will cut off high frequencies. Integral lags,
by the way, introduce physical time.
And I think you're still equating "disturbance" to "change in the
CEV." You've never understood why I make the disturbance, or the
disturbing influence, independent of the state of the CEV. The
reason is precisely to avoid the sort of confusion we have here.
If the CEV is the position of a limb, the disturbance is not a
change in that position, but (for example) the magnitude of a
force applied to the limb. There is no necessary relationship
between the applied force (or its bandwidth) and the resulting
changes in the CEV, because there is at least one other force
being applied -- the output of the system controlling limb
position. And remember that the measure of the CEV is the
perceptual signal, by definition: the input function doesn't even
come into it. To deduce the hypothetical state of an external
equivalent of the CEV you would have to apply the inverse of the
perceptual function to the perceptual signal. You posted a
diagram yesterday that makes exactly that point.
The normal practice is to smooth the samples
until only the envelope is visible. Of what use would a 40 KHz
signal be at the terminals of a loudspeaker?
Yes, I did try to make that point, but you said it was
unnecessary, so I dropped it.
No, you have dropped it because you're now saying that the
bandwidth of the perceptual function has to be greater than that
of the disturbance. If the samples are smoothed, the bandwidth of
the perceptual function can be LESS than that of the disturbance.
I brought it up as an argument against the idea that B > 1 (as
then defined). If you now claim that point, it contradicts your
assertion that the perceptual bandwidth is greater than the
disturbance bandwidth (B > 1 as now defined).
If I said it was unneccessary, it was because the sampling is
imaginary in the first place.
But this is an S-R approach to the problem, something I have
been trying to avoid all along.
You can't avoid the S-R approach when you're talking about a
single function in a control system. All the functions are S-R
devices. In speaking of the effect of the disturbance on the
perceptual signal, it's perfectly legitimate to talk of its
contribution in the absence of output (which can be observed at
every zero-crossing of the output effect).
It should be easy enough to set up a simulation to test the
effect of increasing gain up to and then beyond the limit that
seems to be implied by the information analysis. (I take no
responsibility for the correctness of the algebra; it is the
principle that I stand behind.)
I agree about the simulation test, but come on! If you don't take
responsibility for the correctness of your algebra -- on which
all of your arguments depend -- who will? You've already gone off
on a number of deductive tangents as a result of making errors.
How can you convince anyone that your "principles" aren't equally
tainted by wrong deductions, when your defense of them depends
entirely on the mathematical manipulations you apply to your
premises, without any guarantee that your manipulations are free
of error? Or is your faith in IT so unshakeable that no mere
mathematical proof or disproof (or simulation) could disturb it?
As soon as we begin to treat the disturbance as a systematic
message instead of noise, we are out of the realm of
information theory.
NO.
YES. Why treat a continuous and regular phenomenon as if it is a
random variable? Why not use the simple and direct reasoning
appropriate to continuous variables when that is the kind of
phenomenon we are dealing with? Why not reserve the much more
complex and opinion-weighted and assumption-sensitive arguments
of uncertainty theory for situations in which the variables are
in fact unpredictable and irregular on the scale of interest?
Nope. The uncertainty in the perceptual signal is determined by
the resolution of the perceptual apparatus.
So it's premise against premise. I like my premise better.
The resolution of the perceptual apparatus is infinite. For any
range of the perceptual signal above the dead zone near zero and
below the saturation level near maximum, the frequency of the
neural signal can change by any arbitrarily small amount. It does
not jump from one finite frequency to another. Any change in the
inputs whatsoever is reflected as a corresponding change in the
frequency of the perceptual signal. There is no threshold amount
of input change required to produce a change in the perceptual
signal. There may be a least amount of change required for a
person to judge that a change has occurred (a much more complex
and higher-level process), but even these JNDs are not associated
with fixed intervals on the perceptual scale.
Thus the uncertainty in the perceptual signal is set by channel
noise, not by the resolution of the perceptual apparatus.
Probabilistic calculations do not apply to the whole
effect of the disturbance, but only to the slight deviations of
the perceptual signal from being a perfect representation of
the disturbance.
They apply anywhere, but some applications are more useful than
others. It turns out to be useful for an Engineer/Designer to do
the calculations for the unopposed disturbance.
You miss my point. If the disturbance is a pure sine wave, it
contains no uncertainty at all, although one can treat the sine
wave using the manipulations appropriate to random variables and
get numbers out of the manipulations. The numbers, however, are
meaningless, because there is in fact no uncertainty.
The only uncertainty that exists in the disturbance is the
_unpredictable_ component, which can range from almost none to
the entire signal. The mathematics of uncertainty should be
applied ONLY to the unpredictable component. This has nothing to
do with whether the disturbance is opposed or unopposed.
If the waveform of the perceptual signal is a perfect replica of
the disturbance, or if it can be expressed as a regular noiseless
function of the disturbance (as in a control-system model), then
there is no uncertainty involved. Uncertainty would enter only
when the function used to express the relation between the
perceptual signal and the disturbance fails to predict the
observed relationship; then the mismatch can be called an
uncertainty, and appropriately treated in terms of random
variables.
When the loop is closed, the amplitude of the perceptual signal
variations due to the disturbance is greatly reduced. But the
remaining amplitude is still much greater than the channel
noise until it becomes only a few percent of the amplitude of
the opposing signals.
This is a bone of contention, I think.
Not if we're talking about simulations. If a noise-free
simulation (to which the above remarks certainly apply) fits
behavior within a few percent, then my statement is probably true
of the real system as well.
There are two fundamental reasons why control is not perfect.
One is in the control system dynamics; there may be insufficient
low-frequency gain to oppose a persistent disturbing influence,
or the disturbance may be continually increasing, or the
available output power may be insufficient for the disturbance,
or some such. The other is the perceptual inability to
determine what needs to be corrected, and in an effective
control system, this is the major limit, in my view.
It is not the major limit in our simulations. It is hardly a
consideration at all. And in the behavior that is matched by our
noise-free models, it can't be the major factor. The variations
due to inability to determine what needs to be corrected, in our
tracking experiments, are exactly known: one pixel of change,
which is easily visible. The computed value of the reference
signal in these experiments, with slow disturbances, is usually a
fraction of a pixel away from zero. The actual range of the
variables is from 30 to 100 pixels, and the potential range (set
by VGA screen limits on my machine) is 480 pixels. The RMS
uncertainty between the model and the actual behavior (the
measure you mention above as appropriate for the continuous case)
amounts to 5% or less of the range of the variables.
If we lower the disturbance frequency enough, the person can keep
the cursor on the target with errors of only 1 or 2 pixels. So
the limit set by perceptual resolution (here enforced by the
screen resolution) is negligible with respect to the limits set
by dynamic considerations.
The IT argument is a structural one. Why are control systems as
they are? IT does not deal with specific dynamics of particular
control systems, though the analyses might well be applicable.
It deals with HOW they work and why the fundamental laws of
control systems function in a real, partially lawful, world.
You have yet to show that IT does in fact explain anything about
control systems. You have complained now and then that because
we're still hung up on the basics, you can't get on to the really
interesting stuff. But if you can't defend the basics, it's not
likely that the more interesting stuff will be interesting to
anyone but you.
It seems to me that the reasoning involved in information theory
depends to an inordinate degree on taking just the right attitude
toward it, making just the right interpretation (on which even
you and Allen don't seem to agree and which others versed in IT
like Cliff Joslyn also see differently). Reducing IT to practice
appears almost too difficult to do at all; just look at Allen's
proposed proof of something about entropy, which requires solving
an NP-hard (one might even say NP-impossible) problem on the way.
As a practical approach to explaining behavior, I am not
impressed with IT. I am not one of those who loves complexity for
its own sake. And I am definitely not impressed with the
manipulations I have seen so far, which are loaded with errors
and erroneous predictions. So far your principles have not done
well by you in terms of leading you to verifiable statements
about either real systems or simulations.
···
------------------------------------------------------------
Bill P.