[From Bill Powers (930417.1300)]
Sorry, but I have to go on with this:
Martin Taylor (930417) --
By a stroke of luck I seem to have found the experimental result
to which you referred:
From Allan Randall (930401.1700 EST) --
Before doing my own experiment to compare H(D) and H(D|P), I've
done Martin's "Mystery function" experiment, since it is the
simpler of the two.
Is this the one you meant?
If so, I have to disagree with your characterization of it as
being a refutation of anyone's position. I will agree that Rick
made his claims too strongly and without revealing his extra
conditions, but the bone of contention was whether there was any
information in the perceptual signal about the disturbing
variable. You took it that if the output waveform (or a "mystery"
waveform constructed by an exact duplicate of the output
function) matched the disturbance, this was proof that
information about the disturbance existed in the perceptual
signal, because the real output and the mystery function output
depended on the perceptual signal.
In the first place, neither Rick nor I had ever denied that the
output waveform replicated (very nearly) the waveform of the
disturbing variable when good control existed and with a constant
linear feedback function. So your proof that this was true added
nothing to our positions. It showed only that a duplicate of the
output function would produce the same output as the real output
function. The question was whether this replication could be
ascribed to the passage of information about the disturbance into
the perceptual signal and thence into the output. Your simulation
did not answer that question, but simply assumed that if
replication of waveforms occurred, information MUST have followed
that path. But that is the very point with which I, at least,
took issue.
My argument centered not on the output, but on the perceptual
signal. I pointed out that from what I had learned, the
perceptual signal must contain less information per sample than
the disturbance, using the same measure for both. I then asked
how (for example) 10 bits of information per sample could get
into the output (as necessary to mirror the disturbance), on the
basis of only 4 bits per sample in the perceptual signal (through
which all effects on the output signal must flow). I proposed
that since this was impossible, we could not think of the
situation as one of information flowing from one place to
another, but only as independent measures of a formal number
arbitrarily called "information" being taken from various signals
and variables, just as you could compare measurements of the
position of one end of a seesaw with similar measurements of the
other end without claiming that "position" had flowed from one
end to the other. The measurements of so-called information do
not imply that any quantity "information" is being transferred
from the disturbance into the perceptual signal, or from the
perceptual signal into the output.
Your reply to this was to assert that the sampling frequency
applied to the input function was higher than that applied to the
disturbing variable, so that in the perceptual signal there was
less information per sample, but more samples per unit time, thus
making up the difference and eliminating the paradox. This seemed
to solve the problem for you, although it didn't do so for me. I
wondered, although I didn't comment on it, how taking more
samples of a lower-information content signal could do anything
to increase the information rate, as the samples would be largely
redundant. We then began going around about the relative
bandwidths of the disturbing variable and the input function, and
have now arrived at the point where your claim about these
relative bandwidths has been demonstrated, in simulation, to be
false (today, my earlier post).
You can't get more than one sample of the perceptual signal per
sample of the disturbing variable; if you double the perceptual
signal's sampling rate, you will simply get two identical samples
of each sample of the disturbance's effect, and the second sample
will add no information. If you're going to introduce
hypothetical sampling rates, there's no rationale for introducing
TWO sampling rates, and even if you do you will gain no increase
in the information rate. This device is really a fudge factor
that makes the answer seem to come out right. If you can adjust
sampling rates independently for each signal, you can make each
signal have any information rate you please (ignoring
redundancy). You can make a low information rate into a high
information rate. And none of this takes into account our latest
discussion about the fact that information per sample must
decrease as sampling rate increases.
So the paradox still exists, if you continue to claim that
information from the disturbing variable gets into the control
system and shows up in the output signal. If, however, you admit
that the measure of information is simply a measure, and does not
imply the transfer of any quantity of anything from one place to
another, we will finally agree: any measure of information in the
output is NOT a measure of information in the disturbance. The
transfer of information can't be used to account for the observed
quantitative relationships. Only when the quantitative
relationships have been estabished in some other way can the
information-theoretic measures be made.
It seems to me that behind your arguments there is not only a
concept that "information" carries physical effects, but that
some sort of law of conservation of "information" exists. This is
why you had to introduce a higher sampling rate for the
perceptual signal, to make up (seemingly) for the loss of
information per sample and make sure that all the information
from the disturbance somehow got transmitted into the perceptual
signal.
But what if there is no such law? Suppose that "information" is
simply a number generated by applying a certain computational
procedure to a signal. There is nothing in such a computation to
suggest that the computation must be preserved from one signal,
through a transforming function, into the next; that the next
signal could not yield a higher measure of information than the
previous one. You might ask, "But where would that extra
information come from?" Even to ask the question is to assume a
conservation law. It's perfectly possible that RMS variations in
a second signal would be larger than in the first signal, yet we
don't ask "Where did the extra RMS come from?" We know that RMS
is simply a kind of calculation, and don't expect it to be
conserved.
So if you want a conservation law for information, you must find
some independent justification for it, without assuming it to
begin with. I think that the control-system organization provides
a severe test case, for it seems that the feedback effects on the
input _destroy_ information, yet the measure of information in
the output still matches the same kind of measure obtained from
the disturbance. If information is truly lost at the input, then
there can be no law of conservation of information. And if there
is no law of conservation of information, then information is
simply an arbitrary measure having no physical significance.
ยทยทยท
-------------------------------------------------------------
Best,
Bill P.