Challenge 1; No conservation of information

CSGnet Archive CSG1993
1

[From Bill Powers (930417.1300)]

Sorry, but I have to go on with this:

Martin Taylor (930417) --

By a stroke of luck I seem to have found the experimental result
to which you referred:

From Allan Randall (930401.1700 EST) --

Before doing my own experiment to compare H(D) and H(D|P), I've
done Martin's "Mystery function" experiment, since it is the
simpler of the two.

Is this the one you meant?

If so, I have to disagree with your characterization of it as
being a refutation of anyone's position. I will agree that Rick
made his claims too strongly and without revealing his extra
conditions, but the bone of contention was whether there was any
information in the perceptual signal about the disturbing
variable. You took it that if the output waveform (or a "mystery"
waveform constructed by an exact duplicate of the output
function) matched the disturbance, this was proof that
information about the disturbance existed in the perceptual
signal, because the real output and the mystery function output
depended on the perceptual signal.

In the first place, neither Rick nor I had ever denied that the
output waveform replicated (very nearly) the waveform of the
disturbing variable when good control existed and with a constant
linear feedback function. So your proof that this was true added
nothing to our positions. It showed only that a duplicate of the
output function would produce the same output as the real output
function. The question was whether this replication could be
ascribed to the passage of information about the disturbance into
the perceptual signal and thence into the output. Your simulation
did not answer that question, but simply assumed that if
replication of waveforms occurred, information MUST have followed
that path. But that is the very point with which I, at least,
took issue.

My argument centered not on the output, but on the perceptual
signal. I pointed out that from what I had learned, the
perceptual signal must contain less information per sample than
the disturbance, using the same measure for both. I then asked
how (for example) 10 bits of information per sample could get
into the output (as necessary to mirror the disturbance), on the
basis of only 4 bits per sample in the perceptual signal (through
which all effects on the output signal must flow). I proposed
that since this was impossible, we could not think of the
situation as one of information flowing from one place to
another, but only as independent measures of a formal number
arbitrarily called "information" being taken from various signals
and variables, just as you could compare measurements of the
position of one end of a seesaw with similar measurements of the
other end without claiming that "position" had flowed from one
end to the other. The measurements of so-called information do
not imply that any quantity "information" is being transferred
from the disturbance into the perceptual signal, or from the
perceptual signal into the output.

Your reply to this was to assert that the sampling frequency
applied to the input function was higher than that applied to the
disturbing variable, so that in the perceptual signal there was
less information per sample, but more samples per unit time, thus
making up the difference and eliminating the paradox. This seemed
to solve the problem for you, although it didn't do so for me. I
wondered, although I didn't comment on it, how taking more
samples of a lower-information content signal could do anything
to increase the information rate, as the samples would be largely
redundant. We then began going around about the relative
bandwidths of the disturbing variable and the input function, and
have now arrived at the point where your claim about these
relative bandwidths has been demonstrated, in simulation, to be
false (today, my earlier post).

You can't get more than one sample of the perceptual signal per
sample of the disturbing variable; if you double the perceptual
signal's sampling rate, you will simply get two identical samples
of each sample of the disturbance's effect, and the second sample
will add no information. If you're going to introduce
hypothetical sampling rates, there's no rationale for introducing
TWO sampling rates, and even if you do you will gain no increase
in the information rate. This device is really a fudge factor
that makes the answer seem to come out right. If you can adjust
sampling rates independently for each signal, you can make each
signal have any information rate you please (ignoring
redundancy). You can make a low information rate into a high
information rate. And none of this takes into account our latest
discussion about the fact that information per sample must
decrease as sampling rate increases.

So the paradox still exists, if you continue to claim that
information from the disturbing variable gets into the control
system and shows up in the output signal. If, however, you admit
that the measure of information is simply a measure, and does not
imply the transfer of any quantity of anything from one place to
another, we will finally agree: any measure of information in the
output is NOT a measure of information in the disturbance. The
transfer of information can't be used to account for the observed
quantitative relationships. Only when the quantitative
relationships have been estabished in some other way can the
information-theoretic measures be made.

It seems to me that behind your arguments there is not only a
concept that "information" carries physical effects, but that
some sort of law of conservation of "information" exists. This is
why you had to introduce a higher sampling rate for the
perceptual signal, to make up (seemingly) for the loss of
information per sample and make sure that all the information
from the disturbance somehow got transmitted into the perceptual
signal.

But what if there is no such law? Suppose that "information" is
simply a number generated by applying a certain computational
procedure to a signal. There is nothing in such a computation to
suggest that the computation must be preserved from one signal,
through a transforming function, into the next; that the next
signal could not yield a higher measure of information than the
previous one. You might ask, "But where would that extra
information come from?" Even to ask the question is to assume a
conservation law. It's perfectly possible that RMS variations in
a second signal would be larger than in the first signal, yet we
don't ask "Where did the extra RMS come from?" We know that RMS
is simply a kind of calculation, and don't expect it to be
conserved.

So if you want a conservation law for information, you must find
some independent justification for it, without assuming it to
begin with. I think that the control-system organization provides
a severe test case, for it seems that the feedback effects on the
input _destroy_ information, yet the measure of information in
the output still matches the same kind of measure obtained from
the disturbance. If information is truly lost at the input, then
there can be no law of conservation of information. And if there
is no law of conservation of information, then information is
simply an arbitrary measure having no physical significance.

ยทยทยท

-------------------------------------------------------------
Best,

Bill P.

2

[Martin Taylor 930417 2030]
(Bill Powers 930417.1300)

Second posting carries maybe a hint of where one part of the problem lies.

I
wondered, although I didn't comment on it, how taking more
samples of a lower-information content signal could do anything
to increase the information rate, as the samples would be largely
redundant.

and

You can't get more than one sample of the perceptual signal per
sample of the disturbing variable; if you double the perceptual
signal's sampling rate, you will simply get two identical samples
of each sample of the disturbance's effect, and the second sample
will add no information.

Ah, but it cankO add more information (partdon spurious characters,
again, this is being done fast on a moderately good line).

That's the nub. Provided that the bandwidth of the observed signal
is low enough, the result of adding observations is to improve the
apparent resolution. Think about averaging. The RMS value of the
average of a set of noisy measurements goes down with the square
root of the number of measures entering into the average. The same
rpinciple applies. You can be indefinitely precise in measuring
something known to be unchanging in magnitude. If it changes slowly,
you can make repeated measurements so long as you are within a time-span
in which there remains information about the value at the moment of
interest. That is to say, within the Nyquist time-limit, if the
thing being measured is band-limited.

About our experiment:

If so, I have to disagree with your characterization of it as
being a refutation of anyone's position. I will agree that Rick
made his claims too strongly and without revealing his extra
conditions, but the bone of contention was whether there was any
information in the perceptual signal about the disturbing
variable.

If that's what you thought the bone of contention was, then, without
wanting to be rude, you didn't read our postings very carefully, and
it explains a lot of the pointless argument.

We wanted to show one contradiction, and we said so, many times, explicitly.
It cannot be true (a) that the output mirrors the disturbance exactly
(or even approximately), and at the same time (b) no information from
the disturbance is obtainable from the perceptual signal.

The method was to show that from the perceptual signal and ANYTHING
else that contained no information about the disturbance, we could
generate the output signal. If the strongly proclaimed presumption
(a) was maintained, then (b) had to be false.

Since the result was (as we thought unnecessary to demonstrate) that
the Mystery Function output was identical to the ECS output, then
either (a) or (b) had to be false. Either the output did NOT have
information about the disturbance (contradicting many strong claims)
or information about the disturbance CAN be obtained from the
perceptual signal (contradicting many other strong claims).

Insofar as there is control, the earlier claim that the output has
information about the disturbance has to be true--it is, after all,
generating whatever has to be generated to counter the disturbance.
Therefore, it must be (b) that is false. That's the QED. Not
at all that the Mystery function can regenerate the disturbance.
On that issue, I am on the other side of Rick. He (and I believe you)
argue that there exists a functional relationship between the
ECS output and the effect on the CEV that opposes the disturbance.
I think of it as being a variable function that changes with a
bandwidth potentially as wide as that of the controllable disturbance.
(Oh,oh...sorry about the miswording. I mean, of course, the bandwidth
of the part of the perceptual signal that can be maintained under control).

What Rick kept trying to show was that a non-linear relationship between
output and disturbance would make it difficult to extract the
value of the disturbance from the output. What we were ALL (I think)
trying to show was that the fact of control was the important issue.
If the Mystery function were to be a disturbance-extractor, as you kept
trying to get us to say (though we wouldn't), then it would have to
incorporate the same non-linearity or variability as actually existed
between output and CEV. But that would pose no problem for our argument,
because even THAT information would not be information about the
disturbance. All we showed was that the fact of control both permitted
and required that information from the disturbance was passed through
the perceptual signal. I have not yet understood why that statement,
which seems to me equivalent to saying "a control system can be
physically realized," causes so much resistance in you and Rick.

The point that is important for extension of the discussion, if we
ever get this cleared up, is that some part of the perceptual signal
may well not be controlled, and that part is what is important for
the ECSs at a higher level. Avery's example of the tray held in
two hands, when one is jogged, may possibly provide an instructive
application.

None of this will be new to you, I think.