[From Bill Powers (940731.0635 MDT)]
Martin Taylor (940727.1500) --
If A = B+C and you know the ranges of B and C, then some values of A
constrain the possibilities for B and C absolutely. Other values of A
change the likelihood of different values of B and C, so even in the
everyday sense, A conveys information about B and C.
In all your arguments you have to assume some prior knowledge of the
components of C in order to show that C is constrained by any one of its
components. When ALL you are given is C, it is impossible to deduce
anything about A and B. There is nothing in the value of a single
variable to suggest what constraints on its components might exist, if
any.
The Information in the perceptual signal that comes from the disturbance
is computed in part from U(p|d): the uncertainty in p GIVEN D. But the
control system is not "given" the values of d. Without some independent
way of ascertaining the effects of d, it is impossible to retrieve the
information which you claim is in the perceptual signal concerning the
effect that the disturbance, acting alone, would have had. If the
control system is not given d, it can't extract the information about d
in the perceptual signal; if it is given d, it has no need to extract
that information since it already knows d by some means other than using
the perceptual input function.
The analyst does have an independent way of determining the effects of d
without relying on the perceptual input function or perceptual signal.
Hence the analyst can compute a quantity U(p|d) and an independent
quantity U(d). From these calculations, the analyst can construct an
arbitrary quantity called (unfortunately) "Information."
However, and this is the second point you are overlooking, even if the
analyst does hand the results of this calculation to someone else --
that is, the value of the quantity you call "the Information about p
that is in d," that someone else would be unable to reconstruct the
effect of the disturbance from that number. The result of the
calculation would be a single number derived from the behavior of p and
d over some interval of time: a quantity of Information or rate of
Information flow. From that single number it would be impossible to
derive even the statistical characteristics of the effect of the
disturbance: its bandwidth or amplitude or any other such
characteristic. A single number is a single number.
So a measure of information in p given d in fact conveys no information
about either d or p, in the usual sense of information. If I tell you
that a message carried 10 bits per second of information to a properly-
defined receiver, you still have no idea what that message was, much
less any idea of how the message originated. Any number of different
messages, even messages with contradictory meanings, could be carried to
a given receiver by signals bearing exactly the same amount of formally-
defined "Information." So Information gives you no information about the
content of the message.
Your attempt to link Information as formally computed with information
as usually understood is futile. You can get from information to
Information, but you can't get back. When you compute Information, you
discard information. And it is YOU doing these computations, not the
system of which you speak. The computation of Information is irrelevant
to the operation of the system; the way the system operates determines
completely what Information you will calculate, but the reverse is not
true; the number you call Information plays no part in the operation of
the real system. It is at best an epiphenomenon, and most realistically,
as I see it, a gratuitous invention.
It is most unfortunate that the quantity U(x|y) - U(y) was linked to
common-language terms like uncertainty and information. Doing so forced
an interpretation that was justified only by vague intuitions; when
carefully examined this interpretation ceases to make any sense.
Whatever you prove about Information has no necessary link to
information, or to the mechanisms that make a given system behave as it
does. It is only the pre-empting of common-language terms that makes it
seem that Information has any physical meaning.
···
-----------------------------------------------------------------------
Best,
Bill P.