# Information -- the challenge

[From Rick Marken (930310.1400)]

Allan Randall (930309.1320) --

First, thanks for the paper; it looks like it will, indeed, be
an excellent addition to the PCT library. As to your post --
I'm sure Bill P. will have all kinds of helpful things to say.
I'll just comment on a couple things that caught my attention.

First, you say that Information Theory (IT) is to PCT as
calculus is to Newton's laws; IT is a tool like calculus.
But calculus helps us make detailed predictions from the
basic model (in my business we do this all the time -- it
helps to know the future position and velocity of a satellite).
But at the end of your post you say:

(condition 2 or 3) will be better in a real-world situation.
This seems to be beyond the scope of information theory as it
stands now.

Well, IT isn't much of a tool if it can't help us predict things;
looks like PCT WITH IT is no better off than Isaac's brother Phil
WITHOUT calculus.

At another point in your post you say:

disturbance can be extracted from this data, then there is
no way the system can translate the error from this signal into
an action on the world that will counter the disturbance. Is this
or is this not true?

This is NOT TRUE. Surprise!

If you agree, then you are agreeing with an
information theoretic analysis.

So I guess I disagree with an IT analysis -- and if I'm right
(which I am) then the IT analysis is wrong, right? That's
really what Bill's challenge was about -- demonstrating this
fact experimentally.

It is this fact about control systems that nails everyone to the
wall -- and proving it to myself is what turned me into a PCT
"fanatic" because it kabashes the entire ediface on which the
behavioral sciences are built; the input - output model of
behavior. In a high gain, negative feedback control loop, the
output DOES NOT depend on the sensory input; rather, SENSORY INPUT
IS CONTROLLED BY OUTPUT. What you do in a tracking task is NOT
caused by what you see; there is a LOOP so that what you see is
both a cause AND A RESULT of your output. The nearly perfect
relationship between output and disturbance does NOT exist
about the disturbance. When control is good, there IS NO information
about the disturbance in the stimulus -- NONE, ZILCH, NADA. The output
mirrors the disturbance because this is what the output MUST DO in order
to keep the input IN THE REFERENCE STATE; this is the magic of
closed loop control. We're talking about purposeful behavior, here,
and it is not the result of "feedback guidance", "stimulus control",
or "programmed output"; we're not talking about behavior that is
the control of perception.

If you haven't already done so, I highly recommend the first paper
in Chapter 3 of "Mind Readings". This is an experimental "proof"
that sensory input IS NOT, never was and never will be the cause of
control movements in a closed loop tracking task.

Best

Rick

[Allan Randall (930312.1200)]

Rick Marken (930310.1400) writes:

First, you say that Information Theory (IT) is to PCT as
calculus is to Newton's laws; IT is a tool like calculus.
But calculus helps us make detailed predictions...
...at the end of your post you say:

>(condition 2 or 3) will be better in a real-world situation.
>This seems to be beyond the scope of information theory as it
>stands now.

Well, IT isn't much of a tool if it can't help us predict things;
looks like PCT WITH IT is no better off than Isaac's brother Phil
WITHOUT calculus.

But calculus does NOT allow you to make the kind of prediction
that Bill Powers is asking for. What Bill is asking for is like
asking calculus to predict the orbit of a planet, all by itself
WITHOUT any of Newton's Laws. Bill is specifically asking, unless
I am misunderstanding him, for a PCT-type prediction using
information theory and ONLY information theory. I agree that
this may not be possible. What I do not want to see him do is
to reject what could turn out to be a valuable tool, on the basis
that it cannot do the job all by itself. This is what I was getting
at with the Newton analogy. Contrary to what you imply, calculus
simply can't make physical predictions unless it is used in
combination with a physical model of some kind. All by itself,
it is just a mathematical technique, like information theory.

>disturbance can be extracted from this data, then there is
>no way the system can translate the error from this signal into
>an action on the world that will counter the disturbance. Is this
>or is this not true?

This is NOT TRUE. Surprise!

>If you agree, then you are agreeing with an
>information theoretic analysis.

So I guess I disagree with an IT analysis -- and if I'm right
(which I am) then the IT analysis is wrong, right?

Right. Exactly. Here is something we can actually nail down. The
statement I made, which you say is not true, is the whole crux
of this controversy. It is a fundamentally information theoretic
statement. The reason why I asked whether it was true or not was
that I could not imagine a PCTer disagreeing with it (I was wrong -
you did), but at the same time I considered it an information
theoretic statement. Here you are claiming the statement is actually
false - there is no information about the disturbance in the
perceptual signal - a claim that is in direct contradiction with
Ashby's Law of Requisite Information. If you are correct, Ashby's Law
is completely unfounded. (However, I don't think you actually are
correct.)

It is this fact about control systems that nails everyone to the
wall -- and proving it to myself is what turned me into a PCT
"fanatic" ...
...In a high gain, negative feedback control loop, the
output DOES NOT depend on the sensory input; rather, SENSORY INPUT
IS CONTROLLED BY OUTPUT.

The second statement is true, but I think your first statement is
false. The output DOES depend on the sensory input, and the sensory
input DOES depend on the output. We do not have to choose between
the two. It is better to say they are interdependent, than that
one depends on the other. Isn't your response-stimulus description
just as wrong as stimulus-response?

What you do in a tracking task is NOT
caused by what you see; ...

Magic, then?

... there is a LOOP so that what you see is
both a cause AND A RESULT of your output.

Now you are back to the closed loop and admitting that they are
interdependent.

The nearly perfect
relationship between output and disturbance does NOT exist

This is silly. How can there be ANY relationship between the output and
disturbance, let alone one that is "nearly perfect," if the system has
impossible, Rick. I think either you are misinterpreting what I mean by
"information," or you believe some kind of witchcraft is responsible
for control.

When control is good, there IS NO information

This is true NOT when control is *good* but when control is *perfect*,
which is inherently impossible for an error-control system, as Ashby
said. Let's be clear what we mean by "disturbance." Stop me when you
disagree. The disturbance is the sum of all the various environmental
influences impinging on the CEV. This CEV is not an absolute property
of the environment, but is defined by the hierarchy. In information
theory terms, the hierarchy is an encoding scheme for representing the
environment. Each perceptual signal represents one CEV in the
environment. This does *not* mean that the disturbance exists in
the hierarchy and not in the world. In order to describe anything,
says information theory, it must be described in some language. The
disturbance is in the world, but the description that isolates it as
an entity seperate from the rest of the environment is in the
hierarchy, not the world.

Only that information which is relevant to control of the CEV is
transmitted in the percept. But to say there is NO information at all,
none, zilch, nada, is just incoherent. If I am driving my
car down the road and there is a sudden huge gust of wind from the
right and at the same time I start sliding on the ice, the
"disturbance," in PCT terms, is the disturbance to the CEV that results
from these forces. But an outside observer will tend to describe
the disturbance in a different language (a different encoding scheme).
They will probably describe the gust of wind, its force, and the sliding
motion due to the ice as seperate, complex entities. But this
description is no more an absolute depiction of reality than the
perceptual one inside the driver! The "disturbance" relevant to PCT
is the one described inside the organism that is controlling, not the
one described dispassionately by an external observer. Both use
an encoding scheme to describe the disturbance. One scheme requires
many many bits, while the other requires very few.

This is what I mean when I say that the hierarchy brings the information
content of the disturbance in line with the output capacity. An external
observer describes the disturbance in a language that requires many bits
(such as the detailed description of the molecular positions/momentums
that the compensatory thermostat requires). The internal hierarchy
describes the same real-world disturbance in a language requiring only
a small number of bits - a number that can be handled by the capacity
of the output channel (such as the much simpler description used by
the error-control thermostat). This is Ashby's Law.

The output
mirrors the disturbance because this is what the output MUST DO in order
to keep the input IN THE REFERENCE STATE; this is the magic of
closed loop control.

Yes, this does sound like magic, and not a scientific explanation at
all. There must be an explanation WHY the control system is able to
do what it MUST DO. Just to say that it MUST DO it does not explain
anything.

ยทยทยท

-----------------------------------------
Allan Randall, randall@dciem.dciem.dnd.ca
NTT Systems, Inc.
Toronto, ON

[From Rick Marken (930312.1300)]

Allan Randall (930312.1200) --

But calculus does NOT allow you to make the kind of prediction
that Bill Powers is asking for. What Bill is asking for is like
asking calculus to predict the orbit of a planet, all by itself
WITHOUT any of Newton's Laws.

I don't think Bill or I know what the heck you folks think IT
is good for. Feel free to use IT along with anything you like
to make a prediction. All I want to see is how IT (calculus-like)
can improve what we do in PCT -- which is try to discover controlled
variables and model now they are controlled.

In a previous post you said:

disturbance can be extracted from this data, then there is
no way the system can translate the error from this signal into
an action on the world that will counter the disturbance. Is this
or is this not true?

I said this is not true and you said:

Right. Exactly. Here is something we can actually nail down. The
statement I made, which you say is not true, is the whole crux
of this controversy. It is a fundamentally information theoretic
statement. The reason why I asked whether it was true or not was
that I could not imagine a PCTer disagreeing with it (I was wrong -
you did), but at the same time I considered it an information
theoretic statement. Here you are claiming the statement is actually
false - there is no information about the disturbance in the
perceptual signal - a claim that is in direct contradiction with
Ashby's Law of Requisite Information. If you are correct, Ashby's Law
is completely unfounded. (However, I don't think you actually are
correct.)

I'm correct.

OK. Let's get quantitative here. You can do the information analysis
if you like; I'll just give you the results that I know. In a
compensatory tracking task, when control is good, the correlation
between the disturbance and the output (handle movement) is typically
.99+. The correlation between input (cursor-target variations) and
output is on the order of .02. In other words, what the subject sees
(stimulus) has NO relationship to output; but the output is a perfect
mirror of the disturbance. Now you are claiming that there is information
in the variations in the stimulus that communicate information about
the disturbance -- this, accoring to you, is why the output mirrors
the disturbance.If you are right, then when the SAME disturbance is
used on two different occasions and the output mirrors the disturbance
in both cases then the information in the stimulus on both trials
must be the same -- right? This is what the experiment in Ch 3 of
"Mind readings" tests; the prediction is that the correlation between
traces of the stimulus on two different trials using the SAME disturbance
should be nearly the same; but the typical correlation between the
stimulus traces was ALWAYS less than .2 -- in one case it was .0032.

So there is NO information about the disturbance in the stimulus
(input) to a control system. My experiments show what I would interpret
as nearly PERFECT transmission of information from the disturbance to
the output -- with virtually NO information about the disturbance in
the only channel that could be carrying that information -- the stimulus.

All this experimental rigamarole shouldn't really be necessary
(except that the result is SO startling and unbelievable); all
that should be necessary is the equations of control showing that
o = -d (notice that the perceptual variable does NOT show up
in this equation) and the fact that the stimulus input variable
in a compensatory tracking task is i = o + d; that is, what you
see (the input, i)at any instant is the COMBINED result of your
actions (o) and disturbance (d). So information about d is not even
physically EXPECTED to be visible in the input in control tasks like the
compensatory tracking task. The only way a person could possibly
get information about d from the input data is to know, at
any instant, what EFFECT they themselves are having on the input
(that is, know what o is) -- and this is impossible in principle.
If they could get information about o then they could continuously
solve for d = i-o (they can see i and continuosly subtract the
informaiotn about o). But how could they POSSIBLY know o? DO you
know what that would require?

This is silly. How can there be ANY relationship between the output and
disturbance, let alone one that is "nearly perfect," if the system has

I suggest that you read chapter 3 of Mind Readings and ALL of
Behavior: The control of perception. I think your question above
is a good one; it reveals exactly WHY PCT has NOT gained much of
a following in the behavioral sciences. You can't get many
people to hop on your bandwagon when they think one of your
fundemental observations is "silly". I know it seems silly -- how
about crazy and impossible too. Unfortunately (for those of us who
understand it and, hence, are alienated from mainstream behavioral
science) it it TRUE.

This is just physically
impossible, Rick.

It is not only physically possible, is the only physically correct
interpretation of the situation -- PCT or no PCT (remember that
i = o + d). The interpretation that is actually WRONG is that one
that seems right (and OBVIOUS) to you and 100,000 other behavioral
scientists out there -- that there is information about the
disturbance in the stimulus. There IS NO INFORMATION ABOUT THE
DISTURBANCE IN THE STIMULUS. Goodby behavioral science as usual,
hello looney bin.

I think either you are misinterpreting what I mean by
"information," or you believe some kind of witchcraft is responsible
for control.

Neither. I think I am correctly interpreting the meaning of
information (though I'm willing to defer to you and Martin on
that) and I believe that the nearly perfect transmission of information
from disturbance to output despite the absense of ANY information
about the disturbance in the input is a well understood result of the
behavior of a negative feedback control loop -- no witchcraft necessary.

Only that information which is relevant to control of the CEV is
transmitted in the percept. But to say there is NO information at all,
none, zilch, nada, is just incoherent.

I'm just like that -- incoherent but honest (sort of like the fool
in Lear). Fact is, NO information about the disturbance is
transmitted in the percept in a tracking task. But the ball is in
your court; SHOW ME that I am wrong. I can take it.

There must be an explanation WHY the control system is able to
do what it MUST DO. Just to say that it MUST DO it does not explain
anything.

Yes -- you are corrrect. I was trying to dodge a lot of explanation;
but the explanation is simply that the output of a control
system is continuously driven by an error variable (r-p) whose
effect on output is REDUCED by that output ; ie. the explanation
is the closed loop negative feedback process. That process, with
NO information about the distubance or disturbances that are influening
the perceptual input, produces outputs that are a perfect mirror image
of the net disturbance -- and, hence, CONTROL.

Best

Rick