Testing Information Theory

[From Rick Marken (950217.1100)]

Me to Martin:

You intellect is truly too dizzying for me.

Martin Taylor (950217 10:4) --

Sorry, but I thought that Laplace Transforms were a standard way of
analyzing control systems, not "arcane mathematical gymnastics." The rest
was quite straightforward, with very little of the mathematics that bother
you.

It wasn't the mathematics that bothered me. It was verbalizations like this:

The "incontrovertible demonstration" was precisely the use of data to show
that they [variations in d] can [be inferred], PROVIDED that the output and
feedback functions are known. The information needed to specify those
functions is fixed, AND IS INHERENT IN THE ACTUAL STRUCTURE OF THE CONTROL
LOOP, whereas the information from the disturbance approaches infinity over
infinite time.

Why would the information needed to specify those functions matter to
the operation of the control system itself? How does information
"inhere" in the struture of a control loop? How can I know that it inheres
in this way? How do I know that information from the disturbance approaches
infinity over time? How do I test any of these claims--other than by asking
you, of course;-)?

I like simple, clear tests of theories so it's easy for me to tell what's
going on. It seemed to me that we got to a point in the "information" debate
where there was an opportunity for a simple test of information theory; we
agreed that I would send you a perceptual signal obtained from a control
model and you would send back the corresponding disturbance variable
(assuming there was just one) that was being resisted by the control system.
If you were able to do this with several perceptual signals, it would prove
to me that there IS information in the perceptual signal that can be used to
generate the outputs that are compensating for the effects of the
disturbance. If not, it would show that control systems don't "use"
information in perception to generate outputs; control systems CONTROL
perceptions.

This simple test was never done because you kept saying that information
theory required more and more details about the control system (output
function) and its environment (feedback function) in order to reconstruct the
disturbance. Eventually, it became clear (to me) that information theory was
just another name for control theory; what you needed was exactly what a
control theorist would need in order to determine the disturbance given the
perceptual signal. Moreover, you said that the simple test I thought we
had agreed to was really just a reflection of my dismal understanding of
your explanation of information theory. So there was no test.

Eventually it also became clear that information theory does not fail tests.
The most dramatic demonstration of this occurred when you predicted
(and claimed that the predicitno was based on information theory) a high
correlation between the derivatives of perceptual and disturbance variables.
About one hour after the prediction Bill Powers posted a circular cloud that
represented the actual BEST relationship (at optimal lag) between the
derivatives. This apparently posed no problem at all for information theory
since you quickly came back to say that Bill's result was EXACTLY what
you expected -- information theory confirmed again.

It is difficult for me to get excited about testing a theory (like
information theory) when I know (as I know now) that, whatever the result of
the test, it will be consistent with the predictions of the theory. So it
probably hopeless to suggest this but maybe we can make this discussion of
information substantive if you would suggest one simple test that would put
information theory on the line? Does information theory make some
quantitative prediction about the behavior of a subject in a simple tracking
task-- a prediction that can be tested against data? Can you describe an
experiment I can do to test a QUANTITATIVE predciction of information theory?
Perhaps a prediction regarding the effects of disturbance bandwidth on some
parameter of tracking performance?

Since you say that almost everything I say about information and information
theory is wrong, I think it would be best if you told me exactly how to do
the experiment, how to derive the prediction, how to measure the
variables, etc. I'll just collect the data and see if it matches the derived
prediction. If you do this, please remember to keep it as SIMPLE as possible -
- not only for my sake but also for the sake of people who are watching and
who are reading through this quickly. Try to come up with something that
provide a CLEAR, SIMPLE test of information theory.

Perhaps information theory DOES have something worthwhile to contribute to
our understanding of purposive behavior. I just haven't seen it yet.

I have no personal investment in the failure of information theory. I did
study information theory in graduate school and what I learned was that
it is a way of measuring the channel capacity of organisms viewed as input-
output devices. So the information theory I know seems completely
incompatible with the PCT model of purposive behavior. But maybe it's not.
All I'm asking is that you show me evidence of something information theory
accounts for in actual purposeful behavior -- the kind we see in our simple
little control tasks . I don't want more descriptions of what information
theory can do; I want to SEE what information theory actually does -- in a
real control task.

Best

Rick