[From RIck Marken (930329.2000)]
Allan Randall (930329.1700 EST)--
I will describe a possible
experiment, predict the results, and provide some analysis. You can
respond as to which of these three, if any, you have a problem with.
Wonderful!
Let's take a computer simulation of a simple control system
Excellent idea. Do this information measurement with a simulation
of a control system.
We would first run the control system under normal closed-loop conditions,
recording disturbance and output, and noting that the disturbance is
almost perfectly countered at the output. The control system's perceptual
line would then be cut, and superceded by a new experimental line. All
possible perceptual inputs would then be presented, one at a time,
starting with the shortest and working up.
Let me see if I have this right. I assume that cutting the "perceptual
line" means that the system is now operating open loop. Perceptual
signals are injected directly into the system as Pi, the system does its
usual job of transforming Pi values into output -- we use a simple integrator
in many of our models so O := O + k*(R-P) -- but now O has no effect on the Pi
values. Is this right? The equation for the open loop output, O, is the same
as the one used in the closed loop version of the experiment. Is my
interpretation correct so far? Also, what is the length of a perceptual
input? Are you referring to the length of the vector of values that
make up Pi?
This would be repeated until
the disturbance recorded from the first experiment appears on the output.
Let's assume that the output in the first experiment contained 100% of
the information about D, since we all seem to agree that this is very
nearly the case. Now we need only look for a replication of the initial
experiment, and H(D) will simply be the length of the successful
percept.
I really have to know what you mean by "length"? If it's the number of
samples in the Pi vector, finding the first Pi value that maps into
the disturbance could take a hell of a lot of iterations.
Let's label the possible percepts {Pi} = {P0,P1,P2, ...}. P0 will be
the null message, which is equivalent to simply cutting the perceptual
line and not providing any replacement input at all. Let's call the
first Pi to successfully replicate the initial output Pk. Pk might
very well be equal to the original P, but it could be shorter. We are
guarenteed that it will not be longer. The entropy of the disturbance is:
H(D) = |Pk| <= |P|
Assuming that "length" is the number of samples in Pi, then what you
are saying is that H(D) is proprtional to the length of an input
perceptual signal that is put into the open loop model and from which
O = D is recovered. So if D had 1000 samples and I could recover all
1000 values by plugging in a Pi with 500 samples, then the entropy of
D is 500? Is that right?
We then perform the same procedure to compute H(D | P). This time,
P is a given, so it is always provided to the perceptual line "for free"
and not counted in |Pi|.
Now I really have to be clear about this. To find H(D|P) you do the
SAME THING you did to find H(D)? So you inject a perceptual signal
into the open loop (as you did before) and find the resulting disturbance
predictions? Is this right?
This time we will be successful on the very
first attempt. Since this setup is equivalent to our initial experiment,
without any extra input to the perceptual line, the first successful Pi
will be P0, the null message. The entropy of the disturbance given the
percept therefore is:
Now I'm confused. Is P coming in via the usual closed loop perceptual
input or "for free" as an open loop input? This is a VERY important
point. If P in coming in closed loop then you are not computing
the distrubance in the same way as you were when you injected Pi.
In the first part of the experiment (if I understand you correctly)
you are computing O = O + (R-P) open loop and seeing if the resulting
O matches D. The first (and sortest) P that gives you this match
is the measure of the entropy of the disturbance (measured as the number
of samples needed to reduce the uncertainty). Now you want to find
the entropy GIVEN that you know the P that "works" -- call this P'.
I think the appropriate way to measure this is the same as you measured
H(D) -- ie. compute O := O + (R - (P'+P0)) open loop as you did before.
Since P' presumably produces the values of O that match D, adding P0 to
P' should just screw things up -- which is why you imagine the length
of P0 should be zero so that
H(D | P) = |P0| = 0
Is the above a fair description of what you propose? If so, I approve
and would recomend that you do the experiment.
But let me know if I have it right before you start programming.
My question is: what is wrong with the above reasoning?
Nothing, if my understanding of it is correct.
Do you actually support the idea that the system can control blind?
Of course not. Control is the control of perception.
If not, for your claim to be right, I must have goofed up in my reasoning.
If I have interpreted your reasoning correctly then there is nothing
wrong with it.
Do you disagree about the results that I have assumed we would get
from this experiment?
Yes. But you'll see when you actually do the experiment.
Do you disagree with the way the problem was
divided into program, language and output?
I don't think so. Looks OK.
Do you disagree with the
derivations? Definitions? The discrete nature of the computer simulation?
Nope. Looks good.
There are many places in the above argument where our disagreement might
lie. If we can agree as to where we disagree, we just might be able to
come to an understanding.
There is only one possible place where we might disagree. This is
in how P' is injected into the system. If it comes in via the
usual operation of the loop, then I will not accept the proposal.
There are two reasons for this. The first is logical; in the
computation of H(D|P) you are proposing to measure the length of
the P0 that is added to P' to increase the ability of P' to produce
D. In order to try more than one added P0 string, you must have the
SAME P' available each time. If P' enters through the operation of
the closed loop it is not true that P' will be EXACTLY the same on
each repetition of the operation of the loop. So you must have P'
available (as an independent variable, so to speak) in order to test
for improved prediction with several added P strings. I know you
expect to need only one P0 value -- that any a P0 value of length
0 can be added to P' and maintain the ability of P' to produce O=D.
But you do have to try adding at least ONE P0 string (other than
the zero length one) to show that this is true.
The other reason also seems logical (to me) but I don't think it is
necessarily as strong. The reason is "fairness" really -- you are
measuring H(D) by injecting P strings open loop; it seems reasonable
to measure the change in entropy with P present (H(D|P)) in the same
way -- that is P, is injected open loop to see how well it predicts
the disturbance.
Hope we are in agreement.
Best
Rick