Predictability

CSGnet Archive CSG1996
1

[Martin Taylor 960322 11:30]

Rick Marken (960321.2100)

I know this risks confusing the issue. Sorry about that.

David Wolsk (960321)

the relationship between error reduction and predictability seems easy to
grasp.

But it actually isn't necessary. Have you heard of the E. coli demo? In
that one, a person controls the position of a dot moving on the computer
screen by pressing the space bar. The dot moves in a straight line until
the bar is pressed, after which the dot moves in a new, randomly selected
direction. So the perceptual result (direction of dot movement) of a
response (bar press) is unpredictable; yet a person can easily control
the position of the dot -- keep it near a target -- by pressing the
bar appropriately.

I don't know which aspect of "the relationship" David Wolsk finds easy to
grasp, and Rick is right to point out the success of the e-coli method of
control despite the unpredictability of the direction of movement that
follows a bar press.

But e-coli, seen from another viewpoint, is highly predictable. It is not
influenced by external disturbances, only to the influence of the control
action (thebar press). If the bar-presser does nothing, the future position
of the e-coli can be determined precisely, forever. No matter which direction
it sets off in after a bar press, it keeps going in the same precise
direction at the same speed thereafter until the next bar press. If its
direction were unpredictable instant-by-instant, control would not be
possible. Indeed, the only important unpredictability in the e-coli demo
is worth at most 1 bit per bar press. The actual direction is immaterial
other than whether it is going sufficiently toward the target to satisfy
the bar-presser.

I suggest an experiment (hesitantly, since I don't propose to program it).
Take the standard e-coli demo, but have the direction change incrementally
and randomly at each compute iteration, as if an e-coli with substantial
mass (and therefore inertia) were being buffeted in a Brownian motion
during its travel. It would be interesting to determine the relative
accuracy of control as a function of the size distribution of the
increments. The e-coli is then being subjected to external disturbing
influences, as are the CEVs in most control experiments.

If that experiment shows anything interesting, I suggest a second (which
could be done first, I guess, since they aren't interdependent). Make the
e-coli path curved, with a curvature that varies randomly as before, rather
than with a direction that varies randomly.

Either variant alters the predictability of the e-coli path, and might or
might not affect the ability of the bar-presser to keep it near the target.
My guess, and it is only a guess, is that there won't be much difference
when e-coli is near the target, but there might be a difference in how
long it takes the bar-presser to bring it near the target.

Whether these thought experiments bear on what David Wolsk was thinking,
I don't know. And I'm not trying to downplay the importance of what Rick
said about it, either.

Martin

2

[From Rick Marken (960322.1300)]

Martin Taylor (960322 11:30) --

But e-coli, seen from another viewpoint, [that of the true believer in
information theory;-) -- RM] is highly predictable.

I suggest an experiment...Take the standard e-coli demo, but have the
direction change incrementally and randomly at each compute iteration

Been there; done that.

[The experiment] alters the predictability of the e-coli path, and might or
might not affect the ability of the bar-presser to keep it near the target.

Well what is your prediction? Might or might not? If the e. coli (do) is
under control the prediction is easy: the disturbance _will_ not affect the
ability of the bar- presser to keep the dot near the target.

My guess, and it is only a guess, is that there won't be much difference
when e-coli is near the target, but there might be a difference in how
long it takes the bar-presser to bring it near the target.

Your guess is wrong. This experiment has been done (Marken and Powers,
_Behavior Neuroscience_, 1989, Vol. 103, No. 6, 1348-1355); it is re-printed
in _Mind Readings_ (which you apparently didn't read too carefully; shame on
you;-)). Behavior with the disturbance present is indistinguishable from
behavior when the disturbance is not present: it's called CONTROL.

PCT shows that "predictability", like "redundancy" and "information", is not
likely to be a useful component of an explanation of behavior. Concepts like
these are relics of a science of human nature which placed much of burden
of explaining behavior on imagined properties of the environment
(environmental predictability, redundancy, and information). We don't
need these myths any more.

I know that that Pope will give up on Catholicism before you give up on
information theory. But let's try to do PCT here, OK. The up side of doing
this is that your "guesses" about how people behave will become _much_ more
accurate;-)

Best

Rick

3

[Martin Taylor 960322 17:45]

Rick Marken (960322.1300)

Martin Taylor (960322 11:30)
I suggest an experiment...Take the standard e-coli demo, but have the
direction change incrementally and randomly at each compute iteration

Been there; done that.

Great!

This experiment has been done (Marken and Powers,
_Behavior Neuroscience_, 1989, Vol. 103, No. 6, 1348-1355); it is re-printed
in _Mind Readings_ (which you apparently didn't read too carefully; shame on
you;-)).

More shame on me--I don't have Mind Readings. I think I have all the other
recommended books. Sorry about that. Some day I'll probably get it. And our
library doesn't hold Behaviour Neuroscience.

Maybe you could tell me about the parameters you ran. You say that e-coli
changed its direction on every compute iteration regardless of whether the
bar was pressed. How much, or with what range of variation? As I read your
comment:

Behavior with the disturbance present is indistinguishable from
behavior when the disturbance is not present: it's called CONTROL.

I understand that to mean even if e-coli reversed its direction or went
off at a right angle every 1/60th of a second, there was no difference in
the subject's ability to control.

Is this correct? Or did you do as is usually done in such experiments,
vary the disturbance to the direction gradually so that control was
unaffected? And did you do variant 1 or 2 or both of the experiments
I suggested?

Since I don't have ready access to the report, maybe I will try it myself,
which should gladden your heart. That is to say, if HyperCard is good
enough for the purpose. Or do you have a program you could post, to
allow me to get on with the project I am actually trying to get somewhere
with?

Snowstorm Wednesday, another forecast for Monday. Don't you envy us, you
catching the sun there in LA-la land?

another viewpoint, [that of the true believer in information theory]

Oh, yes. Faith, it's wonderful. Keeps me sane (if you believe that :-).

Martin

PS. Oh, and by the way, before I read your report, and despite what I
understand you to have said:

[The experiment] alters the predictability of the e-coli path, and might or
might not affect the ability of the bar-presser to keep it near the target.

Well what is your prediction? Might or might not?

Not when the disturbance is slow, but increasingly yes as the disturbance
bandwidth increases, the more so the greater the amplitude.

M.

4

[From Bruce Abbott (960325.1040 EST)]

Rick Marken (960324.1500) to Martin Taylor --

I'm afraid that we have a real disagreement; you say that control is
only possible if the results of action are predictable to some extent;
I say that control is possible whether or not the results of action are
predictable.

Rick is right -- the result of an action need not be predictable. If the
(unpredictable) result turns out not to have reduced the error, one only has
to repeat the action until it does.

But Martin is right, too. If e-coli "chooses" a new direction at random on
each iteration, and the result of action is to choose a new direction at
random (rather than have it done automatically on that iteration), it cannot
matter whether the choice is done with or without intervention by the
participant: in either case e-coli wanders off in a new random direction.

On the other hand, if _during the same iteration_ the program first chooses
at random and then the participant can observe that result and, by pressing
at that moment, replace the computer's choice with a new one, this leads to
a biased walk, in that "bad" (error non-reducing) choices are always
overridden by new (_possibly_ better) choices. The biased walk moves the
e-coli, in fits and starts, gradually toward the reference state.

Now if the iteration rate is fast relative to the participant's ability to
perceive e-coli's changes in position and react, the participant will not be
able to control even in this second case. Thus it is necessary to have
e-coli move in a given general direction for at least a short while
following each new change in direction. This could be accomplished by
having the computer choose a new direction at random only after each N
iterations, where N is long enough to take care of the participant's lag, or
by allowing the new random choice to change e-coli's direction by only a
restricted amount (e.g., no more than 2 degrees per iteration).

Although this general result is predictable in that one can compute the
probabilities and show that they necessarily lead to successful control if
the participant behaves as stated, the participant need not predict the
result of any particular action in order to succeed. What is critical is
that the participant have information about the current state of e-coli's
movement relative to target (i.e., change in error) and be able to act on
that information before the information becomes outdated. The disturbance
must fall within the participant's response bandwidth.

Regards,

Bruce

5

[From Bruce Abbott (960325.1855 EST)]

Rick Marken (960325.1300) --

Bruce Abbott (960325.1040 EST)

What is critical is that the participant have information about the current
state of e-coli's movement relative to target (i.e., change in error) and be
able to act on that information before the information becomes outdated.

This is true if the subject is controlling the perception of _movement_
relative to the target. But my revised version of the E.coli demo (now
available as a HyperCard stack called E. coli II) provides no information
about movement relative to the cursor; after a press (action) the dot moves
to a new location and stays there for a while. The press can be based only on
whether or not the dot is on the intended target.

O.K., have it your way. What is critical is that the participant have
information about the current state of e-coli's _position_ relative to the
target following its move (i.e., error) and be able to act on that
information before the information becomes outdated.

The change in task does not change the substance of my argument. How long
does e-coli stay put in the absence of a press? Long enough for the
participant to act on the information (if necessary) before e-coli would
move again on its own, I'll wager.

Regards,

Bruce

6

[From Bruce Abbott (960325.0900 EST)]

Rick Marken (960325.1830) --

Bruce Abbott (960325.1855 EST)

O.K., have it your way. What is critical is that the participant have
information about the current state of e-coli's _position_ relative to the
target following its move (i.e., error) and be able to act on that
information before the information becomes outdated.

Thanks. I would just pick one more nit. The "information about the
current state of e-coli's _position_ relative to the target following
its move" is NOT error. It is just a perception. Error exists if the
perception of e-coli's _position_ relative to the target position
differs from the subjects reference for the value of this variable.

We had, I thought, defined the target position as the participant's
reference. If the participant has defined the target as the reference and
is attempting to control e-coli so as to bring it to target, then the
difference between e-coli's position and the target's position is error.
That is,

e = r - p

But your point is well taken; error is defined relative to whatever the
participant has chosen as reference, regardless of the experimenter's druthers.

And now for my own nit. When p = r, e does not cease to exist, it merely
becomes zero. There is a big difference between no signal and a signal
whose value is currently zero.

Regards,

Bruce

7

[Martin Taylor 960326 14:00]

Rick Marken (960326.0930)

What is predictability? How do I measure it? Once I know how to measure it, I
can measure the predictability of dot position in E. coli II.

Do you want seriously to get into a discussion of uncertainty and information?
If so, I'd be happy to try to get you to a position in which you could
answer this question for yourself. But if it is likely that we would wind
up sinking through the gumbo, clutching a tar-baby, I don't want to.

But I'll suggest a start. Assuming that you are talking about your multi-
circle dot (you can extrapolate to a continuum, but this is easiest):

At time t the position of the dot has been observed. If you "know" that,
the uncertainty is zero. Based on what else you know (from modelling, past
observation,...) you can estimate how likely it is that the dot will
be at location k at time t1 if your action is x (which might be null).
At time (future) t1, you have an imagined probability Pk that the dot
will be in circle k. If Pk is the same for all circles, you have zero
predictability.

Now you say that in e-coli II, the dot moves at some unpredictable time,
and where it goes is equally probably to any of the circles. That means
that _if you know that it has moved_ the predictability will be zero. You
can know that it has moved by waiting (in imagination) an infinite time.
(In a Brownian motion continuous version of e-coli, the probability is
distributed in a Gaussian way with a variance proportional to t1-t0, so
the predictability there is never zero; we'll keep considering only the
case in which it is or it isn't zero, the case you proposed).

Now we get into algebra. Without using the formula for "uncertainty", we
can still get somewhere. From modelling or prior observation our controller
judges that the probability the dot will have jumped after time dt is
J(dt). This necessarily will be a rising function of time. If the dot-jump
time is modelled as a Poisson process (equal probability in every small
increment of time), J(dt) is (I think) an exponential approach to unity
J(dt) = 1 - exp(-rdt) where r is a positive rate constant. (If you want
to check out your e-coli II, you'll have to double check this statement).

Since if we know that if the dot hasn't jumped, it is where it was, and
that if it has jumped we know nothing about where among the circles it
will have gone, we can say that the relative predictability at time dt
after the last observation is
1 * (probability of not having jumped) + 0 * (probability of having jumped)
= 1*(1-J(dt)) + 0*(J(dt))
= exp (-rdt).

Anyway, that's one approach. To do better, you would look instead at the
distribution of probabilities that the dot would be on each of the N circles,
and plug those numbers into the expression for uncertainty. That's a
general approach. But we don't want to get into information theory, do we?

Oh, and be sure to remember that these probabilities are about an imagined
future. They are NOT measured relative frequencies, though they might well
be based on measurements of past event frequencies.

Martin