Control and Predictability

[From Rick Marken (960322.1800)]

Me:

Behavior with the disturbance present is indistinguishable from
behavior when the disturbance is not present: it's called CONTROL.

Martin Taylor (960322 17:45) --

I understand that to mean even if e-coli reversed its direction or went
off at a right angle every 1/60th of a second, there was no difference in
the subject's ability to control.

Basically true. The disturbance in our E. coli study varied slowly; if
it had varied more quickly (higher amplitude) control (measured as
variance of dot position) would, indeed, have deteriorated but it would
have still existed. My point above was that with or without disturbance,
the subject is able to control the position of the dot.

The poorer performance that results when there is a bigger disturbance
has nothing to do with predictability; the same deterioration of
performance happens in any control task -- even one with a regular
(predictable) disturbance; the greater the amplitude of the disturbance,
the poorer the control.

The important point of the E. coli demo is that control has nothing to
do with predictability. Perhaps this would be clearer if you considered
a version of the E. coli program that I wrote when I still believed that
it might actually be possible to reject reinforcement theory. In that
version of E. coli the dot simply jumped to a new, random position after
each press -- and stayed there until the next press. So the result of a
press was a completely unpredictable new dot position. Nevertheless, the
subject can press until the dot ends up at the target; once it ends up at
the target, the subject stops pressing.

The fact that the subject is _controlling_ the dot is evidenced by the
fact that the dot ends up at the target (and remains there) on _every_
repetition of the experiment -- and this, despite the fact that the
consequence of each press during each experiment is completely random.

Predictability (of the results of action) is _not_ necessary for
control. This is surprising, I admit; but when your dealing with
closed loop systems you get used to surprises;-)

Best

Rick

[Martin Taylor 960323 16:30]

Rick Marken (960322.1800)

Something strange is happening here, and I don't know whether it is
with the language or not.

Me:

Behavior with the disturbance present is indistinguishable from
behavior when the disturbance is not present: it's called CONTROL.

Martin Taylor (960322 17:45) --

I understand that to mean even if e-coli reversed its direction or went
off at a right angle every 1/60th of a second, there was no difference in
the subject's ability to control.

Basically true. The disturbance in our E. coli study varied slowly; if
it had varied more quickly (higher amplitude) control (measured as
variance of dot position) would, indeed, have deteriorated but it would
have still existed. My point above was that with or without disturbance,
the subject is able to control the position of the dot.

One language problem: "quicker" is not the same as "higher amplitude."
Higher amplitude does mean a faster slew rate, but quicker means "with
higher frequency variation", at least as I used it in the post you were
answering.

Now, you say that if you had done the experiment with random increments
to the e-coli velocity every 1/60th of a second, there would have been
no difference in behaviour. I found that so strange a notion that I DID
go home and program the experiment in HyperCard (I'll send you an
unsolicited copy). Here are some preliminary results.

The situation is this: in the HyperCard window, there is a small x-shaped
"target" and a square box "cursor". The cursor is the e-coli, and the
target doesn't move. Initially, the cursor is set to move at some
velocity (n pixels in x and m pixels in y on each compute iteration,
which turns out to be around 600 msec on my Mac IIsi). When the mouse
is clicked, m and n are chosen anew from a range of +-k, where k is a
setup parameter (I used k=3 throughout). This, I think, is the classic
e-coli condition. The cursor moves in a straight line at a constant (though
jerky) speed until the mouse is clicked, and then moves off in a new
direction (or might even stay stationary if m = n = 0, but this special
case is actually forbidden in the code).

I define being "on target" as any time the cursor is within 10 pixels of
the target. With this definition, when I started doing the experiment
with myself as subject (and so far the only subject), I was keeping
on target about 80% of the time after the first time the cursor reached
the target. After some training, I was managing 100% for the 2-minute
runs, except for one run when I was getting persistently "bad" random
numbers that always drove the cursor in the same direction.

Now we add a disturbance. The way I do this is to add an increment of p
pixels/iteration to the x velocity, and q pixels/iteration to the y velocity,
where p and q are drawn from a range +-h, h being constrained to be less
than or equl to k. I used h=2 throughout. What I varied was how often
this increment was added. The addition is quite independent of anything
done with the mouse, but when the mouse is clicked, the velocity distribution
is once again the same as it would have been without the disturbance, in
other words, m and n are chosen anew from the range +-k.

The results show a clear effect of the disturbance rate. I started with
one increment per second, and learned to bring the "on target" score up
from an initial value of around 0.4 (boy, was it difficult!!) to about 0.7
late in learning. I then tried a couple of other disturbance rates
intermixed with no disturbance and the 1/sec rate. At 0.5 increments/sec
I could maintain an "on target" score around 0.8. I tried to get a
faster rate of 3 increments per 2 seconds, but I'm not sure I achieved
it with the slow computer. At any rate, when I tried that, I was able
to get to about 0.6.

Subjectively, it is MUCH harder to control when these disturbances are
added to the e-coli velocity, even when they are added infrequently. I'd
be really interested to see your data from your study, because if you
didn't find that, there must be something very different between your
formal experiment and my little trial.

However, I must say that my prediction guess was wrong. I thought that I
would find no effect of the disturbance once I had got on target, and that
it would take longer to get on target. The results are that the time
to get on target fluctuates really wildly for all the conditions, and
you can't tell between them. But after the first "hit" on the target,
the proportion of time that the cursor stays on target is reasonably
consistent across runs for any given condition (with one or two outliers
as I mentioned, when the run of random numbers is "bad"). And it is
strongly dependent on the disturbance rate.

I'll send a copy of the stack to you separately, after I've cleaned it
up and added instructions etc.

Indisputably there is control under all conditions, but that was never an
issue. The issue was whether the quality of control varied with the
predictability of the e-coli course after an unpredictable course change
generated by the controller person. You said it didn't in your experiment,
but in mine it does. We need to find out why the difference.

Martin