[From Bruce Abbott (981020.2200 EST)]
Bill Powers (981020.1340 MDT)
Bruce Abbott (981020.1125 EST)
Take, for example, the usual human tracking study. If the
person does as he or she is instructed, that is, keeps the cursor over
target most of the time despite the disturbances to cursor (or target)
position, then by definition the person is controlling the position of the
cursor. If the person is unable to do so, then by definition the person
isn't controlling the position of the cursor.
That really simplifies the modeling problem: the person either controls or
doesn't control. One degree of freedom, two values, True or False. I guess
I must have been doing it the long way around, by finding the integration
constant, the delay, and the decay factor. If I'd known it was that easy I
could have done this decades ago.
When you think a remark is so stupid that it deserves a reply driping in
sarcasm, it probably isn't. What did you miss?
Actually, it's even simpler than that. If you want to get a person to
juggle three balls and tapdance while whistling Dixie, withhold food for a
while, then give the person a little bit of it every time he comes closer
to doing these three things at the same time. How can he do all those
things? Easy: he was rewarded for doing them. What if he doesn't do them?
Probably the history of reinforcement made it impossible. Discard that
subject and find one more susceptible to reinforcement.
Ouch, I must have _really_ hit a nerve.
Tell me that the person is
controlling the position of the cursor and I will tell you what the person
_must_ do (there's no alternatives) to accomplish that.
No, you won't. You're forgetting something, aren't you?
Yes, I will, and no, I'm not. To move the cursor the person must move the
mouse. As the cursor drifts left of target, the person must move the cursor
to the right fast enough to offset the drift and return the cursor to
target. As the cursor drifts right of target, the person must move the
cursor to the left fast enough to offest the drift and return the cursor to
target.
The much-ballyhooed
predictive accuracy of such an analysis is predicated on the fact that, if
the person is doing the task as directed, then there really isn't any other
way the person could be behaving.
Just for the few readers who aren't already aware that this is nonsense:
What Bruce says is true for a perfect controller in the absence of all
disturbances. But people are not perfect controllers, and in any proper
control-system experiment, the controlled variable is subject to continual
disturbances which neither the experimenter not the controller can predict
because they're generated randomly during the task. The modeling problem is
to create a simulation that behaves in the same way the person does, and
handles unpredictable disturbances in the same way the person does --
mistakes and all.
I'm not assuming that people are perfect controllers; in fact I'm well aware
that they are not. When you set up a task in which you tell a person to
keep the cursor over the target and show the person how to move the mouse to
do so, then _to the extent_ that the person controls the cursor's position
accurately, there are simply zero degrees of freedom in how that will be
accomplished. In fact, for most well-practiced subjects on the tracking
task, performance will be so close to optimal that the improvement in fit
resulting from introducing appropriate integrative lag, etc. is likely to be
small relative to the improvement in predictive ability gained by assuming
that the person does the task perfectly.
Furthermore, a model that does exactly what the instructions say -- keeping
the cursor exactly over the target, for example -- would not match the real
person's behavior nearly as well as a model that has its parameters matched
to the person's actual parameters and thus generates similar tracking
errors. And unless you could predict the disturbances, it couldn't predict
the behavior -- the actions -- at all.
As just noted, a model that does exactly what the instructions say will
already account for most of the variance in mouse movement. That is the
point I was making. I did not and do not dispute that a model with proper
parameters may do slightly better.
As for the disturbances, they are introduced by the experimenter and do not
need to be predicted in order to predict the behavior.
It's like me saying to you, "stand over there." So, you move to the
designated spot. Now I claim that I can predict with high precision your
future location in space.
How, when all you said about location was "over there"? That's "precision?"
I watch where you go and stand. Now precision is a relative thing. It's
extremely precise if your position could have been anywhere in the universe.
Probably a reasonable standard is relative to where you could have gone to
in the time alloted if you had been free to do so. After 30 minutes you
could be miles away, so by comparison, locating your position within inches
is excellent precision.
Heck, I can do even better than that. I can predict that no matter what
happens for the next 30 years (as long as you stay alive), you're going to
be breathing oxygen. If you die, you violated my instruction to stay alive.
Now you're getting the idea. Given the conditional, there's not much choice
in what must be done.
On the other hand, if you show me a model matched to Tom Bourbon's tracking
performance, and give me a new data set for the same task, I can tell you
whether Tom was the tracker, or someone else. If tracking were just
"keeping the cursor over the target," how could I do that? All people would
be alike.
I'm not disputing the improvement gained by fitting proper constants to the
model. What I'm saying is that the extraordinary precision of prediction
claimed is due in large part to the fact that any person doing the task well
couldn't be doing much of anything else (lags, overshoots, etc.); most of
the variance in action could be accounted for simply by assuming that the
person is accomplishing the task as directed, and examing the task to see
what must be done to do so.
I am not, as it may appear, belittling a control-system analysis or its
predictive ability.
No, you're just misrepresenting it. Can it really be that after all this
time you don't understand how the models are matched to the real behavior?
No, it can't really be so. Can it really be that after all the interacting
we've done, all the collaboration on modeling and so on, that you can still
believe that I don't understand how the models are matched to the real
behavior? Shouldn't warning bells be going off that maybe you aren't
understanding what I'm getting at?
The point is, if the subject is performing the task as
directed, he or she has essentially _no_ degrees of freedom, and this
accounts for the predictive accuracy of the analysis.
No, it doesn't. What accounts for the accuracy is finding a model with the
necessary parameters and organization, and then varying the parameters to
find the best fit of the model to the real behavior.
If you know what's involved in performing the task -- what the cv is going
to be, and its means of control, you can model that without knowing anything
about the person, and get very close to a person's performance, if the
person does well on the task. The curve-fitting will improve on this, but
if performance is already near optimum, it won't add much because there
isn't much left to improve on.
What the person is told to "do" is not to produce any specific behavior:
it's to produce a particular perceptual result, by acting in any way that
(invisible) disturbances make necessary. The "predictive accuracy" goes
beyond just saying that the person will do the task; it includes predicting
the exact trajectory of movements for the entire period of the task, and
(as of the latest try using Vensim's matching routines) doing so within
1.8% RMS of the range of handle movments. That is, the range of movements
is 50 times the standard deviation of the mismatch between the model's
movements and those of the real person. Want to calculate the probability
that that match could have occurred by chance?
Is this before or after you employ curve-fitting to find the optimal
parameters? The range of movements is close to the same 50 times the
standard deviation of the mismatch if you assume the the person does as
_required_ by the task -- perfectly. What do you think the probability is
that you will admit that I have a valid point here? (I'm afraid it's
vanishingly small . . .)
But
if your theory can not predict what variables _will_ be controlled, under
what circumstances, then you are left with the job of determining those
facts empirically, on a case-by-case basis. To the extent that this control
is idiosyncratic and subject to moment-by-moment changes in CVs and methods
of CV control, the best you can do is either (a) arrange situations where
what must be controlled and how in order to perform a given task are
severely constrained (e.g., tracking tasks) or (b) resign yourself to merely
explaining observed behavior, after the fact, by identifying what CV must
have been controlled and how.
That makes it sound as if the chance of discovering controlled variables
"in the wild" are just about zero.
. . .
I somehow doubt that
controlled variables are all that hard to find. And once empirically found
(you're right about that), it's not absurd to suppose that the same
controlled variables will be controlled again, or will even continue to be
controlled at similar values, or that we can identify higher levels of
controlled variables that will permit us to predict how reference levels
for lower ones will be set and changed.
I would hope that this would be the case, and in fact it's what I would
expect. Yet in discussions on CSGnet it often sounds as if all a PCT-based
science of behavior can do is predict what a person will do only after we
know what he or she is trying to do, and that this changes constantly both
between and within persons. By the time you painstakingly identify what the
person is controlling here, under this condition, the information is out of
date, because she's gone on to something else.
Oh, I think you can explain it with PCT, if you try. I think the rat is
controlling something affected by food intake -- maybe the food intake
itself. That much should not be hard to verify. Pinning it down more
exactly will require adding disturbances to test each dimension of the
proposed controlled variable, and showing that the rat's actions vary in
the appropriate way. Of course using an experimental regime in which a huge
amount of noise is deliberately inserted into the data makes this job much
harder, and probably limits the precision with which you can identify the
CV. You may end up saying just "the rat tries to maintain its average food
intake."
Actually I don't think there's too much doubt about what the rat is
controlling in the operant chamber by means of lever-pressing, although the
required model is likely to become extremely complex. It presses the lever
in order to produce a food pellet, it produces a food pellet in order to
have food to pick up and consume, and it picks up and consumes food in order
to -- well, this is where the complexity starts to show up. These are going
on regardless of the schedule imposed. But I think VI schedules add
something besides noise; the rat adapts to them and peforms differently than
it does under, for example, ratio schedules.
I guess that whether you think research like this is valuable or not depends
on whether your interests lie in demonstrating that PCT can accurately
describe how some variable is being controlled (_when_ it is being
controlled), or in trying to understand some puzzling bit of behavior for
which an adequate explanation does not yet exist. I'm doing the latter.
We should really be discussing the data, shouldn't we?
Well, yes, but you told me you weren't interested . . .
Regards,
Bruce
) the few