Control of reinforcement

[From Bruce Abbott (981021.1340 EST)]

Bill Powers (981021.0740 MDT)

Bruce Abbott (981020.2200 EST)

When you think a remark is so stupid that it deserves a reply driping in
sarcasm, it probably isn't. What did you miss?

Nothing. Your generalization is not 100% accurate.

Guess again. (Later on you're going to tell me that my point is valid,
hardly deserving of the sarcastic reply it received initially.)

You're still forgetting something. Shall I tell you what it is, or
would you rather figure it out for yourself?

I'm not assuming that people are perfect controllers; in fact I'm well aware
that they are not. When you set up a task in which you tell a person to
keep the cursor over the target and show the person how to move the mouse to
do so, then _to the extent_ that the person controls the cursor's position
accurately, there are simply zero degrees of freedom in how that will be
accomplished.

Oh, well -- that wasn't it. What you're forgetting is that added to the
effect of the mouse on the cursor is a LARGE disturbance derived from a
random-number generator. Sometimes moving the cursor to the right requires
a mouse movement to the right, sometimes a movement to the left, and
sometimes no movement, depending on how the disturbance is changing. If you
look at the pattern of mouse movement you will find essentially no
relationship to the pattern of cursor movement or target movement. So how
are you going to tell a person "how to move the mouse?"

I can't think of a case in which "moving the cursor to the right requires a
movement to the left" -- can you? One might do that to _slow_ its movement
to the right (which suggests that subjects may use rate as well as
positional control; is that in your model?). There's no need to help the
cursor along if it's going where you want it to go on its own. All a person
need do to accomplish the task is watch the cursor, and adjust the direction
and rate of mouse movement so as to null out the cursor's motion (and if
necessary bring it back to target).

As just noted, a model that does exactly what the instructions say will
already account for most of the variance in mouse movement. That is the
point I was making. I did not and do not dispute that a model with proper
parameters may do slightly better.

"Slightly better" = RMS prediction error of 10% reduced to 2%.

Let's not quibble over loosely quantitative statements. Slightly better
compared to RMS prediction error of 99% absent knowledge of the task
requirements (i.e., randomly guessing where the cursor will be at any given
moment). 10%--->2% = 8% further improvement only slightly better compared
to 99%--->10% = 89% improvement.

As for the disturbances, they are introduced by the experimenter and do not
need to be predicted in order to predict the behavior.

In order to predict the _outcome_, cursor over target. There is no way to
predict where the mouse will move. The experimenter does not determine or
know in advance what disturbance pattern will be applied to the live
person; that is left up to a random number generator.

The model has to be given the same disturbances in order to predict what the
ideal control system _must_ do to keep the cursor over target. Thus, the
experimenter knows what disturbances will be, he or she can predict what
movements of the mouse must occur in the ideal case. The subject doesn't
have any choice but to move the mouse in the same way if he or she is to
control perfectly; it is what the task demands.

Heck, I can do even better than that. I can predict that no matter what
happens for the next 30 years (as long as you stay alive), you're going to
be breathing oxygen. If you die, you violated my instruction to stay alive.

Now you're getting the idea. Given the conditional, there's not much choice
in what must be done.

If you pick the right conditional, you can make your prediction as trivial
as you please. But why make trivial predictions?

I think you're misunderstanding me. I'm not saying that the details needed
to get the model to match the person's _mistakes_ are unimportant; in fact I
think they are central to the enterprise. But when you ask a person to
control a cursor position, and they do, most of the accuracy of prediction
comes from the fact that there just isn't any other way to successfully
complete the task given. You can explain 90% of the variance in mouse
position without knowing anything about the person; just analyze the task
requirements.

The extraordinaty precision is largely due, I will admit, to the mere
realization that the person is organized as a good tight negative feedback
control system. You're saying that if a person is organized as a good
negative feedback control system (the only way this task could possible be
done well, or at all), then assuming perfect control will predict the
results almost as well as measuring the actual parameters of control which
are somewhat less than optimum. That says no more than that this person is
acting as an almost perfect control system. What the instructions actually
say is "be a control system." But isn't the whole point to explain how that
could be done -- HOW the instructions could possibly be followed?

If the person performs the task well, then the person has organized himself
or herself as a good negative feedback control system. It shows that a
person _can_ do that, if need be. If a task demands that a person rapidly
and accurately do mental arithmetic, and the person succeeds in doing the
task well, it shows that the person _can_ organize himself or herself as a
good $4.00 calculator and, given that, I can predict what number the person
will announce as the answer to a particular problem by computing the correct
answer to the problem -- accounting for 98% of the variance in numbers
announced. (I'm assuming that the person makes one or two mistakes.)

The point is, if the subject is performing the task as
directed, he or she has essentially _no_ degrees of freedom, and this
accounts for the predictive accuracy of the analysis.

This clearly shows the point you want to make, but your concept of
accounting for something is pretty weird. What PCT explains is how the
subject could be performing the task as directed. The directions describe
only what outcome is to be produced. They don't explain how, given the
circumstances, it is possible for any organism to produce such outcomes.
For that you need a control system model; no other model, reinforcement or
otherwise, can explain it.

Of course. If the task requires that you behave as a control system and you
succeed at it, then a control system model will be required to explain how
you did it. A good reinforcement model _might_ succeed at explaining how
you came to be organized that way, I don't know. Control theory might also,
but that would require another model to explain the existence of the control
system that explains the behavior. Now _that_ would be impressive. I don't
think that random reorganization would be sufficient, in most cases.

If you know what's involved in performing the task -- what the cv is going
to be, and its means of control, you can model that without knowing anything
about the person, and get very close to a person's performance, if the
person does well on the task.

Tnat is to say, if you know the subject is organized as a good control
system with an integrating output function, you can predict that the person
will control well. Does that sound like a great relevation to you?

I looks rather obvious to me. The important question is, what is the actual
nature of that system? The actual system may be very different from the
simple control model that explains 98% of the variance in the person's
actions, and yet behave nearly identically in the test situation.

The range of movements is close to the same 50 times the
standard deviation of the mismatch if you assume the the person does as
_required_ by the task -- perfectly. What do you think the probability is
that you will admit that I have a valid point here? (I'm afraid it's
vanishingly small . . .)

Your statement is correct, and trivial.

Well, I'm making progress. My point has moved from being absurdly incorrect
to correct but trivial.

Of course if the organism is a very
good control system, it will control in nearly the same way that a perfect
control system will control. One of the main points of these experiments is
to verify that the organism is, indeed, acting as a negative feedback
control system rather than some other kind of system.

My point is that if the task is such that no other kind of system can do it,
then there is no need for such a test, except to determine whether the
person can do the task. If the person can do the task, it's the only kind
of system he or she can be acting as.

But once that
has been established, a more advanced consideration is to measure the
parameters of control, to see how one person differs from another and how
stable the parameters are over time. The same model we get from these
experiments can be used to predict behavior when control is not so good --
when the disturbances are larger and faster, so the tracking errors are
much larger. And they can predict what will happen to control accuracy when
the situation is changed -- when, for example, the proportionality constant
in the environmental feedback function is doubled or halved, or when the
EFF is changed from a proportional relationship to one involving one or two
integrations (as in controlling the position of a mass on a spring with
damping, or flying an airplane). One question of great interest is whether
the parameters of control remain the same when the EFF changes, or if they
change, and if they change, with what gain, delay, and integration factor
the changes take place. If all you know is that the instructions were
followed, there's no way to answer such questions.

I've never disputed that. That's why I said that I was not belittling
control theory or its predictive accuracy, and it's why Rick Marken's
example of the fielder is no challenge to my position. In those situations
there are many possibilities as to what is actually controlled, and how.
But PCT doesn't predict which variables are under control in those
situations (although one can come up with good hypotheses by considering the
task requirements and the actual performances of the individuals (e.g.,
fielders) in question). It can predict only conditionally -- _if_ the
subject is controlling X, then the model predicts such and such. I don't
see this as a shortcoming of PCT (there's enough to do for now just figuring
out how the system must be organized given that it _is_ controlling X) so
much as an area in much need of future development. But it's prediction
under severe constraint, and that accounts for most of the accuracy of the
predictions.

But I think VI schedules add
something besides noise; the rat adapts to them and peforms differently than
it does under, for example, ratio schedules.

You'll have to prove that. It remains possible that the rats are organized
in exactly the same way they are under all other schedules, and it is the
random variations in interval that create apparent differences in the rats'
behavior.

The rat is the same rat under all schedules, but I suspect that it learns
things when exposed to one schedule that it cannot learn when exposed to
another, and to that extent becomes "organized" differently. I'm just
starting to think about how to model what I suspect might be going on.

Regards,

Bruce

[From Bruce Gregory (981021.1512)]

Bruce Abbott (981021.1340 EST)

>Bill Powers (981021.0740 MDT)

My point is that if the task is such that no other kind of system
can do it,
then there is no need for such a test, except to determine whether the
person can do the task. If the person can do the task, it's the only kind
of system he or she can be acting as.

If I understand the discussion here, Bruce is arguing that earlier studies
(such as "Models and Their Worlds" by Bourbon and Powers and Mimicry,
Repetition, and Perceptual Control" by Bourbon), demonstrate that in order
to accomplish certain tracking tasks, people _must_ be organized as control
systems. The fact that the tasks _can_ be accomplished therefore allows us
to
conclude that the individuals who accomplish them _are_ organized as control
systems. What remains a puzzle for Bruce, however, is _how_ people get to be
organized in this way. Bill has argued that the simplest answer to this
question involves random reorganization. Bruce is not persuaded that this
mechanism is adequate. Of course, it is possible that the organization has
been developed, not in the course of the task, but over the lifetime of the
organism. In this case, what we might be seeing is time it takes a higher
order system to determine the reference levels for lower order systems
needed to improve the control system performance of this particular task.

From the Enemies List,

Bruce Gregory

[From Chris Cherpas (981022.0420 PT)]

Bruce Abbott (981020.1125 EST)--

Are you suggesting that the rat's behavior on these
schedules has nothing to do with the control of
perception?

I believe we are both assuming that behavior is
the control of perception.

I would find it convenient if the first PCT rat
studies used procedures and measures that were
closer to those used in the experimental base
already established in PCT studies, than those
used for the purpose of studying the supposed
phenomenon of reinforcement.

We talked about this over a year ago, but I don't
recall any major breakthroughs in technology. It
would certainly simplify things, from my point of
view, if the rat could vary, say, how loud a tone
was by operating some sensitive analog device. I
think that once the rat could learn to control
some related variable by varying this, a whole
developmental program could be built on this
foundation. Easier said than done. I would expect
that a graded set of disturbance patterns might
be useful to establish good control at this first
level too.

At least there are precedents in JEAB (under the
aegis of "continuous repertoires") that would
provide some hints about how to procede or what
to avoid. Of course, I'm a pigeon person myself
(although I've got a nice rat-bite scar that
might lend a little credibility to my rap),
so I haven't got as much experience in outfitting
a rat chamber. I'm definitely not sanguine
about that bar-press. How could you get a
rat to move a mouse? Let's say you had two
bars so close together that the rat could
hold down one to move some variable in one
direction (e.g., louder) and the other bar
to move it in the other direction (e.g., softer),
without having to run back and forth. It
seems possible to have the bars set up so that
if both were pressed simulateously, the
relative displacement of the bars would determine
the net effect. Certainly bars have been set
up with force requirements that I imagine made
their operation smoother than the usual "bang-
bang" bar.

At least one could go with duration rather than
these discrete clicks. As a last resort, I imagine
something could be done as with the sliders along
the side of a Windows window, where there is a
clickable arrow-button at each end. Perhaps the
main thing is: can rat control a variable fast,
relatively continuously? The methodology of
establishing a steady state for a couple hundred
years and then changing a parameter, and then
back another century later, etc. is not only
counterintuitive for understanding control, it's
boring. You kill the dynamics and get stuck
with huge barrels of data that have to be
sifted through to find the elusive relation
that might otherwise be snagged in a jiffy.
Skinner even bitched about this when people
started getting so carried away with quantitatively
analyzing thousands of bar-presses, they
forgot to notice the cumulative recorder,
the rat, and the grad slave running the
study were nodding off to sleep. I'm all for
multiple sessions to replicate control, but
let's see the control WITHIN THE SESSION.
Lingering too long in any condition and trudging
through the drill of multiple-session ABAs gets
to be baroque. We're talking "living" control
systems here, so let's "lively up ourselves"
like Bob Marley (alright he's dead, but his
music lives on, and I hope you get the point).

You may recall that Jack Findley, inventor of
the famous Findley-key (changeover) procedure
for concurrent schedules, was not considered
the most "stable" person publishing in JEAB,
but his 1962 (I think) proposal for a program
of developing multi-operant repertoires had
some merit. A research program in which rats
built up a sophisticated repertoire of control
systems would be a riot. Of course, Findley's
program went nowhere, but Sidman's matching-to-
sample, equivalence-class program is not so
different from what I am suggesting, and
shows something of what may be possible, if
not with rats, then with sub-optimal humans.
By the way, I wonder if Sidman regrets the
extent to which EAB folks have insisted on the
steady state comparison methodology, regardless
of its appropriateness for a given purpose,
because they are using his methodology book
as a recipe, instead of as the advice of an
old pro exemplifying principles.

Best regards,
cc

[From Bruce Abbott (981022.0910 EST)]

Chris Cherpas (981022.0420 PT) --

I believe we are both assuming that behavior is
the control of perception.

Yes.

I would find it convenient if the first PCT rat
studies used procedures and measures that were
closer to those used in the experimental base
already established in PCT studies, than those
used for the purpose of studying the supposed
phenomenon of reinforcement.

I agree.

We talked about this over a year ago, but I don't
recall any major breakthroughs in technology. It
would certainly simplify things, from my point of
view, if the rat could vary, say, how loud a tone
was by operating some sensitive analog device. I
think that once the rat could learn to control
some related variable by varying this, a whole
developmental program could be built on this
foundation. Easier said than done. I would expect
that a graded set of disturbance patterns might
be useful to establish good control at this first
level too.

I don't think the technical problems are too difficult; it's mainly a matter
of my finding the time and or $$$ to acquire a proper set-up, and finding
out what sort of continuously variable device the rat can operate smoothly
(in addition to its own muscles). I have a couple of A-to-D boards, one
kindly donated by Bill P. A force-transducer would be nice, but a simple
potentiometer would probably do the trick with the right kind of attachment
for the rat to operate.

But the main problem is finding some cv the rat has an interest in
controlling. I mentioned ambient temperature -- there have been studies
involving more primitive organisms in which a temperature gradient was
established from one end of a long, narrow box to the other. The critters
would move to that portion of the box where presumably they were most
comfortable. The position of this "comfort zone" could be varied
experimentally and the critters moved with it, if I recall correctly. (My
memory on this is rather dim; I don't recall what sort of animals they were
-- bugs of some sort, I think.)

I've already tried using food intake as the action; lever-pressing was
simply a convenient way for me to record when and how much the rat ate, at
what rate (CRF schedule). I varied the amout of supplemental feeding given
outside the session as a disturbance to whatever-it-is that food intake
serves to control, but ran into some problems which apparently have to do
with short-term satiety. (The rats would stop eating before they'd eaten
enough to compensate for the disturbance.) I've acquired a food trough with
a photocell that could be used to dispense with the lever altogether -- the
rat just pokes its head into the recess and this interrupts the beam. But I
think the main problem here is that control over nutrient level is a
complex, multidimentional process that does not lend itself to a simple
analysis based on a simple manipulation such as I performed. There were
apparently several factors at work inside the rat that I was unable to
measure with the resources available to me. That makes it less than ideal
as a first step in demonstrating the power of control theory in accounting
for the data.

So I've been thinking about doing something more along the lines of a
tracking study, in which the rat would track the "target" simply by
following it with its body. Another possibility, more along the lines you
suggest, would be to provide some analog device with which the rat could
control the frequency of a tone. It would have to receive some sort of
reward for doing so, as a way of motivating the rat to keep the tone at the
target frequency. (I'm thinking that this target frequency would serve as a
correlated stimulus in the presence of which food would be occasionally
delivered.)

If you have any better suggestions I would sure appreciate hearing them.

Regards,

Bruce

[From Chris Cherpas (981022.0420 PT)]

Chris Cherpas (981022.0420 PT)--

At least one could go with duration rather than
these discrete clicks.

Rick, Bill, have you done any tracking
experiments/demos in which the left and
right mouse buttons were used, instead
of the track ball, such that holding down
left would move the handle up and
holding down right would move it down?
Seems crude relative to the track ball
(or joy stick), but I wonder what the
how it bad it is at different speeds...

I'm all for multiple sessions to replicate
control,...

...if necessary, for a given purpose.

Regards,
cc

If you'll forgive a layman in these matters making a suggestion, one
obvious continuously variable parameter you could let the rats control
is light level. Provide a lever that they can push, directly connected
to a potentiometer determining the current through the lights. You
could even have the lever connected to a household dimmer. My own
impulse would be to first just provide them with this setup and see what
they do with it.

Disturbances to light level are easily made.

Of course, you might find that they prefer to hide from the experimenter
by keeping the lights turned off.

-- Richard Kennaway, jrk@sys.uea.ac.uk, http://www.sys.uea.ac.uk/~jrk/
   School of Information Systems, Univ. of East Anglia, Norwich, U.K.

[From Bruce Abbott (981022.1255 EST)]

Richard Kennaway --

If you'll forgive a layman in these matters making a suggestion, one
obvious continuously variable parameter you could let the rats control
is light level. Provide a lever that they can push, directly connected
to a potentiometer determining the current through the lights. You
could even have the lever connected to a household dimmer. My own
impulse would be to first just provide them with this setup and see what
they do with it.

Disturbances to light level are easily made.

Of course, you might find that they prefer to hide from the experimenter
by keeping the lights turned off.

Excellent suggestion, Richard. Rats are nocturnal animals and are known to
prefer very low levels of illumination (experimenter present or not). If
they can learn the relation between lever-position and brightness, they will
probably act to keep the lights in the chamber relatively dim.

Regards,

Bruce

[From Chris Cherpas (981022.1111 PT)]

Bruce Abbott (981022.0910 EST)--

But the main problem is finding some cv the
rat has an interest in controlling...

Have you considered a continuously varying
quantity which is correlated with the amount
of time to food delivery, in other words,
a "clock?" My first take on it would be to
use a conjunctive schedule in which the clock
progresses towards the dinner bell as long
as the rat keeps some quantity within some
upper and lower bounds around a target
value, and in fact, it could continuously
vary with distance to the target value, but
I imagine you will need some layers of
contingencies working conjunctively that
use upper/lower bounds, a continuous
gradient (maybe not arithmetic/interval
scale), a set of graduated patterns of
disturbances that the program can switch
between, with parameters around the
time-to-food interval, the upper/lower
tolerances, and so forth.

The program may well evolve to a
probabilistic model-based control system,
and, of course, this approach is what I
take in domain of computer-based instruction
for humans. One may argue that all this
heavy machinery is unnecessary or bound
to complicate, rather than simplify, the
research, but, as Shimp noted long ago,
the simplicity of the standard reinforcement
schedules may be apparent to the programmer,
while not at all representing contingencies
which expose fundamental properties of the
organism.

Best regards,
cc

[From Bill Powers (981022.1055)]

Bruce Abbott (981021.1340 EST)--

I can't think of a case in which "moving the cursor to the right requires a
movement to the left" -- can you?

Certainly. So could you if you would remember that there is a disturbance
adding to the handle effect, and that this disturbance might be moving the
cursor the same way the target is moving, only faster. To keep the cursor
from overshooting the target you often have to move the mouse opposite to
the direction the target is moving.

One might do that to _slow_ its movement
to the right (which suggests that subjects may use rate as well as
positional control; is that in your model?).

Rate control does not improve the fit. So no, it's not in the model. If
you're asking if we thought of that, the answer is yes. Twenty-five years
ago, more or less.

There's no need to help the
cursor along if it's going where you want it to go on its own. All a person
need do to accomplish the task is watch the cursor, and adjust the direction
and rate of mouse movement so as to null out the cursor's motion (and if
necessary bring it back to target).

But the cursor could be moving MORE than you want it to move, in which case
to reduce its movement to the proper amount you must move the mouse in the
opposite direction. If, in your analysis, you could see that the
disturbance might move the cursor in the direction and at the speed you
want it to move, how could you miss seeing that it might move the cursor
_faster_ than you want it to move?

As just noted, a model that does exactly what the instructions say will
already account for most of the variance in mouse movement. That is the
point I was making. I did not and do not dispute that a model with proper
parameters may do slightly better.

This is a REALLY silly argument, Bruce. You are focusing on the fact that
the subject, or model, actually accomplishes the result requested by the
experimenter, and missing the biggest question, which is _how does the
system have to be organized to do that?_ To say that the task is requested
or assigned or demanded or required tells us nothing about how any organism
can do it. The whole point of PCT is to explain how such tasks can be
carried out, if they are carried out.

Slightly better
compared to RMS prediction error of 99% absent knowledge of the task
requirements (i.e., randomly guessing where the cursor will be at any given
moment). 10%--->2% = 8% further improvement only slightly better compared
to 99%--->10% = 89% improvement.

Look, if all you want to do is find a strained interpretation that makes
the model look at bad as possible, the only thing you're proving is that
you're hostile and willing to go to any lengths to prove yourself right. We
do not make predictions absent knowledge of task requirements. Neither does
anyone else who isn't an idiot.

As for the disturbances, they are introduced by the experimenter and do

not need to be predicted in order to predict the behavior.

In order to predict the _outcome_, cursor over target. There is no way to
predict where the mouse will move. The experimenter does not determine or
know in advance what disturbance pattern will be applied to the live
person; that is left up to a random number generator.

The model has to be given the same disturbances in order to predict what the
ideal control system _must_ do to keep the cursor over target.

Correct. But there is no change necessary in the model that depends on what
the pattern proves to be. The experimenter can set up the model to
reproduce the subject's behavior for _any_ unknown future pattern of
disturbance within reasonable limits.

Thus, the
experimenter knows what disturbances will be, he or she can predict what
movements of the mouse must occur in the ideal case.

But the model is not altered in any way that depends on the pattern of the
disturbance. If I set the model's delay to 0.16 seconds, its integration
factor to 0.15, and its leakage factor to 0.0, it will track close to the
way I do for any disturbance with the same degree of difficulty, within 3
or 4 percent RMS. I don't have to know in advance what the pattern will be.

I'm discovering, using Vensim, that there are better values for these
parameters. But I still don't have to know what the next pattern of
disturbance will be to predict what the outcome will be. I simply predict,
using the model, that the model's handle movements will like the subject's,
given the same disturbance, no matter what the disturbance pattern is, as
long as these parameters are used. With different parameters, the fit will
be worse.

The subject doesn't
have any choice but to move the mouse in the same way if he or she is to
control perfectly; it is what the task demands.

The subject can move the mouse in any way whatsoever. The task does not
"demand" _anything_, least of all that the subject accomplish it. The
subject is perfectly free to say "screw the demands of the task" and to use
the mouse to make the cursor dance patterns around the target. The only
thing that determines that the cursor shall remain near the target is that
the subject decides to honor the experimenter's request to make it do that
-- or decides independently to do so.

I think you're misunderstanding me. I'm not saying that the details needed
to get the model to match the person's _mistakes_ are unimportant; in fact I
think they are central to the enterprise. But when you ask a person to
control a cursor position, and they do, most of the accuracy of prediction
comes from the fact that there just isn't any other way to successfully
complete the task given.

Yes, but the central fact there is that the person IS ACTING AS A GOOD
NEGATIVE FEEDBACK CONTROL SYSTEM. The whole point is to explain how the
person could possibly carry out the assigned task; saying that he does it
just because the task was assigned (or because he was rewarded for doing
it) is no explanation at all. That's like explaining how you do mathematics
by saying you were rewarded for doing it. I can assign you tasks which you
would find it impossible to carry out. If you do a task, the modeling
problem is HOW you can possibly do it.

No other kind of system that I know about could possibly "complete the task
given". Specifically, a system which responds to discriminating variables
to produce actions that have been reinforced cannot "complete" this task
(that is, keep carrying it out indefinitely). To say that the person is
acting as a good control system is to say that the loop gain is high and
the delays are short enough that the system is stable. So your ballpark
guess is really a guess at the parameters of control.

You can explain 90% of the variance in mouse
position without knowing anything about the person; just analyze the task
requirements.

Are you saying that the task requirements are sufficient to explain how (or
predict whether) a person will do the task? If you know that the task
requires solving a third-order differential equation, does that explain how
you, Bruce, will do the task, or predict that you will do it? All the task
requirements tell you is what outcomes the person will be producing IF the
task is carried out. The requirements don't predict THAT the task will be
done, or HOW it will be done. If I tell you the task is to put a key in a
keyhole, I can judge whether the task was done by looking to see if the key
is in the keyhole. Is that what you mean? But if the key is in the keyhole,
does that tell you ANYTHING about how it got there -- even who put it there?

If the person performs the task well, then the person has organized himself
or herself as a good negative feedback control system. It shows that a
person _can_ do that, if need be. If a task demands that a person rapidly
and accurately do mental arithmetic, and the person succeeds in doing the
task well, it shows that the person _can_ organize himself or herself as a
good $4.00 calculator and, given that, I can predict what number the person
will announce as the answer to a particular problem by computing the correct
answer to the problem -- accounting for 98% of the variance in numbers
announced. (I'm assuming that the person makes one or two mistakes.)

And that you make none.

......

Of course. If the task requires that you behave as a control system and you
succeed at it, then a control system model will be required to explain how
you did it. A good reinforcement model _might_ succeed at explaining how
you came to be organized that way, I don't know.

No, a reinforcement model would tell you nothing about the organization
needed to be a control system. It would tell you only that reinforcements
occurred and that (you assume) they somehow brought about whatever unnamed
kind of organization is needed to do the task. As to what kind of
organization that is, reinforcement theory tells you NOTHING. The same is
true of reorganization theory. You may be able to do mental arithmetic, but
you have no idea how you do it.

Control theory might also,
but that would require another model to explain the existence of the control
system that explains the behavior.

It's called reorganization, but reorganization theory doesn't explain the
organization that finally results. Anyway, you don't have to know how the
control system came into being to prove that it exists.

Now _that_ would be impressive. I don't
think that random reorganization would be sufficient, in most cases.

I don't think you know enough about control theory to say what is and what
is not impressive about its ability to explain phenomena. I don't think you
have any model of learning that works better than random reorganization. Do
you even know what random reorganization can accomplish by way of creating
new organizations? If you do, you know more than I do.
..............

Tnat is to say, if you know the subject is organized as a good control
system with an integrating output function, you can predict that the person
will control well. Does that sound like a great relevation to you?

It looks rather obvious to me. The important question is, what is the
actual nature of that system? The actual system may be very different

from >the simple control model that explains 98% of the variance in the
person's

actions, and yet behave nearly identically in the test situation.

The more you know about the system, the more likely it is that your model
will be the correct explanation of its behavior, or a simple transformation
of the correct explanation. In the Little Man model, it is highly likely
that the actual control systems match the major features of the model --
many of the circuits have actually been traced, and the muscle model works
in accord with experimental findings from physiology. The sensors are the
sensors that are known to exist, and they are connected as they are known
to be connected. The physical properties of the arm are very close to
reality as accepted in physics.

There is comparatively little guesswork in the Little Man model.
Hypothetically, and speaking only from general experience, one can say that
the Little Man model MIGHT be very different from the actual organization
of these control systems. But it isn't.

The range of movements is close to the same 50 times the
standard deviation of the mismatch if you assume the the person does as
_required_ by the task -- perfectly. What do you think the probability is
that you will admit that I have a valid point here? (I'm afraid it's
vanishingly small . . .)

Your statement is correct, and trivial.

Well, I'm making progress. My point has moved from being absurdly incorrect
to correct but trivial.

I would not be very happy with that kind of progress.

Of course if the organism is a very
good control system, it will control in nearly the same way that a perfect
control system will control. One of the main points of these experiments is
to verify that the organism is, indeed, acting as a negative feedback
control system rather than some other kind of system.

My point is that if the task is such that no other kind of system can do it,
then there is no need for such a test, except to determine whether the
person can do the task.

That IS the Test.

If the person can do the task, it's the only kind
of system he or she can be acting as.

That's right, in a peculiar sort of backwards way. Finding "the only kind
of system he or she can be acting as" seems to be a pretty important
accomplishment, and indeed that is what the Test is designed for. You seem
to think that if a task requires a certain kind of organization to exist,
that kind of organization magically exists, and furthermore, we
automatically know what it is. The whole point of the Test is to see
whether the behavior that is going on is the behavior of A CONTROL SYSTEM.
That requires analyzing what kind of task is being carried out. What we
expect to find, if it's a control system, is that a variable is being
protected by variations in the system's actions against independent
disturbances. We also expect to find out how this is being done. Your
statement above is perfectly correct, but correctly identifying the task is
tantamount to saying that a control system is doing it. You can't separate
the task from the type of system. Control tasks are what control systems do.

it's why Rick Marken's
example of the fielder is no challenge to my position. In those situations
there are many possibilities as to what is actually controlled, and how.

There are many _hypotheses_ you can form. But if you do the Test properly,
you will eliminate all but one of them (if you eliminate them all, you are
left without an explanation; if more than one survives, you are not
finished with the Test). The real system is doing only one thing, and there
is only one correct explanation whether you find it or not.

But PCT doesn't predict which variables are under control in those
situations (although one can come up with good hypotheses by considering the
task requirements and the actual performances of the individuals (e.g.,
fielders) in question).

To complain that control theory doesn't predict which variables will be
controlled is like complaining that Newton's laws don't predict which
objects will be accelerated or in what direction. Do you think that
Newton's laws, which apply to ALL objects, should be able to predict such
things? If you agree that such a demand would be absurd, then isn't it just
as absurd to demand that control theory, which applies to ALL control
situations, should predict which particular variables will be controlled by
a given organism at a given time?

... It can predict only conditionally -- _if_ the
subject is controlling X, then the model predicts such and such. I don't
see this as a shortcoming of PCT (there's enough to do for now just figuring
out how the system must be organized given that it _is_ controlling X) so
much as an area in much need of future development. But it's prediction
under severe constraint, and that accounts for most of the accuracy of the
predictions.

If the idea of "being a good control system" (loop gain high, stability
good) accounts for most of the accuracy of the predictions, the model is
largely vindicated. To say the subject is controlling X automatically
implies a certain range of the parameters of control. One can narrow that
range with further experimentation, to say just how well and in what way
the system controls. But the most important thing is to understand that
ONLY A CONTROL SYSTEM CAN DO THIS. If you set up a task that requires
control, and a subject carries it out successfully, then the Test has been
passed and you have a control system. It is not the setting of the task
that explains the success, but the existence of a control system.

Behaviorism is full of ways to make it sound as if the environment is
causing or accounting for behavior (where "behavior" is deliberately used
in an ambiguous way that can shift back and forth between "action" and
"consequence of action" depending on what you're trying to prove). Saying
that a task "requires" or "demands" or "occasions" behavior is one of these
ways of asserting a causal relation without justification. Saying that
instructions "account" for behavior is another way to assert causation
without actually providing a causal argument, or a mechanism. It's a way of
covering up ignorance by offering an oversimplification instead of an
explanation.

Best,

Bill P.

[From Bill Powers (981022.1310 MDT)]

Bruce Gregory (981021.1512)--

If I understand the discussion here, Bruce is arguing that earlier studies
(such as "Models and Their Worlds" by Bourbon and Powers and Mimicry,
Repetition, and Perceptual Control" by Bourbon), demonstrate that in order
to accomplish certain tracking tasks, people _must_ be organized as control
systems. The fact that the tasks _can_ be accomplished therefore allows us
to
conclude that the individuals who accomplish them _are_ organized as control
systems.

Yes.

What remains a puzzle for Bruce, however, is _how_ people get to be
organized in this way. Bill has argued that the simplest answer to this
question involves random reorganization. Bruce is not persuaded that this
mechanism is adequate.

Neither am I. But we have to accept that whatever the mechanism, it has to
produce control systems because that is what we find. Random reorganization
is far more powerful than it seemed when no mechanism was known for getting
systematic effects out of it. Now, with the E.coli principle, we know how a
truly random process can be coupled with biased selection criteria to
produce nonrandom consequences. So "trial and error" no longer seems like
the stab in the dark we used to think it was.

Of course, it is possible that the organization has
been developed, not in the course of the task, but over the lifetime of the
organism.

Or over an evolutionary time span.

In this case, what we might be seeing is time it takes a higher
order system to determine the reference levels for lower order systems
needed to improve the control system performance of this particular task.

I still persist in the idea that the reorganizing system is not the
highest-order system, but a system completely independent of the hierarchy
that can affect it at any level, not just the top level.

Best,

Bill P.

[From Bill Powers (981022.1322 MDT)]

Bruce Abbott (981022.0910 EST)]

So I've been thinking about doing something more along the lines of a
tracking study, in which the rat would track the "target" simply by
following it with its body. Another possibility, more along the lines you
suggest, would be to provide some analog device with which the rat could
control the frequency of a tone.

These are great ideas. You can create a movable light source with a set of
32 or so LEDs powered by the digital outputs of your A/D board. If these
are arranged in a horizontal line, the rat could be asked to learn to keep
its nose tracking the moving light for some time before a food pellet was
delivered. This is a much more natural sort of task than repetitively
pressing a bar -- rats seem to lead with their noses all the time, by
moving their bodies around.

One very useful and cheap analog device is a cadmium sulfide photocell. It
has a resistance that varies with light intensity (more light, lower
resistance). If you put it in series with a fixed resistor and apply a
voltage across the series pair, you get a voltage output that varies
continuously with changes in light intensity. You can easily arrange for
the rat to vary this voltage by moving its nose in and out of a circular
cutout above the photocell, blocking more or less light. The computer can
then convert the digital representation of the voltage into any other
physical effect you please, such as the tone you mention.

Look up Allied Electronics on the Web. They publish their entire catalog
there, with prices, readable with the Acrobat Reader (obtainable free). You
can even order online (I did it when making handles for Kent McClelland and
it worked fine with a credit card). They carry a number of kinds of cadmium
sulfide and selenide photocells.

Analog rats! Go for it!

Best,

Bill P.

[From Bill Powers (981022.1335 MDT)]

Folks, I realize that there are some posts I haven't answered, but right
now I can't get to them. If I blow them off too long, ask again.

Chris Cherpas (981022.0420 PT)--

Rick, Bill, have you done any tracking
experiments/demos in which the left and
right mouse buttons were used, instead
of the track ball, such that holding down
left would move the handle up and
holding down right would move it down?
Seems crude relative to the track ball
(or joy stick), but I wonder what the
how it bad it is at different speeds...

Remind me of this when we start analyzing tracking data using Vensim. I'll
write a program for doing a tracking task this way. Actually, you can do it
using keypresses on the keyboard, too. I assume that control would be
harder, but it would be very interesting to see how much (or IF) the
control parameters change.

Best,

Bill P.

[From Chris Cherpas (981022.1345 PT)]

Bill Powers (981022.1322 MDT)--

You can create a movable light source with a set of
32 or so LEDs powered by the digital outputs of your
A/D board. If these are arranged in a horizontal line,
the rat could be asked to learn to keep its nose
tracking the moving light for some time before a food
pellet was delivered.

I don't have the published reference on-hand, but
it seems something like this task set-up has been
done with pigeons in the so-called continuous
repertoire area...possibly James Holland.

Regards,
cc

Richard Kennaway wrote:

Provide a lever that they can push, directly
connected to a potentiometer determining the
current through the lights. You could even
have the lever connected to a household dimmer.
My own impulse would be to first just provide
them with this setup and see what they do with it.

Disturbances to light level are easily made.

Of course, you might find that they prefer to hide from the experimenter
by keeping the lights turned off.

Not an unreasonable prediction. Nocturnal, rats should
work to dim the lights.

Best regards,
cc

From Phil Runkel on 24 Oct 98, to Bruce Abbott, nominations for variables

a rat might want to control.

Some time back, on the net, someone described an experiment in France with
chickens, in which the cage was very slowly made smaller, but the chicken
could counteract that shrinking by pecking a target.

Another idea is preference for interior decoration. Years ago, a few
experimenters did some experimengts I admired very much in which they gave
rats a choice or simple or complex wallpaper. If you are curious, you can
read more about it on pages 89-91 of Casting Nets.

Regards, PJR.