Control of Reinforcement

[From Bruce Abbott (950731.1145 EST)]

Tom Bourbon [950730.2245] --

I'm not quite with you on this idea, Bruce. My impression is that Staddon
and others mistakenly tried to model animals as though they (the animals)
control their own _behaviors_. Staddon, and those who follow a similar
lead, speak of _behaviors_ being "in competition" with one another, or of
_behaviors_ being "allocated," or "selected." They have completely missed
the point that animals are controlling their perceptions of someting other
than their "behaviors."

Tom, the paper we are currently discussing (Ettinger & Staddon, 1983) is
entitled "Operant regulation of feeding: A static analysis." Here is a
quote from the introduction:

     Responding on ratio schedules is regulatory in the following sense:
     As the work requirement is progressively increased, response rates
     typically increase so as to maintain reward rate approximately
     constant--when responding and food are directly linked, a change that
     tends to reduce food rate (an increase in ratio value) produces an
     opposing change in response rate.

You may be confusing John Staddon with Bill Timberlake, whose work fits the
mold you describe.

As for the "allocation" of behavior, you are correct that current models
usually speak of behavior being allocated and that this view is
inconsistent with PCT. But I'm speaking of the problem, not the solution.
I would like to see solutions developed based on PCT. Consider a rat in an
operant chamber. It can earn some tasty treats by pressing a lever or it
can get some exercise by running in a wheel. It cannot perform both acts
(lever-pressing and wheel-running) simultaneously. How will the rat
"allocate" its behavioral resources? That is, what will it choose to do and
when?

It turns out that the data collected in situations such as this one reveal
systematic relationships. Because the rat cannot perform both acts at once,
the more time spent wheel-running, the less time remains for lever-pressing,
and vice versa. Quite some time ago (when I first joined CSG-L) I referred
to this restriction by noting that there is only one "final common pathway"
by which the various control systems can act to establish control over their
various quantities. Typically we have considered only one control system at
a time, or control systems operating in a hierarchy in which higher-level
systems satisfy their reference requirements by setting the reference levels
of the lower systems, so that the problem of resource allocation does not
arise. In the real world this problem arises almost continuously: unable to
simultaneously affect all controlled variables, the organism must work to
correct first one, then the other, then the first again, then a third, and
so on, if all quantities are to be kept near their reference values. The
question is, by what mechanism is this achieved? I can imagine how such
allocation might be modeled in terms of a set of control systems, but I'm
not aware that any such model has been developed or tested. A complete
scientific account of control-system behavior must deal with such
interactions among control systems sharing a common general output mechanism
(the musculature) but controlling different variables.

As I suggested in my earlier post (Tom Bourbon
[950730.1313]), I think a series of simple, direct, tests for the animals'
controlled variables would provide real data for PCT modeling. Furthermore,
the results of the modeling effort would include a demonstration of the
serious flaws in all of the research in which people have tried to study
animals as though they are controllers of "behavors."

I agree, and I intend to perform some of those tests. But first it would be
helpful to know what I should be looking for. You have to remember, I'm
still fairly new to PCT in the sense of actually building and testing
running models, and I'm still trying to sort out, for myself, the
implications of PCT regarding the results of operant studies such as those
investigating schedules of reinforcement. If it should turn out that PCT
implies that such studies tell about nothing more than trivial consequences
of apparatus constraints, so be it: I will at least understand that much.
The only way for me to find out whether this is true or not is to work
through the applications and see what a PCT analysis tells me.

Regards,

Bruce