Schedules of Reinforcement

[From Bruce Abbott (941203.1440 EST)]

Bill Powers (931123.0030 MST)

(Old stuff--I'm still playing catch-up.)

This is the same Baum who wrote a letter to Science about my feedback
article in 1973. I corresponded with him for about a year afterward. Our
correspondence ended when I invited him to visit while I was on vacation
in Massachusetts and he declined saying that we had little further to
talk about. He does not cite me or my work.

Bill, I read your Science article years ago and was disappointed by Baum's
reply to it. Last year John Staddon published a piece in JEAB critiquing the
"conventional wisdom of behavior analysis," as he put it, and was met with an
array of responses that suggest a similar lack of understanding or acceptance.
He was trying to tell them that they needed to model the organism, not just
the environment. If you haven't read this "special section," I think you
would find it interesting for the insights it provides about the current state
of EAB. It's in the September, 1993 issue (volume 60), beginning on page 437.
My own response was one of sadness for the field.

But then matters become confused, because apparently Baum as well as
others are trying to characterize the feedback function in a way that
will explain observed reinforcement and behavior rates on different
schedules as well. So they try to find formulas that will, for example,
"include parameters and a compromise between random and regular
responding." Despite what Baum said about O-rules and E-rules, the
attempt is made to make the E-rules include the O-rules.

This is right on the mark, and I must confess that I didn't see that until you
pointed it out. Fortunately, how one characterizes the "VI feedback function"
does not matter if what you are going to actually do is construct a computer
model of the organism and have it interact with the actual schedule itself and
not with its representation as a proposed mathematical formula.

The problem faced by Baum and others is that their black-box philosophy does
not permit them to model the organism. Instead they are constrained to model
only the observable behavior of the organism and the interaction of that
behavior with schedule parameters. So where are you going to put those O-
rules if you have no organism to put them in? With all terms located in the
environment, there is little reason to keep O-rules and E-rules separate in
the "functional description."

The data to which all three models are fitted show a positive slope of
reinforcements/hr vs responses/min. This suggests that the animals were
behaving in the way I refer to as the left side of the Motherall data in
Staddon's Fig. 7.18. In other words, they are well outside the region of
normal control.

I have a strong feeling that your analysis of the situation is incorrect,
although I have not yet had the time yet to really sit down and think about
alternatives. I do not believe that in these studies the rats were operating
on the left side of the figure (which represents responding on ratio, not
interval schedules). In your model, there is a perceptual variable that drops
steadily between reinforcements, so that longer inter-reinforcement intervals
result in larger perceptual errors, which in turn lead to higher response
rates. Such a model would predict that the animal should respond at maximum
rates when the inter-reinforcement interval is infinite. This might make
sense if keypecking were the only source of food, but in reality the pigeons
only have to wait until the session is over in order to receive enough grain
to keep their weights stable.

In my view, longer inter-reinforcement intervals should lead to less vigorous
responding, not more, especially since most VI schedules tend to generate much
more responding than is necessary to obtain most or all available reinforcers.
Assuming that there are other interesting things the pigeon could occupy its
time doing (e.g., foraging for other, possibly richer sources of food), the
attractiveness of staying at the key and pecking at it may diminish relative
to these other activities as the schedule associated with the key becomes
leaner. Of course, this is an empirical question we can address when I get my
research on-line. I expect to have approval for use of animals this month,
assuming that the committee has no questions for me to address before they
take a vote.