[From Bruce Abbott (970817.1250 EST)]
Bill Powers (970817.0720 MDT) --
Bruce Abbott (970817.0135 EST)
I've been puzzling over your two responses to my post on the relationship
between the empirical phenomena of reinforcement and control theory, and
think I know where the difficulty is. I'll try to make this clear in my
replies to key comments from your 970817.0720 post.
Of course I do not deny that your explanation of operant behavior and
learning in terms of control theory is correct; I know you understand it
and I have said so repeatedly.
Good. In other words, we have no disagreement about the _explanation_.
What you are asking of me, however, is like asking Lavoisier to accept an
explanation of combustion that might have been offered by Priestley to show
that the phlogiston explanation handles all the same facts that oxygen
theory covers, but from a different point of view, given only a few special
definitions of terms. In that case I do not think that Lavoisier would have
accepted the explanation. He would have said that no matter how well
phlogiston theory might be able to explain combustion, if we understand its
terminology in carefully-selected ways, it is still wrong. Phlogiston does
not exist.
I am not talking about anything like Phlogiston, which was a (as it turned
out, mythical) substance. I am talking about a set of empirical phenomena
which have been _labeled_ as "reinforcement," "punishment," and
"establishing operation."
And that is what I am saying. No matter what special way you use your
language, hedging it about with disclaimers about not meaning causality and
insisting on strictly operational definitions, reinforcement still does not
exist. There is no special physical effect of a food pellet _or_ a
contingency that reinforces the tendency to perform any particular
behavior. Reinforcement does not exist.
Yes, Bill, the magical "special physical effect," a theoretical concept
sometimes appealed to in order to _explain_ the empirical phenomenon of
reinforcement, does not exist. I keep telling you that I am not talking
about this concept, but you don't seem to be listening.
The problem is that the same term, "reinforcement," has been used both as a
label for an empirical phenomenon and (in a different way) as a mechanism
that is supposed to explain the empirical phenomenon. I am not responsible
for this potentially confusing state of affairs, but I have tried hard to be
clear about which "reinforcement" I am talking about. This is why I keep
using the phrase "empirical phenomenon of reinforcement" rather than just
saying "reinforcement."
Unfortunately, you keep insisting (even against my protests) that I am
referring to the mechanism rather than to the phenomenon. I can see how
incoherent my words must seem from this perspective, but that is your
problem, not mine.
To review, the empirical phenomenon to be explained is this:
1. When a rat has been deprived of food and given the opportunity to earn
food by pressing a lever, the rat acquires lever-pressing behavior and
presses the lever at a relatively high rate.
2. When the contingency between lever-pressing and food delivery is broken,
then after some time has passed one will observe that a high rate of lever
pressing is no longer being maintained. When the contingency is
reestablished, lever-pressing again increases in frequency.
We shall call the observed increased rate of lever pressing in the
contingency phase, relative to the contingency-absent (baseline) phase,
"phenomenon X." We shall say that we have observed "phenomenon X" when the
rate of lever pressing is higher during the contingency phase than during
the baseline phase.
We also observe that phenomenon X is observed under the test conditions
described above only when the rat has been without food for some time; this
turns out to be a prerequisite for observing phenomenon X when lever
pressing delivers food pellets. We shall call this condition "prerequisite
Y." The task before us is to explain phenomenon X, and to explain why
prerequisite Y is necessary before phenomenon X will be observed.
Because phenomenon X is simply a name given to an observed phenomenon, there
is no point in arguing that it does not exist.
To demonstrate to me how reasonable the reinforcement interpretation is,
you presented, in the post previous to the one I'm answering, an imaginary
scenario in which the contingency was turned on and off, while the behavior
involved with it appeared and disappeared. This is how you imagined the
reinforcing effect would be observed. But you haven't been paying
attention: the control theory prediction is not like the reinforcement
prediction. Control theory predicts that when you remove the contingency,
the behavior will not die out; instead, it will redouble, becoming more and
more energetic as the animal attempts to produce the food pellets. If you
then turn the contingency back on, the behavior will decrease again, back
to its former level or thereabouts. . . .
Your claim is that I have made my case by ignoring the fact that response
rate initially increases immediately after the transition from reinforcement
conditions to extinction conditions. The claim is easily dispatched. Here
is what I said:
Now break the contingency by disconnecting the feeder, so that pressing the
lever no longer produces a food pellet. After a time, we observe that the
rat is no longer pressing the lever at the high rate that had formerly
characterized its performance. In fact, it rarely presses the lever at all.
The phrase "after a time" is there for a reason. There is an initial
increase in the rate or force of responding which is sometimes (but not
always) observed the first time the animal experiences the extinction
condition following reinforcement. It is transient; the steady-state result
is a decline of responding, usually back down to pre-reinforcement levels.
Later in the post, I said this:
Now we disconnect the feeder, thus opening the loop through the environment.
After a time, we observe that lever pressing has essentially ceased. Lever
pressing no longer allows the rat to control its CV, and some higher level
in the rat's system has stepped in to prevent the runaway lever-pressing
that would be expected of a simple one-level control system when its loop
has been opened on the environment side. So, in the steady state, we get
lever pressing when lever pressing produces food, and we do not get lever
pressing when lever pressing does not produce food, the pair of outcomes
which together define the food as a reinforcer.
Note my reference to "runaway lever-pressing," and inclusion once again of
the qualifying phrase "after a time." This is hardly what I would call a
failure on my part to pay attention to the control theory prediction. In
fact, I had to introduce the notion of "some higher-level system" stepping
in to prevent runaway lever-pressing, because runaway lever-pressing is not
what is observed under the procedure by which one tests for the presence of
the reinforcement phenomenon.
I will skip over your comments about word usage as they are entirely off the
mark.
The only reason I bother to continue with this discussion is that I know
you are perfectly capable of consistent logical thinking. You do do
mathematics, and you write excellent programs. You would never substitute
the name of a function for the name of a variable in a computer program.
You would never write z := x/y when you meant z = x*y -- or if you did, you
would quickly correct your own mistake. It's only when presenting verbal
arguments intended to preserve the heritage of radical behaviorism that you
abandon these rigorous ways of thinking and complain that I am quibbling
over words.
I am really afraid that your response to this post is going to be more of
the same, although I hope not. Despite your assertions to the contrary, my
position is consistent and logical. If it does not appear that way to you,
then you might wish to consider the possibility that you do not understand
what I am saying, and try to work from there. That possibility seems to
have been overlooked up to now.
Regards,
Bruce