The S-R Zone

[From Rick Marken (951204.2200)]

Chris Cherpas --

Well, I finished looking over the Killeen paper (JEAB, 1995, 64, 405-431)
and thought I would report my impressions to the net.

I think you sent this paper because it suggested to you that Killeen
might be inclined toward a PCT approach to behavior. I can't tell, from
the paper, whether that might be true -- but I can tell that if he were so
inclined, he'd have to be willing to make big change in his view of
what's going on in operant conditioning. In particular, he would have
to be able to see reinforcement as a controlled (rather than a
"controlling", "strengthening" or "reinforcing") varIable -- a step that
most reinforcement theorists seem reluctant to take.

As you point out, Killeen does talk about control theory concepts
("defense of setpoints", "control systems analysis") but his analysis of
operant behavior (to the extent that I undertand it; there's a lot of what
for me is pretty high falutin' math) is thoroughly S-R; Killeen gives no
evidence of having any idea that the organisms in the experiments he
describes might be controlling "reinforcements". Indeed,
reinforcements are always the independent variable in his models of
behavior. His basic equation for behavior is:

1. B = kaR/(aR+1)

where B is response rate and R is reinforcement rate. Variants of this
model are derived by making all kinds of complicated assumptions
about hunger, arousal and what not. But the variants of the basic
equation that result from this theorizing are not fundamentally
different from the original. For example, if you want drive to grow
exponentially over time, you get this model:

2. B = kR/(R+1/vh))

The form of the equation is the same; and the model of behavior is the
same; behavior rate (B) is determined by reinforcment rate (R).

Killeen is aware of the fact that behavior determines reinforcement as
much as reinforcement determines behavior; he even writes a feedback
function for a constant probability VI schedule which describes the
effect of behavior rate on reinforcement rate:

3. R = B(1-exp(-R'/B)

So Killeen knows that the behavior in the operant conditioning
situation is described by two simultaneous equations, one describing
the effect of R on B via the organism and the other describing the
simultaneous effect of B on R via the environment. Yet somehow,
when he solves these equations simutaneously (he says that he inserts
the feedback function into the appropriate "motivational" equations and
inserts these into the organism equation (1)), he gets:

4. B = (k-1/a)aR'/(aR'+1)

where R' is scheduled reinforcement rate.

Once again, we end up with behavior,B, as a function of reinforcement, R;
of output that is a function of input. We have entered a dimension of
cause and effect, of procrustean solutions to puzzling problems; look
out for that reinforcer up ahead; next stop, the S-R zone.

The fact that Killeen is able to fit equations like (4) to operant data
at all suggests that organisms must not be controlling very well in these
experiments,; there is plenty of variance in reinforcement rate (the
controlled variable, R) that can be associated with variance in behavior
rate, B, (assuming a constant reference for R). The plots of
reinforcement rate vs behavior rate presented in the Killeen paper
suggest that there is, indeed, very little control going on at all.

By the way, in the PCT model for operant behavior would be something
like this:

R = r

and

B = -1/g(D)

If reinforcement rate, R, is a controlled variable, then its value
is determined by the organisms internal specification for a particular
perceived value of R. Behavior rate depends on any disturbance, D, to
the reinforcement rate; if the disturbance is consant, behavior rate
depends on the feedback function (mainly the schedule), g(), relating
B to R.

Killeen might want to see how well his model fits operant data when
organisms have good control over reinforcement rate; in that case,
Killeen will be using a constant R to predict large schedule induced
variations in B. He can have even more fun by adding a variable
disturbance to R; again R will remain essentially constant while B
varies all over the map (to compensate for the disturbance). If he (or
anyone) does these studies they might (maybe) see that the organism is
controlling a quantity that has been mistakenly referred to as a
reinforcement.

Best

Rick