[From Rick Marken (951204.2200)]

Chris Cherpas --

Well, I finished looking over the Killeen paper (JEAB, 1995, 64, 405-431)

and thought I would report my impressions to the net.

I think you sent this paper because it suggested to you that Killeen

might be inclined toward a PCT approach to behavior. I can't tell, from

the paper, whether that might be true -- but I can tell that if he were so

inclined, he'd have to be willing to make big change in his view of

what's going on in operant conditioning. In particular, he would have

to be able to see reinforcement as a controlled (rather than a

"controlling", "strengthening" or "reinforcing") varIable -- a step that

most reinforcement theorists seem reluctant to take.

As you point out, Killeen does talk about control theory concepts

("defense of setpoints", "control systems analysis") but his analysis of

operant behavior (to the extent that I undertand it; there's a lot of what

for me is pretty high falutin' math) is thoroughly S-R; Killeen gives no

evidence of having any idea that the organisms in the experiments he

describes might be controlling "reinforcements". Indeed,

reinforcements are always the independent variable in his models of

behavior. His basic equation for behavior is:

1. B = kaR/(aR+1)

where B is response rate and R is reinforcement rate. Variants of this

model are derived by making all kinds of complicated assumptions

about hunger, arousal and what not. But the variants of the basic

equation that result from this theorizing are not fundamentally

different from the original. For example, if you want drive to grow

exponentially over time, you get this model:

2. B = kR/(R+1/vh))

The form of the equation is the same; and the model of behavior is the

same; behavior rate (B) is determined by reinforcment rate (R).

Killeen is aware of the fact that behavior determines reinforcement as

much as reinforcement determines behavior; he even writes a feedback

function for a constant probability VI schedule which describes the

effect of behavior rate on reinforcement rate:

3. R = B(1-exp(-R'/B)

So Killeen knows that the behavior in the operant conditioning

situation is described by two simultaneous equations, one describing

the effect of R on B via the organism and the other describing the

simultaneous effect of B on R via the environment. Yet somehow,

when he solves these equations simutaneously (he says that he inserts

the feedback function into the appropriate "motivational" equations and

inserts these into the organism equation (1)), he gets:

4. B = (k-1/a)aR'/(aR'+1)

where R' is scheduled reinforcement rate.

Once again, we end up with behavior,B, as a function of reinforcement, R;

of output that is a function of input. We have entered a dimension of

cause and effect, of procrustean solutions to puzzling problems; look

out for that reinforcer up ahead; next stop, the S-R zone.

The fact that Killeen is able to fit equations like (4) to operant data

at all suggests that organisms must not be controlling very well in these

experiments,; there is plenty of variance in reinforcement rate (the

controlled variable, R) that can be associated with variance in behavior

rate, B, (assuming a constant reference for R). The plots of

reinforcement rate vs behavior rate presented in the Killeen paper

suggest that there is, indeed, very little control going on at all.

By the way, in the PCT model for operant behavior would be something

like this:

R = r

and

B = -1/g(D)

If reinforcement rate, R, is a controlled variable, then its value

is determined by the organisms internal specification for a particular

perceived value of R. Behavior rate depends on any disturbance, D, to

the reinforcement rate; if the disturbance is consant, behavior rate

depends on the feedback function (mainly the schedule), g(), relating

B to R.

Killeen might want to see how well his model fits operant data when

organisms have good control over reinforcement rate; in that case,

Killeen will be using a constant R to predict large schedule induced

variations in B. He can have even more fun by adding a variable

disturbance to R; again R will remain essentially constant while B

varies all over the map (to compensate for the disturbance). If he (or

anyone) does these studies they might (maybe) see that the organism is

controlling a quantity that has been mistakenly referred to as a

reinforcement.

Best

Rick