Darth Vader speaks

CSGnet Archive CSG1996
1

[From Bruce Abbott (961125.1650 EST)]

Bill Powers (961124.2330 MST) --

And one warning, Darth Abbott. It will avail you nothing to reveal to me
that you are my father.

Bill Powers (961124.2355 MST) --

I feel that this result is sufficiently well-established that it requires
re-examining all experimental operant conditioning data. Unfortunately, raw
data have not been retained in most experiments in a form that would permit
removing the collection time effect from the reported average rates of
behavior. This means that an enormous body of experimental results must now
be called into question, and the experiments must be re-done to see if the
results are actually what they were reported to be. This isn't a question of
cheating; it's just that the possibility of a significant collection-time
effect on apparent rates or responding was not quantitatively investigated.
Unfortunately, the standard method of reporting behavior-rate data is to
divide total responses by total time during a session of one hour or more, a
method that not only includes collection time, but time during which the
animal is not even at the bar or key -- as I mentioned some time ago, it
included time, for all rats in our experiments, when a rat in the operant
chamber was actually asleep.

This, of course, applies to ALL the experiments Skinner used as the basis
for formulating his laws of reinforcement. They must all be done over again.

Son, as you get older you will eventually learn to harness your youthful
exuberance and become more cautious about making such sweeping statements,
as I, your father, have learned to do. Wait a moment -- it's hard to see
here over on the Dark Side -- there, this is better, under the light. Now,
what was I saying?

Oh yes, the analysis involving collection time applies strictly to
performance on ratio schedules, which tend to promote maximum rates of
responding. Very little work in EAB has been done using ratio schedules,
for the very reason that they lock reinforcement and response rates
together. In addition, the FR-1 schedule we have been using for the weight
study produces rapid satiation (usually in less than a half hour), and what
rats generally do after they've had their fill is to explore briefly,
perhaps groom, and then take a nap. This is not the sort of thing they do
on the usually employed schedules in EAB research, which do not offer a
sufficient rate of food delivery to bring on napping prior to the end of the
experimental session. Furthermore, the recording of cumulative records
(pioneered by Skinner) allows one to assess not only the average rate of
responding in a session but local patterns as well, so that one may judge
whether the averaging process is an adequate summary of session performance
or not. Classic findings such as the dynamic form of responding during
various schedules (e.g., the fixed-interval scallop) were not presented by
Skinner as averages but as individual examples of such patterns, displayed
as they were directly recorded.

And one more thing. A modern reinforcement analysis would take into account
the opportunity available to an animal (or person!) to engage in other
activities (and receive other reinforcers) than those available on the
programmed schedule of reinforcement. Average rate of responding on the
programmed schedule would be expected to decline as alternative sources of
reinforcement became relatively more attractive or required less effort
and/or time to obtain them. The observed pattern would be expected to
incude stretches of time during which the animal was doing something else
other than responding on the programmed schedule, as observed. The PCT
concept of competition between control systems is comparable to this notion.

Our findings show that the control model proposed by John Staddon to explain
the observed relationship between response (and reinforcer) rate and fixed
ratio value is incorrect; our rats just respond at the same high rate
regardless of the ratio, while the collection time remains constant. This
means that a higher proportion of the time is spent responding (and a lower
proprotion collecting and eating) as the ratio requirement increases. As a
result, the average rate of responding (and reinforcement) increases with
the ratio. Our conclusion is that rats, on this schedule at least, to not
control _rate_ of reinforcement according to some reference rate. More
likely, they control _time to reinforcement_, attempting to bring that time
to a reference of zero delay.

Regards,

Darth

2

[From Bill Powers (961125.1710 MST)]

Bruce Abbott (961125.1650 EST)--

This, of course, applies to ALL the experiments Skinner used as the basis
for formulating his laws of reinforcement. They must all be done over again.

Son, as you get older you will eventually learn to harness your youthful
exuberance and become more cautious about making such sweeping statements,
as I, your father, have learned to do. Wait a moment -- it's hard to see
here over on the Dark Side -- there, this is better, under the light. Now,
what was I saying?

Oh yes, the analysis involving collection time applies strictly to
performance on ratio schedules, which tend to promote maximum rates of
responding. Very little work in EAB has been done using ratio schedules,
for the very reason that they lock reinforcement and response rates
together. In addition, the FR-1 schedule we have been using for the weight
study produces rapid satiation (usually in less than a half hour), and what
rats generally do after they've had their fill is to explore briefly,
perhaps groom, and then take a nap.

OK, Dad, so all we have to do is reinvestigate all ratio experiments. If we
revised all conclusions based on statements about dependency of pressing
rates on ratio, it seems to me that would weed out a lot of erroneous
reports, as well as conclusions which graduate students have dutifully
memorized.

However, I'm wondering what would happen if we took the same new approach to
other kinds of schedules. Those schedules, too, yield results that are
reported as session averages. Even Ferster and Skinner's plots don't tell us
what the rats were actually _doing_ during those experiments. Is there, in
fact, any change in "running rate" of pressing during variable ratio or
interval experiments? The only way to tell is to do the same kind of
analysis you're doing with the Staddon replication. Somehow I doubt that
rats undergo a major reorganization when they are presented with other kinds
of schedules; I think they probably go right on behaving the same way they
always do, pressing the lever as fast as they can comfortably do, and
pausing to collect the food whenever it mysteriously appears. When the yield
of food is low enough, they probably spend more time looking for food
elsewhere, making the pressing rates look lower. When the analyst looks only
at the recorded data, none of these details jump out and announce
themselves. The analyst can weave very complicated stories about what is
going on, far more complicated than what the animal is actually doing. It
would help to look at what the animals are really doing.

Our findings show that the control model proposed by John Staddon to explain
the observed relationship between response (and reinforcer) rate and fixed
ratio value is incorrect; our rats just respond at the same high rate
regardless of the ratio, while the collection time remains constant. ...
More likely, they control _time to reinforcement_, attempting to bring that
time to a reference of zero delay.

The only way to support that last idea would be to show that as time to
reinforcement increased, pressing rate increased, and that as it shortened,
pressing rate decreased. We don't observe that. The rate of pressing simply
doesn't change. The rat doesn't press any faster when it's 64 presses away
from reinforcement time than when it's 4 presses away. It simply doesn't
know when the reinforcement is going to come. It presses until it comes,
eats it, and goes back to pressing. We can't justify any more complicated a
way of describing what's happening.

Best,

Luke P.

3

[From Bruce Abbott (961126.0825 EST)]

"Luke" Powers (961125.1710 MST) --

However, I'm wondering what would happen if we took the same new approach to
other kinds of schedules. Those schedules, too, yield results that are
reported as session averages. Even Ferster and Skinner's plots don't tell us
what the rats were actually _doing_ during those experiments. Is there, in
fact, any change in "running rate" of pressing during variable ratio or
interval experiments? The only way to tell is to do the same kind of
analysis you're doing with the Staddon replication. Somehow I doubt that
rats undergo a major reorganization when they are presented with other kinds
of schedules; I think they probably go right on behaving the same way they
always do, pressing the lever as fast as they can comfortably do, and
pausing to collect the food whenever it mysteriously appears.

The high rate of pressing is a characteristic of ratio schedules. Indeed
if, as I have hypothesized, the rats are attempting to reduce the delay of
pellet delivery (timed from initiation of the first lever-press) toward a
zero-delay reference, one would expect rapid responding. If responding is
already maximal on low ratios, it would not be able to increase on higher
ratios.

Other schedules (e.g., interval schedules) do not promote such high rates.
Fixed interval schedules produce a characteristic acceleration from low to
high rates that has been characterized as the "fixed-interval scallop"
because of its appearance on the cumulative record. The rate reaches its
peak as the fixed interval elapses. Thus observation does not support your
hypothesis that a high fixed rate of presses will occur regardless of schedule.

It is not that the rats undergo "massive reorganization" that brings this
about, either. It's just that the rats learn something about the delivery
of food that they cannot learn on the ratio schedule, having to do with the
time following the previous delivery at which another press will again be
followed by the arrival of a pellet.

When the yield
of food is low enough, they probably spend more time looking for food
elsewhere, making the pressing rates look lower. When the analyst looks only
at the recorded data, none of these details jump out and announce
themselves. The analyst can weave very complicated stories about what is
going on, far more complicated than what the animal is actually doing. It
would help to look at what the animals are really doing.

This is certainly a mistake that some analysts have made -- Staddon is a
prime example, and I wouldn't be surprised to find others. Not everyone has
been this sloppy, however. I'll be very interested to see what emerges when
we take a look at performance on some of the other schedules.

Our findings show that the control model proposed by John Staddon to explain
the observed relationship between response (and reinforcer) rate and fixed
ratio value is incorrect; our rats just respond at the same high rate
regardless of the ratio, while the collection time remains constant. ...
More likely, they control _time to reinforcement_, attempting to bring that
time to a reference of zero delay.

The only way to support that last idea would be to show that as time to
reinforcement increased, pressing rate increased, and that as it shortened,
pressing rate decreased. We don't observe that. The rate of pressing simply
doesn't change.

Imagine that when the rat returns to the lever, it wants the next pellet
_now_. To get the next pellet, it must press (and release) the lever N
times. We can model this as a control system, with delay-to-pellet the
controlled variable and the reference set to zero delay. If the gain in
this system is high enough, even a small delay will be sufficient to produce
a high rate of pressing. When N is low, the rat will press as rapidly as it
can. When N is high, the longer delay imposed by the higher ratio would be
compensated for by an even higher rate of pressing, except that the rat is
already pressing at as high a rate as it can manage. Given the amount of
time required to press the lever several times, there is no way the rat can
even come close to producing a zero delay, so behavior just runs up against
the stops. That's my current hypothesis, subject to modification as more
evidence becomes available.

Note that this hypothesis presupposes that the rat has learned that faster
pressing yields shorter pellet-delivery delays -- i.e., that it has learned
the ratio transfer function. When it learns a different transfer function
associated with a different schedule (e.g., on interval schedules, there is
no worthwhile decrease in delivery time beyond a certain rate of lever
pressing), we can expect a different outcome, i.e., maximal response rates
are not to be expected). So yes, there will be reorganization, although I
would not characterize it as "massive."

It simply doesn't
know when the reinforcement is going to come. It presses until it comes,
eats it, and goes back to pressing. We can't justify any more complicated a
way of describing what's happening.

You're right, of course. What I propose is not a conclusion but an
hypothesis to be tested -- my best current guess.

I must leave you now: the Emperor calls. [Time to put the rats back to work.]

May the Force be with you,

Darth

4

[From Bill Powers (961126.0800 MST)]

Bruce Abbott (961126.0825 EST) --

Other schedules (e.g., interval schedules) do not promote such high rates.
Fixed interval schedules produce a characteristic acceleration from low to
high rates that has been characterized as the "fixed-interval scallop"
because of its appearance on the cumulative record. The rate reaches its
peak as the fixed interval elapses. Thus observation does not support your
hypothesis that a high fixed rate of presses will occur regardless of schedule.

I know that is what the traces on a cumulative record look like. I would
like to see what the rats are actually doing. After a reinforcement, and
after the collection time, do they sit before the lever, reach out, and go
tap ......... tap ........... tap .......tap ..... tap..... tap... tap..
tap.. tap.. taptaptap? That's what the record suggests (and what your verbal
description suggests), but somehow I doubt that this is what the rats are
actually doing. I was fooled once with respect to the fixed-ratio
experiments; I had a completely wrong mental picture of what I would see in
the operant cage, which your videos cured me of. I don't want to be fooled
again.

I don't have the Ferster and Skinner volume on my shelves, but in Cumulative
Record, p. 104, there is Fig. 2 showing "Characteristic performance by
pigeon under fixed-interval reinforcement." In the initial part of the run
the rate of pecking starts out flat after a reinforcement, then apparently
rises in a smooth curve to a final rate which continues until the next
reinforcement. Under a magnifying glass I can see that the first part of the
initial curve is really made of flat segements with multiple steps between
them, the rising portion having a much higher slope than appears to the eye
(which smooths out the curve). After the second reinforcement there is a
much longer flat part of the curve and the rise to a steady rate is more
abrupt. By the fourth reinforcement, the transition from no pecking to
pecking at a steady high rate is still more abrupt, and continues to be
abrupt until the end of the tracing in the Figure. Once the high rate
appears, it seems to continue at a constant value for several minutes
(several hundred pecks) until the reinforcer appears. Skinner describes
these tracings this way:

"The overall pattern of performance on a 'fixed-interval' schedule is a
fairly smoothly accelerating scallop in each interval, the acceleration
being more rapid the longer the initial pause." He then adds, "Local
effects due to separate reinforcements are evident, however, which cannot be
discussed here for lack of space."

Evidently, Skinner decided he was seeing smooth accelerations, and from then
on, no matter what the actual behavior of the tracing, he saw all
transitions as examples of smooth accelerations -- even when there was an
instantaneous transition from no pressing to pressing at a constant rate, as
in the sixth segment.

The "pauses" of which he speaks, judging from the scale of the drawing,
range from a minimum of about 1 or 2 minutes at the left of the drawing to
about 15 minutes at the right side. That is a very long time for the pigeon
to be standing in front of the key not doing anything. I suspect that it was
pecking at the wall and the floor, and in the food dish, but not on the key.
The gradual rise in rate that is seen in some of the segments could easily
be recording the movement of the mean pecking place across the edge of the
key. In the latter segments, the transition from no pecking to pecking is
quite abrupt, and the pecking rate seems perfectly constant once it starts,
until the reinforcer occurs. In the last four segments you can lay a
straight-edge along the curve with no visible deviation for 200 and more
pecks. The appearance of curvature that one sees when taking in the whole
pattern is an optical illusion.

The "cumulative record" is a very poor way to present the data. Human vision
tends to read large patterns into it which aren't actually there on close
inspection. I would say that a much more accurate general description of the
curves would be "periods of no pecking alternating with periods of pecking
at a constant rate."

The way to settle this, of course, is to record the individual pecks and
plot the data in bins to show the distribution of inter-peck intervals.
Until that's done I'm not going to believe any description.

It is not that the rats undergo "massive reorganization" that brings this
about, either. It's just that the rats learn something about the delivery
of food that they cannot learn on the ratio schedule, having to do with the
time following the previous delivery at which another press will again be
followed by the arrival of a pellet.

I doubt that rats are smart enough to learn that. Your interpretation
depends on seeing the cumulative records as a gradual increase in pecking
rate as the time of reinforcement nears. If that isn't the case, if the
pecking simply comes to a constant rate which is maintained until the food
appears, then a different interpretation is needed.

When the yield
of food is low enough, they probably spend more time looking for food
elsewhere, making the pressing rates look lower. When the analyst looks only
at the recorded data, none of these details jump out and announce
themselves. The analyst can weave very complicated stories about what is
going on, far more complicated than what the animal is actually doing. It
would help to look at what the animals are really doing.

This is certainly a mistake that some analysts have made -- Staddon is a
prime example, and I wouldn't be surprised to find others. Not everyone has
been this sloppy, however. I'll be very interested to see what emerges when
we take a look at performance on some of the other schedules.

Yes, that's what we must do. If an influential person like Staddon can make
such blunders, so can other influential people. It's easy for an atmosphere
to develop in which the leaders in the field, just because they are leaders
and because the rest of the world is ranged against them, are not
scrutinized very closely. Also, in a small field, the most prominent people
inevitably are involved in refereeing each others' papers, and it would be
only human to give one's fellow revolutionaries the benefit of the doubt. If
Skinner says that we see scallops under interval schedules, then scallops we
see. By golly, there they are!

Imagine that when the rat returns to the lever, it wants the next pellet
_now_. To get the next pellet, it must press (and release) the lever N
times. We can model this as a control system, with delay-to-pellet the
controlled variable and the reference set to zero delay. If the gain in
this system is high enough, even a small delay will be sufficient to produce
a high rate of pressing. When N is low, the rat will press as rapidly as it
can. When N is high, the longer delay imposed by the higher ratio would be
compensated for by an even higher rate of pressing, except that the rat is
already pressing at as high a rate as it can manage. Given the amount of
time required to press the lever several times, there is no way the rat can
even come close to producing a zero delay, so behavior just runs up against
the stops. That's my current hypothesis, subject to modification as more
evidence becomes available.

I don't want to imagine that. That's too much imagining with no evidence. I
don't know that the rat can tell the difference between 4 presses and 64
presses, or between 1 minute and 10 minutes of time. And you're "proving"
that your hypothesis is right by pointing to what _doesn't_ happen -- you
say the rat would press even faster according to the hypothesis, except that
it can't, which is why we don't _ever_ see the rat pressing faster as the
hypothesis demands. You're turning a total lack of evidence into evidence
for your proposal!

Note that this hypothesis presupposes that the rat has learned that faster
pressing yields shorter pellet-delivery delays -- i.e., that it has learned
the ratio transfer function.

How the hell did it learn that, when it never presses any faster because it
can't, or for that matter, any slower?

When I say that organisms learn transfer functions, I don't mean that they
learn to _perceive_ transfer functions. I mean that learning (of the
reorganizing type) shows up as a change in the organism's own transfer
functions, rather than as a pairing of specific actions with specific
consequences. I think that the same transfer function will probably describe
the rat's organization under any schedule at all. I claim that unless we
prove otherwise, all the different appearances of behavior under different
schedules are being created by the schedules and the differing
interpretations put on them, while the rats themselves are not doing
anything differently at all.

When it learns a different transfer function
associated with a different schedule (e.g., on interval schedules, there is
no worthwhile decrease in delivery time beyond a certain rate of lever
pressing), we can expect a different outcome, i.e., maximal response rates
are not to be expected). So yes, there will be reorganization, although I
would not characterize it as "massive."

But you're _imagining_ that on an interval schedule, we do not get maximal
(or at least constant) response rates. Erase that fact. It's not a fact
until you and I determine, in an experiment performed and analyzed correctly
by public means, that it is a fact. As you said yourself some time ago, we
can't trust ANY results reported in the past until we can replicate them
ourselves. The figure I cited from Skinner suggests that it is not a fact.

It simply doesn't
know when the reinforcement is going to come. It presses until it comes,
eats it, and goes back to pressing. We can't justify any more complicated a
way of describing what's happening.

You're right, of course. What I propose is not a conclusion but an
hypothesis to be tested -- my best current guess.

I must leave you now: the Emperor calls. [Time to put the rats back to work.]

May the Force be with you,

[The ghost of Obiwan Kenobe enters stage left and gazes sorrowfully at the
looming figure bent over the rows of glass chambers, its face no more
obscured from our view by the dark mask than the world is obscured for the
peering eyes within by the same dim filter.]

Ah, Darth, Darth, there are dark days ahead. The Emperor is using you for
his own evil ends. You must save yourself before it's too late. May the
Force be with you, too, my old friend.

Wha -- wha --??? I must have dozed off.

Best,

Bill P.