[From Bruce Abbott (941101.1720 EST)]
Bill Powers (941031.1430 MST) --
Bill, by way of preparation for the operant schedules project, I've begun to
look up some of the literature on FR schedules. It's refreshing to get back
to basics....and they don't come much more basic than FR. Here's some things
I've learned so far.
Ferster and Skinner (1957) discuss performance on FR schedules extensively,
and under several conditions of interest from the PCT perspective.
Performance of well-practiced pigeons under typical deprivation conditions
generally consists of a brief acceleration to a high terminal rate of about 4
to 6 responses per second, which is then interrupted by the delivery of the
reinforcer. Ignoring the time required to visit the food magazine and ingest
the grain reinforcer, there is usually little post-reinforcement pause as long
as the fixed ratio is relatively low. At higher ratios (e.g., FR-40) a brief
post-reinforcement pause (PRP) of a few seconds develops. As the ratio
requirement is increased, the length of the PRP increases, whereas the
"running" rate is relatively little (or inconsistently) affected. Thus the
effect of increasing ratio is primarily on PRP length.
Once initiated, a ratio "run" is rarely broken off prior to completion.
The effect of "insufficient reinforcement" was studied accidentally when the
food magazine became partially blocked. Its effect was similar to that of
increasing the ratio size: increased pausing but a high, steady rate of
responding once a ratio run was started.
Ferster also examined the effect of deprivation level of FR performance. The
pigeons were placed on FR 110 for several months while their body-weights were
An ad lib weight determined more than 6 months before the experiment was
begun is probably not very meaningful. However, a body-weight could be
found at which very little pecking occurred on the fixed-ratio schedule,
and this was used as a reference level. We will call the reference
level the inactive weight. (p. 73)
Values up to 112% of inactive weight were studied. Again, the primary effect
was in the duration of the PRP, which lengthened as body weight increased.
Ratio runs, once begun, tended to occur at approximately the same high rate
over a wide range of weights, decreasing only at the highest values.
Overall response (and reinforcement) rates thus decline with increases in
ratio requirement and with decreases in deprivation level, but the effect is
mainly on the PRP. The series of required responses tends to be executed as a
Mechner (JEAB, 1958, pp. 109-121) reports some interesting observations of FR
performance on a special variant of the FR schedule. Rats responded on FR
schedules having ratios of 4, 8, 12, or 16 lever-presses per reinforcement.
Most of the time completing the ratio on the lever produced the reinforcer but
on some occasions it merely "set up" the reinforcer on a second lever, which
then had to be pressed to collect the pellet. This "probe" allowed Mechner to
assess the rat's ability to "count" the number of lever-presses completed
during a run: when the probe condition occurred, performance on the first
lever was not automatically interrupted by reinforcer delivery. Thus the
failure to receive reinforcement after the schedule requirement had been
completed would tell the rat that it was time to try the other lever.
Responses that occurred too early (before the ratio had been completed)
canceled the reinforcement on that run.
Graphs giving the probability of pressing lever 2 as a function of the number
of presses on lever 1 (for each subject separately) show approximately normal
distributions whose means and standard deviations both increase with the size
of the ratio requirement. In each case the distribution is located in such a
way that reinforcement was collected on about 90-97% of probe trials,
according to my measurements off the graph for one animal, N2. What this
indicates, of course is (a) the rats know when reinforcement is overdue and
(b) the compensate for the considerable spread in their "estimates" by
emitting extra responses so as to shift the distribution to the right enough
to collect the prize on better than 90% of probe trials. An example for the
FR 4 schedule appears below (Subject N2):
Probability Distribution of a Run of Length N
> * FR 4 Schedule
> Mean = 6.65
> * SD = 1.73
> * *
> * | *
> *03%| 97% * *
2 4 6 8 10 12 14 16 18
The vertical line over an N of 4 marks the beginning of values that produce
reinforcement on Lever 2. It would be interesting to see the effect of
deprivation level on these distributions. I have a feeling our PCT model will
make some predictions here.
Somewhere there has to be some data (besides Motherall's) on overall response
and reinforcement rates at various FR values. I'll report on that when I find
I ran your FR simulation. I know this is just the first approximation, but
just to be sure we're on the same frequency, I'll note that the model as it
now stands produces behavior unlike that actually observed. As I'm sure you
know, the reason is that the disturbance is acting to increase the error
signal on the same time-scale on which the responses are occurring on the
ratio. The time required to complete the ratio is sufficient to produce a
large error increment, which is then reduced by reinforcer delivery. In the
real rat what probably happens is that the error builds gradually, finally
crossing a threshold that then triggers food-seeking behaviors. One
reinforcer is not sufficient to cancel this error. This would lead to the
rat's emitting a series of ratio runs, finally canceling the perceptual error,
allowing the rat to go back to "other" behavior.