[From Bill Powers (950724.1330 MDT)]
A very successful 11th CSG meeting is over, the guests have departed,
and tranquillity descends once again on 73 Ridge Place. I will leave it
to others to review the highlights. The basic principle of mixing
applications with theory is still working well.
···
-----------------------------------------------------------------------
Bruce Abbott (950721.1100 EST) --
Ettinger and Staddon reported data for the same four subjects (C1,
C2, C3, and C4) in three experiments. Experiment 1 simply compared
the performances of these subjects on the cyclic-ratio schedule to
those of four other subjects on the standard (single-ratio)
schedule (in which ratio was manipulated over sessions).
Experiment 2 manipulated pellet taste (quinine was added) and
deprivation level (80% ad lib weight versus 95% ad lib weight).
Experiment 3 manipulated amphetamine dosage. Experiment 2 included
a replication of the Experiment 1 cyclic-ratio conditions (80% ad
lib weight); experiment 3 included two such replications (one non-
drug control for each of two amphetamine levels); together this
provides four cyclic-ratio assessments of the same animals under
the same conditions.
This is my idea of shotgun research. When it's all over, you have a
giant collection of indigestible facts of no particular importance or
applicability.
If you plot Seconds/Rft as a function of the ratio requirement a
strange and wonderful thing happens. Within a small amount of
error you can fit four straight lines to these data, all having the
same Y-intercept at 5.5 seconds for 0 responses. I interpret this
as the time required to move from the lever, collect the
reinforcer, and return to the lever. It would appear that under
80% ad lib weight and using 45 mg noyes pellets as reinforcers, all
four animals required the same amount of time to perform this
collection of actions.
The number of reinforcements per unit time r is calculated as the
behavior rate b divided by the ratio m: r = b/m. Therefore the time per
reinforcement is m/b (omitting constants for changing time units). When
you plot ratio against time per reinforcement, you are therefore
plotting the relationship of m/b with m. The meaning of such a plot is
not completely clear, nor is the meaning of the y-intercept for b/m = 0.
It is not usual to look at a relationship in which the same variable
appears on both sides of the equation without being separated first!
I agree that the collection time must be taken into account. It would be
far better to have a direct estimate of the collection time. However,
computing it as you did I then converted back to the behavior rate by
subtracting the collection time from the total time per reinforcement,
giving the time during which pressing behavior was actually taking
place, scaling up the observed behavior rates by
(Total_Time)/(TotalTime - CollectionTime),
and computing a corrected behavior rate. Here is the output of the
program:
Calculate seconds per reinforcement
Ratio Rat
1 2 3 4
2 7.15 6.86 6.62 6.40
4 8.15 7.89 7.63 7.12
8 10.62 9.52 9.23 8.18
16 15.74 13.89 12.20 10.99
32 27.07 23.90 19.79 17.76
64 48.91 39.10 35.04 30.80
Calculate intercept of [sec/reinf vs ratio]
Rat 1 intercept = 5.36 slope = 0.68
Rat 2 intercept = 5.77 slope = 0.53
Rat 3 intercept = 5.47 slope = 0.46
Rat 4 intercept = 5.21 slope = 0.40
Calculate peak behavior rate per hour with intercept time removed
RATIO
2 4 8 16 32 64
Rat 1 4013.36 5154.89 5467.55 5548.19 5304.33 5290.36
Rat 2 6599.83 6779.01 7677.32 7087.75 6352.63 6912.48
Rat 3 6286.10 6670.07 7662.62 8567.13 8045.73 7791.80
Rat 4 6070.50 7548.26 9719.11 9978.97 9181.19 9005.60
The picture we get from these data is that the animals pressed the bar
at a very high rate at all ratios (while not collecting rewards), with
the fastest rate occurring at a ratio of 16. The collection time makes
less difference at the higher ratios because it is a smaller proportion
of the total time between reinforcements. The peak behavior rates change
less than 50% over the 32:1 range of ratios.
What this suggests to me is that the loop gain of the lowest-level
systems is much higher than what we have estimated, at least in these
experiments. To bring the behavior rate close to zero we would have to
make the error much, much smaller than it is here.
The estimates of peak behavior rate above assume that behavior took
place at a uniform rate during the time remaining after collection,
which is not necessarily true. Your formula using the slopes assumes
essentially the same thing.
Your measure of corrected reinforcements per hour is not appropriate,
because the reinforcements did not take place between collection times,
but during them. We need, of course, a record of the actual
reinforcement times. All your calculations were based on the observed
behavior rate and the ratio; there is no independent direct data on
reinforcement rate. Since the only actual data refer to behavior rates,
we have to do the final correction in terms of behavior rates.
The shapes of these curves at low ratios are strongly dependent on the
actual collection time. This, too, needs to be observed.
--------------------------------------------
So what does it mean? The simplest interpretation is that
reinforcement rate is not being controlled.
The Test for control requires that we estimate the rate at which
reinforcements would be delivered in the absence of behavior, then the
rate at which they are delivered when behavior has an effect. Also, to
estimate the reference level we must find the reinforcement rate at
which the behavior rate would just fall to zero. The loop gain can then
be estimated by applying known disturbances.
Apply the Test, and see if your conclusion is appropriate.
The actual position of the reference level is not revealed by the
present data, because the collection time makes the extrapolation to
zero problematic. Using the corrected "peak" behavior rates, we would
extrapolate to a very different zero point than we do with the
uncorrected data.
In addition, varying the ratio directly varies the loop gain of the
control system, so the quality of control declines steeply as the ratio
grows from 2 to 64. If the rat is controlling well at FR-2, it may
hardly be controlling at all at FR-64. This is no problem for a control
model, because the model, too, will control poorly at FR-64, assuming
that its parameters remain the same and that it controls well at FR-2.
What does stay the same across all ratio requirements are (a) the
time required to collect the reinforcer and return to the lever
(about the same for all subjects) and (b) the average rate of
responding (which differs across subjects). At a given level of
deprivation, and for a given reward, the rat maintains a given rate
of responding.
The rate of responding _between collections_ varies only slightly with
the schedule ratio, although the variations seem consistent with a peak
rate at FR-16. I find no simple proportional relationship between rate
of responding and reward rate. In fact, the rate of responding remains
essentially constant (plus-minus 25%) while the reward rate varies by a
factor of 5 to 6. Your generalizations above don't seem to fit the data
I have before me.
These data don't include numbers concerning level of deprivation. I
would expect that the almost-constant rate of responding might show a
relationship to level of deprivation. Deprivation, of course, means a
_decrease_ in the mean sustained reinforcement rate. I would expect
behavior to increase with deprivation, up to a point, and thus to
decrease with increases in sustained reward rate.
The decline in rate of reinforcement with increasing ratio
requirement is exactly that which would be expected given the
longer time required to complete a ratio while maintaining the same
rate of responding.
If you maintain the same rate of responding, the reward rate will
decrease as the ratio increases. That's just arithmetic. So it seems
that the reinforcement rate has no effect of its own on the behavior
rate; what does affect behavior rate is the error signal, the level of
deprivation.
-----------------------------------------
Back to the drawing board?
I think I'll pass until we have some data of our own. Trying to guess
what actually happened in someone else's experiment is just too
exhausting.
-----------------------------------------------------------------------
Best,
Bill P.