[From Bruce Abbott (950627.1645 EST)]
Bill Powers (950627.1012 MDT)]
Bruce Abbott (950626.2200 EST)
The reinforcement rate remains constant and independent of the ratio m.
I think you had better check this result before we go on!
Good idea. Here's where the problem begins:
I think I understand your model now. The time taken per response is 0.2
sec and the minimum interreinforcement time is 5.8 sec.
The total time to produce m responses is 0.2*m/b, where m is the number
of responses per reinforcement, and b is the number of responses per
second. The units come out to seconds per reinforcement. Add to that the
minimum time to retrieve one reinforcement, 5.8 sec per reinforcement,
and we have
(1) r = 1/(5.8 + 0.2*m/b),
reinforcements per second as a function of behaviors per second.
The problem I was posing was, what happens when responding at maximum rate
is not enough to bring the rate of reinforcement to its reference level?
For the purpose of investigating this question I assumed that maximum
response rate is 0.2 seconds per response (i.e., 5 responses/sec). If the
reinforcement rate on the CRF schedule is 10 rft/min, this implies that the
interreinforcement interval is 6 sec. With 0.2 sec of that interval taken
up by the lever-press, that leaves 5.8 sec for collecting the reinforcer.
Adding the ratio requirement m (responses per reinforcement) into the
equation, we get
(1) t = 5.8 + 0.2*m seconds per reinforcement.
The number of reinforcements per second will be the inverse of this:
(2) r = 1/(5.8 + 0.2*m) reinforcements per second.
If you wish to produce an equation that gives the reinforcement rate as a
function of the response rate (b), you must substitute 1/b for 0.2 in the
above formula, giving
(2) r = 1/(5.8 + m/b) reinforcements per second.
This equation will hold until 1/b = 0.2, below which 1/b is a constant.
This is equivalent to a rate of 5 responses/sec, the assumed maximum rate.
If the reference rate of reinforcement is very close to the maximum rate,
there will be a region in which, as the ratio requirement increases, the
resulting increase in error will drive the response rate up. When response
rate reaches its maximum, that rate will be independent of error size, but
interreinforcement rate will continue to decline because of the increasing
time required to complete the larger and larger ratio requirements.
I've been looking for data in the literature on the relationship of behavior
to the size of the ratio in variable ratio schedules. (It is surprising how
little published data there are in which the ratio parameter was the
specific subject of the investigation!) Here's one I think is interesting,
although the study did not include enough different ratios (only three).
The study used condensed milk as the reinforcer, delivered via a dipper as
in Motheral's study. Several dilutions were tested (more about that later).
During initial (baseline) testing, the concentration was 30% and the
subjects (4 rats at 80% ad lib weight) were exposed to VR-10, -40, and -80
schedules. The data reported include the overall response rate, the
"running rate" (rate during completion of the ratio, excluding the "post
reinforcement pause," and the duration of the post reinforcement pause (the
time from delivery of the reinforcer until the next lever press). Here are
the data for individual rats (estimated from the graph):
VR-10 VR-40 VR-80
Rat 1 0.8 1.5 1.8 responses/sec
Rat 2 0.8 1.4 1.7
Rat 3 0.7 1.5 1.5
Rat 4 0.8 1.5 1.5
···
-----------------------------
Average 0.8 1.5 1.6 responses/sec
2880 5400 5760 responses/hour
288 135 72 reinf/hour
Rsp/hour
6000 *
* +
* +
*
* *
4500 *
* *
*
* *
*
3000 * +
*
* *
*
* *
1500 *
* *
*
* VR: 80 40 10 * ref
* | | | |
0 **********|*********|*********|*********|*********|
0 100 200 300 400 500
Dippers/Hour
The actual points are shown as plus signs in the graph. Using only the
VR-10 and VR-40 points, the fitted line gives an output sensitivity of 16.5
and an estimated reference level of about 460 reinforcements/hour. Although
it is dangerous to make much of the deviation of the VR-80 point from this
line (given that the line is based on only two points), its position would
be consistent with that portion of the curve which is beginning to level off
prior to the downturn expected if we had the left limb of the complete function.
Now we come to some data we didn't have for Motheral's experiments:
Running Rate (rsp/sec) Post-Reinf Pause (sec)
VR 10 VR-40 VR-80 VR-10 VR-40 VR-80
Rat 1 4.3 2.8 3.4 09 11 19
Rat 2 3.5 2.6 2.5 08 11 11
Rat 3 3.0 2.6 2.3 08 08 14
Rat 4 4.6 2.6 2.7 08 08 16
------------------------------ -------------------
Average 3.9 2.7 2.7 rsp/sec 08 10 15 seconds
234 162 162 rsp/hr
It would appear that increasing the ratio requirement from VR-10 to VR-40
reduced the response rate during ratio runs but that a further increase of
the ratio to VR-80 has little or no additional impact. This would be
consistent with the rats maintaining a somewhat reduced running rate during
longer ratio runs in order control the sensory effects of overexertion (long
ratios at the higher rate might lead to oxygen starvation of the muscle or
the buildup of lactic acid, for example).
However, increasing the ratio from VR-10 to VR-40 had little or no impact on
the length of the postreinforcement pause but a further increase to VR-80
was assocated with a lengthening of the pause. This latter effect would be
consistent with a increase in time "doing other things" as the cost of
earning the dippers of milk begins to exact too high a penalty for the
benefit returned. (Note: The article states that the time required to
consume the reinforcer is no more than 3 seconds.)
This is, of course, speculation, and I may be over-interpreting the data,
which may not be reliable enough to trust the reality of the relationships
indicated by the averages given above (see exceptions for individual rats).
During a subsequent test phase, the concentration of the milk was varied
within-session from 10% to 75% and the duration of the postreinforcement
pause was recorded FOLLOWING delivery at each concentration. Higher
concentrations produced longer postreinforcement pauses, and the increase in
pause with increased concentration was larger the higher the ratio
requirement (although the data at VR-80 are somewhat messy). If higher
concentrations are more effective in temporarily reducing the error in
nutrient intake, this is what would be expected under the PCT model.
Here's the reference:
Priddle-Higson, P. J., Lowe, C. F., & Harzem, P. (1976). Aftereffects
of reinforcement on variable-ratio schedules. _Journal of the Experimental
Analysis of Behavior_, _25_, 347-354.
Regards,
Bruce