[From Bruce Abbott (960220.2020 EST)]
There seems to be a difference of opinion as to which VI feedback function
best represents the relationship between response rates and reinforcement
rates. I've been using the following formuala in my control-system simulations:
(1) R = 1/(1/B + VI),
where R is the rate of reinforcer delivery, B is the rate of the operant,
and VI is the average interval of the VI schedule. Re-expressed in terms of
interresponse time (IRT) and delivered interreinforcement interval (VId),
this formula becomes
(2) VId = IRT + VIp,
where VIp is the programmed average interval (same as VI in Equation 1). I
will refer to this as the "linear" function.
Killeen has suggested
(3) R = (1 - exp(-IRT/VIp))*IRT
as the VI schedule function. Which is best? To answer this question I
wrote a simulation in which a "statrat" would be made to respond in some
specified way and collected data on the resulting VId. Below I present the
results for VIp = 15 s. The schedule reinforces with constant probability
per dt; when a reinforcer is "set up," it is held until collected and then
the next scheduled interval begins timing ("unlimited hold").
In the first simulation, the rat is assumed to respond with some constant
probabilily in each dt; the probability is set to produce IRTs of 1, 2, 4,
8, 16, 32, and 64 seconds (average). The simulation continued until 2000
reinforcers had been programmed. Results appear in the table below:
IRTp IRTd VIp VId Linear Killeen
1 1.0073 14.77 15.74 15.77 15.27
2 2.0089 14.81 16.74 16.81 15.83
4 4.0201 14.62 18.64 18.64 16.72
8 8.0888 15.11 23.12 23.20 19.52
16 16.4278 14.69 31.09 31.12 24.41
32 31.5257 15.61 47.47 47.15 36.36
64 63.8306 15.53 80.37 80.36 65.84
In this simulation the linear function is dead on; Killeen's is seriously in
error at IRTs above 2 s. By the way, VIp is not exactly 15 because of the
variability entailed in simulation, but is always close to the requested
value. Computations for the linear and Killeen formulas used the actual
value for VIp rather than the ideal.
Perhaps the superiority of the linear formula is due to the use of an
unlimited hold schedule. The next table presents the results for a "limited
hold" schedule: the schedule begins timing the next interval as soon as the
previous one elapses; if the reinforcer for the previous interval has not
been collected before the next "setup," it is lost.
IRTp IRTd VIp VId Linear Killeen
1 1.0010 14.80 15.74 15.80 15.18
2 2.0170 14.63 16.77 16.65 15.66
4 4.0201 14.86 18.68 18.84 16.94
8 7.9995 15.06 23.32 23.05 19.41
16 16.3654 15.20 30.54 31.56 24.82
32 32.2056 15.07 47.17 47.28 36.52
64 64.6739 15.04 81.72 79.71 65.56
Once again the linear function wins. But what if the rat does NOT respond
with constant probability in each successive time interval dt? In the next
simulation, the limited hold schedule was retained but the rat now produces
a normally distributed series of IRTs with mean as specified and standard
deviation = 0.25*mean.
IRTp IRTd VIp VId Linear Killeen
1 1.0064 15.30 15.80 16.30 15.81
2 2.0040 14.01 15.11 16.02 15.04
4 4.0033 14.72 16.58 18.87 16.82
8 8.0109 15.03 19.88 23.31 19.65
16 16.5814 14.84 24.43 30.89 24.28
32 31.9848 15.48 36.68 47.47 36.62
64 64.9894 15.57 66.66 80.55 66.00
Now the simulation clearly favors Killeen's formula. It is evident that
BOTH formulas provide good summaries of the VI schedule function UNDER
DIFFERENT DISTRIBUTIONS of RESPONDING. The linear formula does well at
predicting VId when the rat responds with constant probability at the given
average IRT, whereas Killeen's formula does well at predicting VId when the
rat responds at the given average IRT but responding varies according to a
normal distribution around this mean, with a standard deviation proportional
to the average IRT. Other distributions of responding will produce yet
different results.
Regards,
Bruce