[From Bruce Abbott (960220.2020 EST)]

There seems to be a difference of opinion as to which VI feedback function

best represents the relationship between response rates and reinforcement

rates. I've been using the following formuala in my control-system simulations:

(1) R = 1/(1/B + VI),

where R is the rate of reinforcer delivery, B is the rate of the operant,

and VI is the average interval of the VI schedule. Re-expressed in terms of

interresponse time (IRT) and delivered interreinforcement interval (VId),

this formula becomes

(2) VId = IRT + VIp,

where VIp is the programmed average interval (same as VI in Equation 1). I

will refer to this as the "linear" function.

Killeen has suggested

(3) R = (1 - exp(-IRT/VIp))*IRT

as the VI schedule function. Which is best? To answer this question I

wrote a simulation in which a "statrat" would be made to respond in some

specified way and collected data on the resulting VId. Below I present the

results for VIp = 15 s. The schedule reinforces with constant probability

per dt; when a reinforcer is "set up," it is held until collected and then

the next scheduled interval begins timing ("unlimited hold").

In the first simulation, the rat is assumed to respond with some constant

probabilily in each dt; the probability is set to produce IRTs of 1, 2, 4,

8, 16, 32, and 64 seconds (average). The simulation continued until 2000

reinforcers had been programmed. Results appear in the table below:

IRTp IRTd VIp VId Linear Killeen

1 1.0073 14.77 15.74 15.77 15.27

2 2.0089 14.81 16.74 16.81 15.83

4 4.0201 14.62 18.64 18.64 16.72

8 8.0888 15.11 23.12 23.20 19.52

16 16.4278 14.69 31.09 31.12 24.41

32 31.5257 15.61 47.47 47.15 36.36

64 63.8306 15.53 80.37 80.36 65.84

In this simulation the linear function is dead on; Killeen's is seriously in

error at IRTs above 2 s. By the way, VIp is not exactly 15 because of the

variability entailed in simulation, but is always close to the requested

value. Computations for the linear and Killeen formulas used the actual

value for VIp rather than the ideal.

Perhaps the superiority of the linear formula is due to the use of an

unlimited hold schedule. The next table presents the results for a "limited

hold" schedule: the schedule begins timing the next interval as soon as the

previous one elapses; if the reinforcer for the previous interval has not

been collected before the next "setup," it is lost.

IRTp IRTd VIp VId Linear Killeen

1 1.0010 14.80 15.74 15.80 15.18

2 2.0170 14.63 16.77 16.65 15.66

4 4.0201 14.86 18.68 18.84 16.94

8 7.9995 15.06 23.32 23.05 19.41

16 16.3654 15.20 30.54 31.56 24.82

32 32.2056 15.07 47.17 47.28 36.52

64 64.6739 15.04 81.72 79.71 65.56

Once again the linear function wins. But what if the rat does NOT respond

with constant probability in each successive time interval dt? In the next

simulation, the limited hold schedule was retained but the rat now produces

a normally distributed series of IRTs with mean as specified and standard

deviation = 0.25*mean.

IRTp IRTd VIp VId Linear Killeen

1 1.0064 15.30 15.80 16.30 15.81

2 2.0040 14.01 15.11 16.02 15.04

4 4.0033 14.72 16.58 18.87 16.82

8 8.0109 15.03 19.88 23.31 19.65

16 16.5814 14.84 24.43 30.89 24.28

32 31.9848 15.48 36.68 47.47 36.62

64 64.9894 15.57 66.66 80.55 66.00

Now the simulation clearly favors Killeen's formula. It is evident that

BOTH formulas provide good summaries of the VI schedule function UNDER

DIFFERENT DISTRIBUTIONS of RESPONDING. The linear formula does well at

predicting VId when the rat responds with constant probability at the given

average IRT, whereas Killeen's formula does well at predicting VId when the

rat responds at the given average IRT but responding varies according to a

normal distribution around this mean, with a standard deviation proportional

to the average IRT. Other distributions of responding will produce yet

different results.

Regards,

Bruce