[From Bruce Abbott (960214.1730 EST)]

Control systems (e.g., rats and pigeons) that must operate through an

environmental feedback function provided by interval schedules of

reinforcement have to contend with a loop gain that depends on the rate of

the operant. If the operant is assumed to be emitted at a steady rate (not

too unreasonable on variable interval schedules), the feedback function can

be modeled as follows:

(1) R = 1/(1/B + VI),

where R is the rate of reinforcement in reinforcers/second, B is the rate of

the operant in responses/second, and VI is the average length of the

programmed intervals (in seconds) on the VI schedule. The gain g of this

function is the ratio of reinforcement rate to response rate, or R/B. This

can be obtained by dividing both sides of Equation 1 by B to yield:

(2) g = 1/(1 + VI*B).

As response rate approaches zero, the gain can be seen to approach 1.0; as

response rate increases, the denominator approaches VI*B and the gain

approaches zero. If B is given as an interresponse time (IRT = 1/B), then

Equation 2 becomes:

(3) g = 1/(1 + VI/IRT).

As this form makes clear the gain of the VI feedback function will be

exactly 0.5 when the IRT equals the size of the average scheduled interval.

The loop gain of a control system working through an interval schedule

function will be the product of all the gains around the loop, including the

gain of the schedule function. If the output gain = 100 and gains other

than this and the schedule function gain are 1.0, then the loop gain of the

system equals 100g. The table below gives loop gains for several rates of

responding on various VI schedules.

B VI Gain VI Gain VI Gain

5.000 15 1.315 60 0.332 240 0.083

1.000 6.250 1.639 0.415

0.100 40.000 14.286 4.000

0.010 86.957 62.500 29.412

0.001 98.522 94.339 80.645

5.000 30 0.663 120 0.166 480 0.042

1.000 3.226 0.826 0.208

0.100 25.000 7.692 2.041

0.010 76.923 45.455 17.241

0.001 97.087 89.286 67.568

With an output gain of 100, the loop gain will fall to 1.0 at the following

response rates:

VI B Resp/min

15 6.600 396.0

30 3.300 198.0

60 1.650 99.0

120 0.825 49.5

240 0.413 24.8

480 0.206 12.4

At this value the effect of a disturbance is reduced only by half and with

no disturbance, the perceptual signal has a magnitude only half that of the

reference signal. The system is rather insensitive to disturbance and a

poor controller of the perceptual signal. And this assumes that the gain in

the remainder of the loop = 100; with lower values the loop gain approaches

1.0 at even lower rates of behavior. For example, at an output gain of 10

on VI 15 the value is 0.6 resp/sec or 36 resp/min, and on VI 480 it is 0.19

resp/sec or 1.125 resp/min.

Regards,

Bruce