Gain of interval feedback function

[From Bruce Abbott (960214.1730 EST)]

Control systems (e.g., rats and pigeons) that must operate through an
environmental feedback function provided by interval schedules of
reinforcement have to contend with a loop gain that depends on the rate of
the operant. If the operant is assumed to be emitted at a steady rate (not
too unreasonable on variable interval schedules), the feedback function can
be modeled as follows:

(1) R = 1/(1/B + VI),

where R is the rate of reinforcement in reinforcers/second, B is the rate of
the operant in responses/second, and VI is the average length of the
programmed intervals (in seconds) on the VI schedule. The gain g of this
function is the ratio of reinforcement rate to response rate, or R/B. This
can be obtained by dividing both sides of Equation 1 by B to yield:

(2) g = 1/(1 + VI*B).

As response rate approaches zero, the gain can be seen to approach 1.0; as
response rate increases, the denominator approaches VI*B and the gain
approaches zero. If B is given as an interresponse time (IRT = 1/B), then
Equation 2 becomes:

(3) g = 1/(1 + VI/IRT).

As this form makes clear the gain of the VI feedback function will be
exactly 0.5 when the IRT equals the size of the average scheduled interval.

The loop gain of a control system working through an interval schedule
function will be the product of all the gains around the loop, including the
gain of the schedule function. If the output gain = 100 and gains other
than this and the schedule function gain are 1.0, then the loop gain of the
system equals 100g. The table below gives loop gains for several rates of
responding on various VI schedules.

            B VI Gain VI Gain VI Gain
        5.000 15 1.315 60 0.332 240 0.083
        1.000 6.250 1.639 0.415
        0.100 40.000 14.286 4.000
        0.010 86.957 62.500 29.412
        0.001 98.522 94.339 80.645

        5.000 30 0.663 120 0.166 480 0.042
        1.000 3.226 0.826 0.208
        0.100 25.000 7.692 2.041
        0.010 76.923 45.455 17.241
        0.001 97.087 89.286 67.568

With an output gain of 100, the loop gain will fall to 1.0 at the following
response rates:

          VI B Resp/min
          15 6.600 396.0
          30 3.300 198.0
          60 1.650 99.0
         120 0.825 49.5
         240 0.413 24.8
         480 0.206 12.4

At this value the effect of a disturbance is reduced only by half and with
no disturbance, the perceptual signal has a magnitude only half that of the
reference signal. The system is rather insensitive to disturbance and a
poor controller of the perceptual signal. And this assumes that the gain in
the remainder of the loop = 100; with lower values the loop gain approaches
1.0 at even lower rates of behavior. For example, at an output gain of 10
on VI 15 the value is 0.6 resp/sec or 36 resp/min, and on VI 480 it is 0.19
resp/sec or 1.125 resp/min.