VI schedule function(s)

[From Bruce Abbott (960221.0855 EST)]

Bill Powers (960220.2300 MST)

When you measure behavior in VI schedules with a closed loop system, the
closed loop will modify the actual distribution induced by the irregular
schedule to produce a different apparent distribution. If you're just
trying to characterize the feedback function (which I think is the main
point), all you need to do is supply various steady rates of behavior
(variance of zero) and measure the mean rate and distribution of
reinforcement, for each VI schedule. That will give you the true
character of the feedback function itself.

Using a steady response rate would be rather uninformative, I believe. For
example, imagine using a steady rate of 1 resp/s, with dt = 0.1 s. A
response will be generated every tenth iteration. Meanwhile the VI interval
timer is decrementing toward zero; when it reaches zero a reinforcer is set
up. It may be that this coincides with one of those MOD 10 responses, in
which case the reinforcer gets delivered immediately. Or it could happen on
the next iteration, or the next, etc, out to nine iterations. Since the VI
setup is random with respect to the response generator, the probability that
setup will occur on any given iteration is constant, yielding a rectangular
distribution of delays between setup and collection with a mean of 10/2
iterations or 0.5 s. This, I believe, is the limit under a zero-variance
IRT: VId = VIp + 0.5 IRT.

If you set up the rat's behavior open-loop as an independent random
variable with a particular mean and distribution, and run that through
the schedule, you'll get the product of two stochastic variables. This
may be a good way to measure the effective feedback function, assuming
that the rat's variability in rate of pressing is due entirely to
irreducible noise in the rat's output. However, this is a shaky
hypothesis when you are using a variable schedule, because even with a
perfectly regular rat function (with no internal noise), the statistical
variations in the feedback function will lead to random-appearing
variations in the system output. Unless you can show that these same
variations would be present with a perfectly regular rate of
reinforcement, you can't assign them to behavior as if they were an
inherent part of the rat's actions. Nor is the resulting product
representative of the feedback function.

I agree with your main point, which is that variations in the rat's output
are likely to be as much a product of variations in the schedule feedback
function as intrinsic to the rat. My interest in assessing how the feedback
function changes as a function of response distribution came about initially
because a plot of IRT versus VId for McSweeney's data revealed a rather
"scattered" set of points. When I superimposed Killeen's function and the
linear function on these points, all but one point fell below Killeen's
line; the general trend followed the linear function, but with quite a bit
of variability. That got me to wondering how the function-lines actually
vary with response distribution. Normal variation around the mean IRT
(Killeen) and an exponential distribution of IRTs around the mean IRT
(linear) cover a good range of different possibilities. With less irregular
data it might be possible to work backward and infer something about the
general distribution of IRTs from the obtained schedule function.

I kept meaning to bring this up way back in the battle over the E. coli
model, and kept forgetting. By generating behavior using a probabilistic
output function (constant probability of a press per unit time, etc.),
you are making a claim not only about observed mean rates, but about
observed variances in the behavior rate. If the actual behavior is not
as variable as the model's behavior, a different means of generating the
behavior has to be found. Your model of E. coli showed much too much
random variation in the measured dependence of tumbling delay on rate of
change of concentration, as measured by Koshland using perfusion of
tethered bacteria. The actual response was quite regular.

Good point, and an excellent test of model fit; one has to be as concerned
with matching variation of model and data as with matching the changing
values of the cv. But of course, my e. coli model was not intended as a
serious model of real e. coli.

What news on prospects for getting the rat experiments started?

I will be ordering the rats this week.