VI Feedback functions

[From Bruce Abbott (941024.1740 EST)]

Bill Powers (941021.0930 MDT)

I've been pondering the variable (or random) interval schedule,
constant-probability type, looking for a way to express the average
relationship between behavior rate and reinforcement rate. The purpose
is to see whether the same "matching" problem occurs with the VI
schedule as with the FR schedule.

It's obvious that at very low behavior rates, the reinforcement rate
will be proportional to behavior rate, and that at very high behavior
rates the reinforcement rate will tend to a constant value. I sort of
guessed that the relationship might be of the form r = 1 - exp(-ab)

Bill, the following equation was provided by Prelec and Herrnstein (1978), who
modified it from Baum (1973):

(1) R = ------,
              B + ar

where R is the delivered rate of reinforcement, B is the rate of responding, r
is the programmed rate of reinforcement, and a is a constant that is supposed
to be around 0.5 for regular responding and 1.0 for random responding (Poisson
model, I believe). Manipulating this equation, I get

(2) VId = VIp + aIRT,

where VId is the delivered average schedule interval, VIp is the programmed
average schedule interval, and IRT is the average interresponse time of the
subject. I stimulated responding using our constant-probability models for
schedule and subject, holding VIp = 30-s (or in a separate run, VIp = 15-s)
and varying IRT from 1 to 512 s by successive doubling. Regressing VId on
observed IRT gave

(3) VId = 30.0 + 0.985*IRT + 1.9,
(4) VId = 15.0 + 0.994*IRT + 1.4,

which are equivalent within statistical error to equation (2). Thus the
delivered average reinforcement interval is a simple linear function of the
programmed average interval, and equation (1) provides a reasonable function
for delivered reinforcement rates, at least for a constant-probability pigeon.
Of course, other distributions of responding would alter this relationship.

JEAB's "special issue on behavior dynamics" (May 1992) includes an interesting
article by Bill Palya that examines the fine structure of responding on simple
schedules, including the VI. Figure 3 shows that a bird responding on a VI
60-s constant probability schedule produced a fairly normal-looking
distribution of interresponse times with a mean of 1.05 s and standard
deviation of 0.28 seconds. The post-reinforcement pauses followed a
positively skewed distribution with a mean of 1.84 s and a standard deviation
of 1.16 s. An interesting feature was the pigeon's tendency to produce
responses at 0.35 s intervals or multiples of them, although this tendency was
less in evidence on the VI schedule than on some of the others (Figure 5).
Palya reported that some behavior (e.g., abortive keypeck) tended to occur at
0.35 s (or some multiple of it) when a keypeck did not occur, suggesting that
the pigeon's behavior was entrained to some kind of clock.

The same issue of JEAB also contains an article by William Baum entitled "In
search of the feedback function for variable-interval schedules," in which
Baum examines several possibilities and concludes than none are completely

Bill Powers (941022.0815 MDT)

This is clearly control behavior on the part of the experimenter (as
well as the pigeon). For some reason (which perhaps you can explain),
the experimenter did not want to see the pigeon using a simple
alternation as a way of providing rewards for itself

I think the experimenters viewed simple alternation as a kind of "degenerate"
solution which prevents differences in schedule parameters from influencing
performance. If you have one variable (immediate reinforcement of switching)
overwhelming the effects of another (relative schedule values), and you want
to study the latter, you eliminate the former.

It seems to me that EAB researchers, like those in the rest of
psychology, don't take each others' findings very seriously. Each
investigator, or group, starts essentially from scratch, ignoring all
the variables that others have tested and not even reporting on their
status. What was the size of the cage and the lighting? Were there
potential mates or rivals within eye, ear, or nose range? How much work
was required to depress the key or bar? What was the nutritive content
of the food pellets actually swallowed? What was the free-feeding rate
of ingestion of the type of reward used for each individual? How were
the animals handled? Since all these factors have been studied, and many
more, and have been found to have enough effect to warrant a
publication, why were these factors not at least noted and recorded as
part of the experimental conditions?

In asking why they aren't, you beg the question of whether they were. The
short answer is that, insofar as the researcher understood them to have a
bearing on the outcome of the experiment in question, they were. Factors such
as cage size, lighting schedule, housing conditions, size of the test chamber,
lights, force required to depress the key or bar, type of food, amount of food
or duration of access to food, level of food deprivation, species, sex,
strain, age, and source of animal, handling, and many other descriptions
typically appear in the methods sections of EAB articles. While it would be
impossible to list the state of every potentially important variable, those
thought to be relevant to the particular experiment are given (although they
may not include conditions thought potentially important from a PCT

Isn't this begging the question a bit? When you say they "tend to do
well" on VI-VI schedules, what this tells me is that they don't show
matching behavior -- some do, perhaps, but some don't, so the matching
law isn't a law at all. If you have a real law of behavior, it's got to
work all the time, with essentially all organisms to which it applies.
And why on earth would one talk of a "matching law" if "animals do not
display matching under most two-key conditions"? To me, that suggests
that this law has in fact been disproven: it's not a law. Why are we
even talking about it?

As I pointed out in an earlier posting, the matching law as I understand it is
purely descriptive. Matching is observed on concurrent VI-VI schedules, but
the data show the same typical unsystematic deviations from the ideal line
that any small sample of behavior will show owing presumably to fluctuations
in variables not completely under experimental control. To the extent that it
is a valid summary of the experimental results, it will apply under certain
specified conditions, in the same way that Boyle's law describes the
relationship between a gas's pressure and temperature, but only when it is
confined to a specific volume.

You say you don't work with pigeons. What _do_ you work with? Getting
the kind of data needed for PCT modeling from the literature has proven
a frustrating job; what's needed is the raw data, a record of every key-
press and every reward against time, as well as observations of what the
animal was really DOING all of the time. If you're doing experiments,
all this data must be available. Do you have anything (simple) we could
start with?

All the professional research I have conducted used rats, and the majority of
these studies looked at behavior in aversive situations and looking at very
global measures (e.g., session averages). So I'm afraid I don't have anything
I can just pull out of a drawer.

You must have some data we could work with, or perhaps you could even devote
some time to doing a real PCT experiment for a simple situation. What
about it?

I'm planning to set up some simple schedules in order to collect the type of
data we are looking for here, but that will take time as I will have to get
the project approved by our local animal care and use committee, then order
the rats, receive them, and bring them down to 80% ad libitum weight. It will
be several months before we will have any data from this source. Meanwhile
I'll see what I can do to round up some of those other data you are interested
in. I've done the ol' measure-the-figure routine myself (once after writing
to the authors FOUR TIMES and never receiving a reply of any kind), so I fully
agree that we do not want to be doing that if we can avoid it.

By the way, I've decided to try reproducing some of the published PCT models
as a way to test whether I really understand how to construct PCT simulations
properly. (Rick Marken's 1986 JEP:HPP model of coordinated action looks like
a nice place to start.) With this VI-VI simulation I feel as though I'm being
asked to fly before I'm sure I can walk: it may get quite complicated before
we're done. And I agree with you that perhaps we should start by modeling a
simpler situation, such as FR or FI schedule performance.