[Bruce Abbott (941016.1530 EST)]
Bill Powers (941015.1335 MDT) --
I agree that a joint simulation project is a good idea. Let's work it
out here on the net, because I'm sure there are others who would be
interested.
Great! I've actually WRITTEN programs that execute VI (and other) schedules
of reinforcement and collect the data, so if you like we can use one of those
as the basis for the environmental side of the simulation. Actual schedules
usually provide reinforcement on a "constant probability" basis: in theory the
probability of a reinforcer becoming "set up" in a given interval is constant
regardless of the number of intervals since last reinforcer delivery. That
is, p(Reinf|T) = K, where T is the elapsed time since last delivery and K is a
constant. Practical schedules that approximate this property have intervals
that range from a few seconds (e.g., 2) to a few minutes (the actual values
depending on the average interval length and number of intervals used).
How is a variable-interval schedule with a nominal interval of t
defined? See program below for my guess.
One way to create a constant-probability schedule is to sample a random number
(say, between 0 and 1) at equal time-intervals and set up the reinforcer if
the number is smaller than some criterion value, K. For example, if the
computation were being performed once each second, setting K = 1/60 = .017
will provide one reinforcement each minute, on average. Some upper limit must
be imposed to prevent intervals from becoming overly long. There are other
methods; I'll provide an example using pre-defined intervals when I get a
chance to retrieve one of my programs from the lab.
I don't think you can say that a schedule of reinforcements sets up any
particular average delivery rate, since the delivery rate depends just
as much on the rate of pecking as on the schedule.
Yes, you are correct. (I ignored this issue in my earlier response as it
seemed peripheral to the argument.) If response rates are high enough,
reinforcements will be delivered at very close to the scheduled rate and the
correlation between response rate and reinforcement rate on a given schedule
will be nearly zero over time. This is often the case. Matching-law studies
in JEAB usually include an appendix full of data which include the "obtained"
reinforcement rate corresponding to a given "programmed" reinforcement. Of
course, it is precisely this co-dependency that the traditional linear
analysis neglects. Researchers will sometimes note, in a study where obtained
and programmed rates differed, that matching occurred to the obtained rates
rather than to the programmed rates. This is why the "matching" that occurs
trivially on VR-VR schedules is held to "support" the matching law: those guys
are using obtained rather than scheduled rates on the reinforcement side of
the equation.
I don't see how the variable-interval schedule can program the number of
reinforcement opportunities, since the number of such opportunities
depends on the distribution of pecking as well as on the schedule.
Although the schedule does not guarantee a given rate of reinforcement, it
does specify the probability that a reinforcer will become available after a
given interval of time. Such opportunities can be lost (the pigeon may fail
to "collect" the reinforcer after it becomes available) but this in no way
negates the fact that they were there.
Therefore if we hypothesize that the bird pecks at a constant high rate
and switches from one key to the other at regular intervals, we can
explain why there is matching in a variable-interval 2-key experiment.
Empirically, both relative response rates and relative time spent pecking on a
key (time spent being measured as time between switches) yield matching,
although rates tend to fit with less noise. I may be missing something, but I
think this is inconsistent with your analysis. Perhaps it has something to do
with your assumption of a rectangular distribution of schedule intervals vs
the constant probability distributions actually used. I'll have to give that
some more thought when I get some time to do so.
My own schedule is getting hectic (which is frustrating for me as I am
definitely having more FUN doing this, and learning something to boot!), so
there will be a delay of at least several days before I can respond with some
programming. Meanwhile, I hope this reply has at least addressed those points
you were unclear about.
Regards,
Bruce