[From Bill Powers (951029.1430 MST)]
Bruce Abbott (951028.1605 EST) --
Great! Do I have enought credits to graduate? (;->
Graduate? Sorry, we haven't got to 401 yet, not to mention creating any
Masterites or Doctoroids in PCT. Then there's the matter of that overdue
library book...
I've decided to simply videotape the experimental sessions rather
than attempting to automate detection of the rat's position, etc
OK -- also a good source of material for others who think up new things
to look for after it's all over. Accurate time recording on the video-
tape is a must, either as a clock in the field of view or internally
generated. For synchronizing with the computer data, we need single-
frame resolution. A frame counter?
That way we can determine _everything_ the rat does in this
situation, which should provide a good basis for constructing that
"do everything" model.
Right, I like it.
···
=============================================
Herrnstein was no fudger of data.
I hope I made the conditional clause plain.
RE: interval schedules
... the observations, which indicated that the pigeons, to a fairly
close approximation, matched their relative rates of pecking at the
two keys to the relative rates of reinforcment scheduled on those
keys:
P1 R1
(1) ------- = -------,
P1 + P2 R1 + R2
Continuing,
P1*(R1 + R2) = R1 * (P1 + P2)
P1*R1 + P1*R2 = R1*P1 + R1*P2
P1*R2 = R1*P2
(2) P1/R1 = P2/R2
When you reduce the original equation (1) to its simplest form (2), you
find that the animals behaved so as to make the number of behaviors per
reinforcement equal on the two keys. This is not matching the rates of
pecking to the scheduled reinforcements (by which I take it you mean 1/I
with I being the mean interval).
Let me use fixed intervals as an approximation, here. Where I say
"behavior rate" the meaning is also "total number of behaviors in a
fixed-duration sample."
The rate of behaving at which maximum reinforcement is obtainable on
either key is b1' or b2', equal to 1/I1 and 1/I2 respectively.
(br = behaviors per reinforcement, b/r)
br1 = 1 (b1 <= b1') or
br1 = b1/b1' (b1 > b1')
br2 = 1 (b2 <= b2') or
br2 = b2/b2' (b2 >= b2')
Is there any combination of behavior rates that will yield rp1 = rp2?
Here is one: b1 < b1' and b2 < b2'
Any others? Obviously, b1/b1' = b2/b2'. In other words, the rate of
pressing on each key is the same proportion above the limiting rate of
pressing. So clearly it is always possible to vary the behavior rates to
equalize the return per press from two keys. Both behavior rates must be
above their respective critical rates 1/I1 and 1/I2. If both behavior
rates are below the critical rates, the condition is satisfied for any
behavior rates since b/r in both cases is constant at 1. It is not
possible to satisfy the condition if one behavior rate is below and the
other is above the critical rate.
If for some reason the total behavior rate is constant at B, then
b2 = B - b1.
b1/b1' = (B - b1)/b2'
From this we get
b1 = B*b1'/(b2' - b1')
(b2' > b1')
b2 = B*b2'/(b2' - b1')
The reinforcement rates on the two schedules are just b1' and b2', so we
can check by substituting into the (simplified) equation 2 above:
B1/R1 = B2/R2 or
B*b1'/(b2'-b1') B*b2'/(b2'-b1')
---------------- = --------------
b1' b2'
B = B (check).
---------------------------------------------
Sorry if this is just the long way around for saying what Herrnstein
meant by "matching," but I had to go through it for myself. The way I
would have described this would be to say that the birds apportion their
pecking so that they use the same number of pecks per reinforcement on
both keys -- the same cost-to-benefit ratio.
----------------------------------------------
What I said about ratio schedules still holds, although I'll take your
word that the original matching hypothesis concerns interval schedules.
You say
For example, if the pecking rate on key 1 is 200
pecks/min, the pecking rate on key 2 is 0 pecks/min, and the
schedule on key 1 is FR 50, then
200 4
------- = -----; thus 1 = 1 and matching holds.
200 + 0 4 + 0
This is just a mathematical accident. If the pressing rate on key 2 had
not been exactly zero, your formula wouldn't have worked. When
Herrnstein said that this is a trivial result, he was right. When you
simnplify this equation, you get b1/r1 = b2/r2, and if both
b2 and r2 are zero, you get b1/r1 = 0/0, an indeterminate form. Your
so-called matching formula will work for ANY value of the second ratio,
1 or 1,000,000, as long as there are no keypresses on the second key.
That hardly sounds like "matching" to me, no matter what you mean by
matching.
--------------------------------
Of course, the ratio analysis in terms of _obtained_ reinforcement
doesn't make much sense
I still don't think you get my point about ratio analysis. On a ratio
schedule there can never be any difference between "scheduled" and
"obtained" reinforcement rates. In fact, no rate of reinforcement can be
"scheduled" in advance; the rate of reinforcement depends strictly on
the behavior rate, which is unpredictable (except from previous
experience). All that the schedule determines is the _ratio_ of
behaviors to reinforcements.
... unless one views the stable end-point of performance as
resulting from a dynamic process in which response rates affect
reinforcement rates, which in turn affect response rates. (I'm
describing this sequentially but it's a simultaneous, closed-loop
process.) One can then view the outcome (exclusive preference) as
the product of an unstable (positive feedback) process that drives
relative response and reinforcement rates toward exclusive
responding on the richer schedule.
The exclusive preference is much easier to describe as a control process
that is driving the reinforcement rate toward a reference level that is
higher than the maximum attainable reinforcement rate. If, in a two-key
ratio experiment, behavior ends up on one key exclusively, that is
because that results in the highest possible reinforcement rate (as I
showed that it does, a few posts ago). If the reference level for
reinforcement rate were lower than that maximum, you would not see
exclusive preference. You could achieve this by making the reward size
large enough so it is not necessary to get the maximum possible
reinforcement rate to match the reference level.
Remember, "matching" is an attempted summary description of the
observations, not an explanation. One can say that pigeons match
(trivially) on concurrent ratio schedules _because_ exclusive
responding on the richer schedule maximizes the average rate of
reinforcement.
The fact that a variable reaches a limit does not mean that it is being
maximized (in the sense of hill-climbing). The only reason the
reinforcement rate doesn't go higher is that the allocation of behavior
to one key can't go higher than 100%. If the maximum achievable rate of
reinforcement were above the reference level, the allocation of behavior
would not go all the way to the limit, because zero error would be
achieved before it did.
Meanwhile, have you taken a look at the McSweeney data? If we need
to communicate with Fran, I received an e-mail note from her a
couple of weeks ago informing me that she's finally given in and
gotten her own e-mail address.
Not yet. I'm getting frazzled from trying to keep up with the net and
also struggling to find a way to write the book Fred Good wants. I'm
going to have to cut back on something, and the McSweeney data is it for
now. I've been working on e-mail on and off since dawn, and it's now
dark again, and I haven't put in a lick on the book, not to mention
putting screens away for the winter. Mary's beginning to wonder who that
man in the basement is. I've got to acquire some self-control here.
-----------------------------------------------------------------------
Best,
Bill P.