Matching data

[Bruce Abbott (990515.0915 EST)

Rick Marken (990514.1030) --

Me:

Now what I'd like to see is a table of values for a number of
selected response rates for, say, VI 15-s, VI 30-s, VI 60-s,
and VI 120-s. Use response rates of 4, 2, 1, .5, and .25
responses/sec. Thanks.

Ye:

Here are some data. They are not for the schedules you specify
but they are for the schedules used by Herrnstein (in his 1962
paper that is available at your site).

Thats nice. Now, is there some reason I can't have the data I asked for?
It should be a simple matter for you to plug the response and reinforcement
rates into your VI simulation and generate the relevant numbers.

All data (the output
of the simulation) was collected at an average inter-response
interval (IRI) of 3 secs. This value gave response rates on each
key that were in the range of those reported by Herrnstein.

The IRTs (inter-response times) you report are in the neighborhood of 7.8
sec average on each key. This is over twice the value you said you used.

Regards,

Bruce

[From Rick Marken (990516.1630)]

Me:

Here are some data.

Bruce Abbott (990515.0915 EST)

Thats nice. Now, is there some reason I can't have the data
I asked for?

I set up the simulation to do concurrent VI schedules. But I
think I can get you what you want by just making the probability
of pressing one key very low. So what data would you like?
Reinforcement rates delivered by the different schedules
for the different response rates?

The IRTs (inter-response times) you report are in the neighborhood
of 7.8 sec average on each key. This is over twice the value you
said you used.

Yes. The value of IRT I said I used was the value of overall
IRT -- the IRT between each response, regardless of the key.
The data I report in the table are the measured IRTs, which are
about twice the overall IRT since the program is measuring the
rate on a key in terms of the time between responses to that key.

By the way, I don't see any evidence that the organisms in the
Herrnstein study are controlling any variables at all (with the
possible exception of the relative rate of responding to rate of
reinforcment at the two keys, though it's not clear _why_ the
organisms would control this variable since it seems to get them
nothing that would see to be important, such as more food or
less work).

The Herrnstein study seems like a good example of research done
without the slightest idea that organsims _might_ be controlling
variables, like their own food input, for example. I'm thinking
that this matching data is the kind of data we might want to revist
_after_ we have done some tests to determine which of the variables
in operant studies organisms (like pigeons) actually do control.

Best

Rick

···

---
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken/

[From Bruce Abbott (990516.2045 EST)]

Rick Marken (990516.1630) --

I set up the simulation to do concurrent VI schedules. But I
think I can get you what you want by just making the probability
of pressing one key very low. So what data would you like?
Reinforcement rates delivered by the different schedules
for the different response rates?

I believe that your _first_ simulation was a single VI schedule. You
reported several observations from that simulation, such how the minimum
interval affected the obtained rate of reinforcement at a given response
rate. The data I asked for were the rates of response and reinforcement for
IRTs of 0.5, 1.0, 2.0, 4.0, 8.0, and 16.0 seconds for VI 15-s, 30-s, 60-s
and 120-s.

The IRTs (inter-response times) you report are in the neighborhood
of 7.8 sec average on each key. This is over twice the value you
said you used.

Yes. The value of IRT I said I used was the value of overall
IRT -- the IRT between each response, regardless of the key.
The data I report in the table are the measured IRTs, which are
about twice the overall IRT since the program is measuring the
rate on a key in terms of the time between responses to that key.

I get response rates _on each key_ varying from about 1000/hr (3.6 sec IRT)
for the largest-interval schedule to about 4500/hr (0.8 sec IRT) for the
smallest-interval schedule. On concurrent VI 3-min VI 3-min a value of
around 2200/hr (1.6 sec IRT) would be typical. In no case is there an
average IRT of 7.8 sec.

By the way, I don't see any evidence that the organisms in the
Herrnstein study are controlling any variables at all (with the
possible exception of the relative rate of responding to rate of
reinforcment at the two keys, though it's not clear _why_ the
organisms would control this variable since it seems to get them
nothing that would see to be important, such as more food or
less work).

The pigeons seem to be working fairly hard to produce those accesses to
grain; in the absence of responding the number of grain accesses would be
zero, so it would appear that by responding the pigeons are greatly
increasing the time they spend consuming grain. I think it's a fair guess
that the pigeons peck at the keys for the opportunity to consume grain.

The Herrnstein study seems like a good example of research done
without the slightest idea that organsims _might_ be controlling
variables, like their own food input, for example. I'm thinking
that this matching data is the kind of data we might want to revist
_after_ we have done some tests to determine which of the variables
in operant studies organisms (like pigeons) actually do control.

I doubt that Herrnstein or anyone else has missed the fact that the pigeons
are working for access to grain. As for controlling for matching, I doubt
that the pigeons are doing that. In my judgment it is more likely that
matching emerges as a side-effect of the pigeon's attempts to control
something else -- e.g., minimizing the delay to food access. That,
basically, is what Bill's "ecoli" proposal amounts to.

One step we can take now is to evaluate the environmental feedback function
for VI schedules under various assumed conditions. You've created a VI
schedule with arithmetic intervals and a similar IRT distribution. I've
been looking at what happens when both are constant-probability
distributions. It looks like the results are very similar, but I'd like to
see your data on this to confirm.

Regards,

Bruce

[From Rick Marken (990517.0930)]

Bruce Abbott (990516.2045 EST)]

I believe that your _first_ simulation was a single VI schedule.

Right. My assumptions about how VI schedules are implemented
were wrong so I rewrote the program to do concurrent schedules
correctly. But I can get you the data you want.

The data I asked for were the rates of response and
reinforcement for IRTs of 0.5, 1.0, 2.0, 4.0, 8.0, and 16.0
seconds for VI 15-s, 30-s, 60-s and 120-s.

OK. Here it is. These data include a 5 sec minimum feeding time.
The actual IRT used in the program was one that give a rate
of responding on a VI 15 - s schedule that was close to the
desired IRT (eg. 1/observed rate = IRT; 1/1.94 = .51)

       IRT .5 1.0 2.0 4.0 8.0 16.0

VI 15-s Pi 1.94 .96 .5 .242 .123 .062
         ri .063 .064 .061 .061 .060 .031

VI 30-s Pi 2.09 1.05 .546 .264 .134 .062
                 .033 .034 .032 .032 .031 .031

VI 60-s Pi 2.19 1.09 .57 .275 .140 .064
                 .017 .016 .016 .016 .017 .017

VI 120-s Pi 2.23 1.12 .58 .279 .142 .065
                 .008 .008 .008 .008 .008 .008

On concurrent VI 3-min VI 3-min a value of around 2200/hr
(1.6 sec IRT) would be typical. In no case is there an
average IRT of 7.8 sec.

I must have picked the wrong IRTs. It looked like my results
were in the range of what Herrnstein was getting. But I might
have made errors going from response rate to IRT and back. I'll
try again. Do the results above seem right?

Me:

By the way, I don't see any evidence that the organisms in
the Herrnstein study are controlling any variables at all

Bruce:

The pigeons seem to be working fairly hard to produce those
accesses to grain;

All that shows is that the pigeons peck at keys. It doesn't
show that they are _controlling_ anything; and it certainly
doesn't show _what_ they are controlling (if they are controlling).

My guess is that the pigeons are _trying_ to control the amount
or rate of food and aren't coming close to succeeding. So the
birds are probably reorganizing like crazy; that means there is
a large random component to the birds' behavior.

What I think we are seeing in these VI and other scheduling
studies is the effect on reinforcement rate of the responding
produced by animals who have little control over the variables
that matter most to them (nutritional variables). The observed
relationships between the bird's essentially random responding
and reinforcement rate is simply the feedback function determined
by the scheduling apparatus.

Since there is no evidence that _any_ variables are under control
in these scheduling experiments, it seems to me like any control
model of such behavior is just going to be a fairly arbitrary
shot in the dark. We don't know, for example, what to use as the
variable the model is (unsuccessfully) trying to control. That's
why I mentioned the lack of evidence of controlled variables;
without evidence that variables are under control it seems to me
to be a rather arbitrary exercise to apply a control model to
explain the pigeons' behavior in these experiments. It's certainly
possible to build a control of of such behavior; but it's also
possible (and equally ridiculous, I think) to build a control
model of falling objects.

Me:

The Herrnstein study seems like a good example of research done
without the slightest idea that organsims _might_ be controlling
variables, like their own food input, for example.

Bruce:

I doubt that Herrnstein or anyone else has missed the fact
that the pigeons are working for access to grain.

If that's true, then why does Herrnstein or anyone else think
that the pigeons are "working" (responding at a particular rate)
_as a result of_ (rather than _for_) access to grain (reinforcement).
If neither Herrnstein nor anyone else has missed the fact that
the pigeons are working for access to grain, then why haven't
these folks done studies to determine the aspects of the grain
the pigeons are trying to access (controlling for)?

Best

Rick

···

---
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken

[From Bruce Abbott (990518.0915 EST)]

Rick Marken (990517.0930) --

Bruce Abbott (990516.2045 EST)

The data I asked for were the rates of response and
reinforcement for IRTs of 0.5, 1.0, 2.0, 4.0, 8.0, and 16.0
seconds for VI 15-s, 30-s, 60-s and 120-s.

OK. Here it is. These data include a 5 sec minimum feeding time.
The actual IRT used in the program was one that give a rate
of responding on a VI 15 - s schedule that was close to the
desired IRT (eg. 1/observed rate = IRT; 1/1.94 = .51)

      IRT .5 1.0 2.0 4.0 8.0 16.0

VI 15-s Pi 1.94 .96 .5 .242 .123 .062
        ri .063 .064 .061 .061 .060 .031

VI 30-s Pi 2.09 1.05 .546 .264 .134 .062
                .033 .034 .032 .032 .031 .031

VI 60-s Pi 2.19 1.09 .57 .275 .140 .064
                .017 .016 .016 .016 .017 .017

VI 120-s Pi 2.23 1.12 .58 .279 .142 .065
                .008 .008 .008 .008 .008 .008

On concurrent VI 3-min VI 3-min a value of around 2200/hr
(1.6 sec IRT) would be typical. In no case is there an
average IRT of 7.8 sec.

I must have picked the wrong IRTs. It looked like my results
were in the range of what Herrnstein was getting. But I might
have made errors going from response rate to IRT and back. I'll
try again. Do the results above seem right?

Thanks -- this is what I was looking for. These appear to be in the right
ballpark, although I'm a bit surprised that there is not a larger decrease
in reinforcement rate at the higher response rates. On how large a sample
are these results based? What is the time-increment (dt)? I've found that
with constant-probability schedules even samples as large as 10 hours (at a
dt of .1 second) produce standard errors on the order of several seconds
across a series of 5 such samples.

Regards,

Bruce

[From Rick Marken (990518.0920)]

Bruce Abbott (990518.0915 EST)

Thanks -- this is what I was looking for. These appear to be
in the right ballpark, although I'm a bit surprised that there
is not a larger decrease in reinforcement rate at the higher
response rates.

Do you mean higher inter response times (IRTs)?

On how large a sample are these results based?

A total of 500 food deliveries. Depending on schedule and
rate this could be from 1000 pecks (IRT = 16 sec) to over
18000 pecks (IRT = .5 sec).

What is the time-increment (dt)?

I think it's essentially 0. I don't use time increments;
I just generate pecks and trigger rewards after the
appropriate (real valued) intervals.

I've found that with constant-probability schedules even
samples as large as 10 hours (at a dt of .1 second) produce
standard errors on the order of several seconds across a
series of 5 such samples.

I haven't calculated standard errors but they seem to be
very small. When I re-run the simulation I get results that
are very similar each time. Here are the results of 7 runs
on VI 30 - s with an IRT of 4 s.

Pi .262 .264 .262 .261 .263 .262 .263
ri .033 .031 .033 .034 .032 .033 .033

Best

Rick

···

--
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken

[From Bruce Abbott (990518.1310 EST)]

Rick Marken (990518.0920) --

Bruce Abbott (990518.0915 EST)

Thanks -- this is what I was looking for. These appear to be
in the right ballpark, although I'm a bit surprised that there
is not a larger decrease in reinforcement rate at the higher
response rates.

Do you mean higher inter response times (IRTs)?

Yes. Or lower response rates.

On how large a sample are these results based?

A total of 500 food deliveries. Depending on schedule and
rate this could be from 1000 pecks (IRT = 16 sec) to over
18000 pecks (IRT = .5 sec).

O.K.

What is the time-increment (dt)?

I think it's essentially 0. I don't use time increments;
I just generate pecks and trigger rewards after the
appropriate (real valued) intervals.

I don't follow. How do you know when an "appropriate (real valued)
interval" is up if you aren't counting ticks of a definite size (e.g.,
incrementing time at .1 sec per iteration of the program loop)?

Regards,

Bruce

[From Rick Marken (990518.1230)]

Bruce Abbott (990518.1310 EST)]

I don't follow. How do you know when an "appropriate (real
valued) interval" is up if you aren't counting ticks of a
definite size (e.g., incrementing time at .1 sec per iteration
of the program loop)?

Here's the program that does it. Basically, it compares the
running integral (Respintegral) of response intervals (Respint)
to the key-appropriate running integral (RewIntegral(key)) of
reward intervals (RewInt(key)) for each key.

Best

Rick

···

-----------------
Dim VI(3), SumRespInt(3), SumRewInt(3), NRespInt(3), NRewInt(3)
Dim RewIntegral(3), LastResp(3), LastRew(3), Pi(3), ri(3), RewInt(3)

    VI(1) = ((Cells(2, 2).Value) * 60) * 2
    VI(2) = ((Cells(2, 3).Value) * 60) * 2

    IRT = Cells(3, 2).Value

    EatTime = Cells(5, 2).Value

    SwitchProb = Cells(6, 2).Value
    KeyAProb = Cells(7, 2).Value

    SumRespInt(1) = 0: NRespInt(1) = 0
    SumRespInt(2) = 0: NRespInt(2) = 0
    SumRespAll = 0: NRespAll = 0
    SumRewInt(1) = 0: NRewInt(1) = 0
    SumRewInt(2) = 0: NRewInt(2) = 0

    RewIntegral(1) = Rnd(5) * VI(1) + 1
    RewIntegral(2) = Rnd(7) * VI(2) + 1
    LastResp(1) = 0: LastResp(2) = 0
    LastRew(1) = 0: LastRew(2) = 0
    Respintegral = 0
    key = 1:
    Foodtime = 0

    For i = 1 To 500

Re: Respint = IRT + Rnd(3) * (IRT / 2) + Foodtime
     Foodtime = 0

     SumRespAll = SumRespAll + Respint
     NRespAll = NRespAll + 1

     Respintegral = Respintegral + Respint

     If (Rnd(3) < SwitchProb) Then
         If (Rnd(5) < KeyAProb) Then key = 1 Else key = 2
     End If

     SumRespInt(key) = SumRespInt(key) + Respintegral - LastResp(key)
     NRespInt(key) = NRespInt(key) + 1
     LastResp(key) = Respintegral

    If (Respintegral < RewIntegral(key)) Then GoTo Re

    rewardkey = key

    Respint = IRT + Rnd(3) * (IRT / 2) + Foodtime
    SumRespAll = SumRespAll + Respint
    NRespAll = NRespAll + 1

    Respintegral = Respintegral + Respint

    If (Rnd(3) < SwitchProb) Then
        If (Rnd(5) < KeyAProb) Then key = 1 Else key = 2
    End If

    SumRespInt(key) = SumRespInt(key) + Respintegral - LastResp(key)
    NRespInt(key) = NRespInt(key) + 1
    LastResp(key) = Respintegral

    If (key <> rewardkey) Then GoTo CODL

        SumRewInt(key) = SumRewInt(key) + Respintegral - LastRew(key)
        NRewInt(key) = NRewInt(key) + 1
        LastRew(key) = Respintegral
        Foodtime = Eatime + Rnd(3) * EatTime

        RewInt(key) = (Rnd(5) * VI(key) + 1)
        RewIntegral(key) = RewIntegral(key) + RewInt(key)
        GoTo Norm

CODL: RewIntegral(rewardkey) = RewIntegral(rewardkey) +
      RewInt(rewardkey)
      GoTo Re

Norm: Next i

    For k = 1 To 2

    If (NRespInt(k) > 0) Then Pi(k) = (1 / (SumRespInt(k) /
    NRespInt(k)))
    If (NRewInt(k) > 0) Then ri(k) = (1 / (SumRewInt(k) /
    NRewInt(k)))

    AVPi = (1 / (SumRespAll / NRespAll))

    Cells(9, 1 + k).Value = NRespInt(k)
    Cells(10, 1 + k).Value = NRewInt(k)

    Cells(11, 1 + k).Value = Pi(k)
    Cells(12, 1 + k).Value = ri(k)

    Next k

    Cells(13, 2).Value = (Respintegral / 60) / 60
    Cells(14, 2).Value = AVPi

--
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken

[From Bruce Abbott (990518.2015 EST)]

I have implemented a simulated pigeon pecking on two available keys on which
concurrent VI VI schedules run, having mean intervals equal to those used by
Herrnstein (1961). For the purpose of the simuation I set the overall
response rate equal to that typical of Herrnstein's pigeons when responding
on VI 3-min VI 3-min; the actual responses are generated by a
constant-probability scheme in which the decision to respond or not is made
each dt (0.1 sec) with a fixed probability of success. On such a schedule,
the pigeon is just as likely to respond in each succeeding dt whether or not
it responded in any previous time interval.

In this preliminary stab the pigeon is assumed to switch keys whenever some
fixed time period t has passed without reinforcement. The time t was set to
give a rate of switching comparable to those demonstrated by Herrnstein's
pigeons on VI 3-min VI 3-min. Typical values are 5.4 seconds/switch at VI
3-min VI 3-min and 27 seconds/switch at VI 1.8-min VI 9-min. Because the
delivery of reinforcement within time t resets the switch-counter, the
pigeon will stay with a given key so long as the programmed interval is less
than t. Such intervals are much more common on higher-rate schedules (e.g.,
VI 1.8 min) than on lower-rate schedules (e.g., VI 9-min), so when the two
schedules differ in mean rate, the pigeon using this simple rule will tend
to stay longer at the key offering the higher rate of reinforcement. This
produces a tendency toward matching (more responses emitted on the key
offering the higher rate of reinforcement) and reduces the switch-rate. The
question was whether the effect of this rule would be sufficient to produce
matching, and if so, whether the observed switch rate would decline as much
as actually observed when going from VI 3-min VI 3-min to VI 1.8-min VI 9-min.

The simulated data do not conform well to Herrnstein's observations. On VI
3-min VI 3-min the response rates were 0.55 pecks/sec on each key, with a
mean time between switches of 5.1 sec. p1/(p1+p2) = .50 and r1/(r1+r2) =
.49, which do agree with Herrnstein's data because the parameters of the
simulation were set to produce good agreement. However, for VI 1.8-min VI
9-min, _using the same parameters for the pigeon_, p1/(p1+p2) = .51,
r1/(r1+r2) = .83 (definitely not matching!) and the switch-time averaged 5.1
seconds (as opposed to 27 seconds observed).

I think we can safely conclude that pigeons do not simply switch keys
whenever they have responded on one key for a certain length of time without
receiving reinforcement (one version of an ecoli-type control system).

In addition to this simulation, I have explored the effect of inter-response
time (IRT) on the obtained inter-food time when both IRTs and VI values are
generated on a constant-probability basis. To a fair approximation, the
obtained IFI can be computed simply by adding the mean IRT to the mean
interval of the schedule (i.e., the VI value). Thus the IFI = VI + IRT.
Example: the obtained interval between reinforcements on a VI 30-s schedule
with a mean IRT of 4 seconds would be around 34 seconds. You can use this
relationship to estimate the relative rate of reinforcement obtained at any
mean IRT. For example, if we assume that the pigeon just responds randomly
on the two keys at a mean IRT of 3 seconds/response, and the two current VIs
are VI 30-s and VI 90-s, then the obtained IFIs will be about 33 and 93
seconds and the relative rate of reinforcement on the VI 30-s schedule will
be 93/(33+93) = .74, as opposed to a programmed relative rate of 90/(30+90)
= .75. It will be seen matching does not result from having a pigeon
randomly respond on the two keys, except when r1/(r1+r2) = .50.

Regards,

Bruce

from [ Marc Abrams (990518.2237) ]

[From Bruce Abbott (990518.2015 EST)]

I have implemented a simulated pigeon pecking on two available keys on

which

concurrent VI VI schedules run, having mean intervals equal to those used

by

Herrnstein (1961). For the purpose of the simuation I set the...

Bruce could you please explain to me why this study is important?, How it
relates to control? and how you see it relating to humans? Why should I care
about this?

Marc

[From Bruce Abbott (990518.2315 EST)]

Marc Abrams (990518.2237) --

Bruce Abbott (990518.2015 EST)

I have implemented a simulated pigeon pecking on two available keys on which
concurrent VI VI schedules run, having mean intervals equal to those used by
Herrnstein (1961). For the purpose of the simuation I set the...

Bruce could you please explain to me why this study is important?, How it
relates to control? and how you see it relating to humans? Why should I care
about this?

Presumably a pigeon is a bit simpler critter than a human (fewer levels in
the heirarchy, for one thing), but the basic operating principles ought to
be similar (e.g., perceptual control). Under certain conditions pigeons and
people behave in remarkably similar ways; thus there is some justification
for the hope that principles of behavior discovered in pigeon studies will
prove to apply under proper conditions to the human case as well. Given
that the same end can be achieved by different means, how will a pigeon (or
a person) allocate his or her behavioral resources (e.g., effort, time) to
those different activities? Given that different variables must be
controlled, but that behavior must be time-shared among the different
control systems, on what basis will the switch from one behavior to the
other be made? These are rather fundamental questions whose answers may
help us to better understand how these behavioral systems are organized.

How does it relate to control? If PCT's fundamental assertion is correct --
behavior is the control of perception -- then all that behaving recorded on
the keys of the operant chamber must be happening for the purpose of
bringing or keeping under control certain of the pigeon's perceptions (if it
is not just a side-effect of control actions). I think it's quite clear
that the pigeon is controlling for having access to grain and has learned to
peck at keys as a means of doing so. But in this case there are two
alternate means of achieving access to grain (pecking at the right or left
keys); furthermore, greater access to grain may be achieved with these
schedules by at least occasionally sampling the key offering the lower rate
of grain access, than by responding exclusively on either key. As it turns
out, what generally happens is that the pigeons end up allocating their time
and responses on the two keys so as to match their relative time and
responses on a key to the relative rate of payoff associated with that key.
So, if we assume that the pigeon's behavior -- including switching behavior
-- is directed toward controlling some perception(s), then what perceptions
are they and how is the pigeon attempting to control them? My simple model
proposes that the pigeon is attempting to minimize the time to next grain
access by switching keys whenever the current key has not "paid off" in some
time. It's an ecoli-like control system in which success (getting access to
grain) postpones the next "tumble" (key-switch) while continued failure
results in a "tumble" (key-switch). In ecoli, a tumble sets the bacterium
off in a new direction which may or may not be better; similarly a switch to
the other key may or may not yield sooner access to grain; if not, then
another "tumble" ensues (switch back to the first key).

Why should you care? Perhaps you shouldn't. It's basic research and may or
may not turn out to have practical application in the world of human
affairs. But from my point of view, any improvement in our understanding
how living, behaving organisms are organized is one more step toward a
fundamental understanding of ourselves.

Regards,

Bruce

[From Rick Marken (990519.2030)]

Bruce Abbott (990518.2015 EST)--

I have implemented a simulated pigeon pecking on two available
keys on which concurrent VI VI schedules run, having mean
intervals equal to those used by Herrnstein (1961)...

Marc Abrams (990518.2237)

Bruce could you please explain to me why this study is
important?, How it relates to control?

I've got to go with Marc on this, Bruce: what in the world does
all this have to do with control? The more I work it, the more
it looks like the answer is "nothing". What am I missing?

Best

Rick

···

---
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken/

[From Bill Powers (990519.0315 MDT)]

Bruce Abbott (990518.2015 EST)--

In this preliminary stab the pigeon is assumed to switch keys whenever some
fixed time period t has passed without reinforcement.

You note that while t can be adjusted so that switching occurs at the
observed intervals under the VI 3-3 schedule, this same time is not correct
for the VI 1.8-9 schedule. Clearly, the actual switching time is some
function of the schedules. The proposal above is clearly refuted by the
data immediately:

"Typical values are 5.4 seconds/switch at VI
3-min VI 3-min and 27 seconds/switch at VI 1.8-min VI 9-min," you say.

Any proposal has to take all the data into account, doesn't it?

Since we're only curve-fitting at this point, rather than trying to propose
a plausible model, we can try various ways of representing these
observations that might be helpful. For example, the _ratios_ of the two
schedules vary by a factor of 5 (from 3/3 = 1 to 9/1.8 = 5.0), while the
switching time also varies by a factor of 5 (27/5.4 = 5.0). This suggests
that the switching time can be expressed as

t = k * VI2/VI1 seconds per switch, where k is 5.4.

It might be that this value of k produces a fit only for these two values
of the ratio of schedules, or it might fit over a wider range. If the fit
held up over other pairs of schedules, it might then be worthwhile to try
to find a mechanism that could create this relationship. One hint about a
model is that in physical systems, constant ratios typically arise from
differences between exponential decays, on a log scale. After a food
delivery, we might reasonably guess that the initial sensory effect of
ingesting the food decays exponentially (roughly) to zero, so we might
propose a model in which sensory effects of food intakes generated by the
two keys are kept separate and compared. But of course one would prefer
something simpler.

Also, the switching time from the briefer to the longer schedule might well
be different from the switching time from the longer to the briefer
schedule. Information about this might give us a different picture of
what's happening. Do you happen to have it?

We might consider that there are control systems running concurrently, with
their outputs pecking each key independently without any actual "switching"
taking place, from the standpoint of either control system (time-sharing).
Or these two control systems might be in conflict, each trying to maintain
the _place_ of pecking on one key while the other tries to maintain it on
the other key. Many hypotheses about models are possible.

The E. coli model would suggest that the time between switches is some
function of the difference between actual food intake and desired or needed
food intake. The greater the shortfall, the shorter the time before the
next switch. The linear form of this model offers two parameters for
adjustment, the reference level and gain, which might be sufficient.

In addition to this simulation, I have explored the effect of inter-response
time (IRT) on the obtained inter-food time when both IRTs and VI values are
generated on a constant-probability basis. ...
It will be seen matching does not result from having a pigeon
randomly respond on the two keys, except when r1/(r1+r2) = .50.

Of course we expected this, but now the question is, how significant is a
given departure from the rate of reinforcement predicted by the random
model? In comparing RA/(RA + RB) to rA/(rA+rB), we are not starting from
zero, but from the rate predicted by the random model. How many standard
deviations different are the rates predicted by matching and by the
random-behavior model? The effect of the variability in interval is going
to create a pretty humongous standard deviation, isn't it?

Best,

Bill P.

[From Bruce Gregory (990519.0944 EDT)]

Bruce Abbott (990518.2315 EST)]

Why should you care? Perhaps you shouldn't. It's basic
research and may or
may not turn out to have practical application in the world of human
affairs. But from my point of view, any improvement in our
understanding
how living, behaving organisms are organized is one more step toward a
fundamental understanding of ourselves.

Gambling experiments done in the early 1960's showed a behavior
superficially at least similar to that of the pigeons. Subjects tended
to place bets on outcomes with a frequency matched to the frequency of
the outcome (if the left light was illuminated 60% of the time, the
gambler bet on the left light 60% of the time; presumably a "rational"
gambler would bet on the left light all the time.) As I recall, not
until the frequency was of the order 90%, did the gambler shift to
betting all the time on the most frequently illuminated light.
Unfortunately I do not recall who conducted these studies.

Bruce Gregory

[From Bill Powers (990519.MDT)]

I'll rely on Lloyd Klinedinst to give a more detailed report on the
Conference on Internal Control Psychology, but I thought a brief report
would be appropriate.

The four speakers were, in order, myself, Albert Ellis, Bill Glasser, and
Alfie Kohn. I gave a quick overview of PCT followed by the rubber bands, an
introduction to conflict, and a brief discussion of the role of conflict as
the chief problem psychotherapists must face. I'll post it after it appears
in the International Journal of Reality Therapy. You know what I would have
said.

Albert Ellis, originator of Rational-Emotive Behavioral Therapy, spoke
about his theory that people's problems come from insisting on satisfying
their wants and desires exactly, while the rational person would simply
prefer that they be satisfied without going to emotional extremes over
them. Outside of that, his psychology was fairly conventional. His
therapeutic method seemed to consist mainly of confronting and opposing the
client's irrational desires as "aggressively" as necessary. Ellis
criticized Glasser's "basic needs" as being too needful, as Mary puts it.

Bill Glasser showed us an animation (that is, a role-play) of his new
"Structured Reality Therapy" as applied to a marriage counselling process.
It consisted of asking the clients five questions, at each stage offering
them the option of giving a negative answer and terminating the
counselling, at no charge to them. The last question entailed a committment
to go home and do something that would improve the marriage for the next 7
days, then return for the last session in which Glasser exhorted the couple
to go on doing the same sorts of things as long as they wanted the marriage
to last. The charge for both sessions, if successful, was $250, or nothing
if the couple decided not to continue their marriage. Those two sessions
were the entirety of the counselling for a couple. Glasser was conciliatory
to me and even incorporated the rubber-band demo into his talk. He seemed
surprised that I wasn't mad at him.

Alfie Kohn gave a spirited lecture in which he reviewed his arguments
against competition and "punishment by reward." He then went through a list
of a few criticisms of Ellis's approach and many of Glasser's. Before this
was finished, Ellis complained loudly that Kohn had run over his time
(Ellis is very deaf and admitted he didn't hear much of what Kohn said),
and then Glasser got to his feet and objected in strenuous and extended
terms to everything Kohn had said about him. That effectively ended Kohn's
presentation. I enjoyed Kohn's presentation very much, but don't remember
much of what was in it, possibly because there was little to disagree with.

The schedule for the next day was revised by popular vote to become a panel
discussion with questions from the audience, rather than a series of
detailed questionings of each presenter in turn. By that time, Kohn and
Glasser had managed to simmer down and the panel went smoothly. Glasser
even went out of his way to use words like "reorganization," and commented
that he was relieved to find that he and I were getting along as if our
previous troubles (unspecified) were no longer a problem. I said I had no
problems with his presentation, or him. I suggested that it might be
worthwhile for him to consider the possibility that conflict was an
important kind of problem to be addressed in therapy.

I'm sure Lloyd will have more details to add, especially about how the
IAACT people who attended felt about being in the midst of the Glasserians,
and how they were received in the working groups that formed twice the
first day and once on the second. Two people from Ed Ford's group attended
and were enthusiastic about my talk.

Larry Litwack, the organizer and also editor of the International Journal
for Reality Therapy, said that articles from Ed Ford's group, IAACT, and
the CSG would be welcome. He indicated a possibility that the name of the
journal might be changed to the International Journal for Internal Control
Psychology. He says his editorial policies are entirely independent of
Glasser, and I believe him.

The sessions were videotaped; more on how to obtain the tapes later, when I
hear (or Lloyd hears) from Litwack.

Best,

Bill P.

[From Bill Powers (990519.0826 MDT)]

Bruce Gregory (990519.0944 EDT)--

Gambling experiments done in the early 1960's showed a behavior
superficially at least similar to that of the pigeons. Subjects tended
to place bets on outcomes with a frequency matched to the frequency of
the outcome (if the left light was illuminated 60% of the time, the
gambler bet on the left light 60% of the time; presumably a "rational"
gambler would bet on the left light all the time.)

Perhaps a rational gambler does not bet with the idea that the odds are
unchanging. Does anyone know what the rational strategy is if the odds are
randomly changing over time?

Best,

Bill P.

[From Rick Marken (990519.0830)]

Bruce Abbott (990518.2315 EST)

How does it relate to control?...I think it's quite clear
that the pigeon is controlling for having access to grain

"Access to grain" is just one possible controlled variable.
And it's a binary variable at that since it has only two
states: yes or no. If "access to grain" is the controlled
variable, then why does the bird peck at both keys? It has
access to grain (access = yes) if it just pecks at one key.
Since the VI research was done without any notion that some
variable might be under control, we just have to guess at what
the bird is controlling -- or _trying_ to control. I would
rather see some tests to determine what aspects of grain
input a pigeon controls when it _can_ control that input before
trying to figure out what the pigeon might be _trying_ to
control in situations (like the VI schedule studies) where it
_can't_ control that input.

furthermore, greater access to grain may be achieved with these
schedules by at least occasionally sampling the key offering the
lower rate of grain access, than by responding exclusively on
either key.

This is true. But my simulations show that "occasionally" can
be _very_ occasional. That is, for any concurrent VI schedules,
the overall rate of reinforcement is _nearly_ the same whether
the probability of a peck on the high pay-off key is .9, .5 or
.1. For example, here are the overall reinforcement rates
(reinforcements/sec) on different VI schedules as a function
of the bias for pecking the high pay-off key (key A):

                        P(Peck Key A)
Key A B .9 .5 .1

VI 1.8 VI 9 .0106 .0106 .0105

VI 3 VI 3 .0106 .0105 .0104

VI 2.25 VI 4.5 .0107 .0104 .0103

There is a _slight_ tendency for the overall reward rate to
go up if the bird pecks with a higher probability at the
key that "pays off" at a higher rate. But this tendency is
very weak and highly variable (the numbers in the table are
averages over many runs; there were often runs when the overall
rate of reward was _greater_ when the probability of pecking
the high rate pay off key was only .1). This seems to rule out
the possibility that matching is a side effect of controlling
for overall rate of reward in these experiments.

As it turns out, what generally happens is that the pigeons
end up allocating their time and responses on the two keys
so as to match their relative time and responses on a key to
the relative rate of payoff associated with that key.

Again, why should this observation be of any interest to a
control theorist? What does it have to do with control?

So, if we assume that the pigeon's behavior -- including
switching behavior -- is directed toward controlling some
perception(s)

Why assume it? Why not test it?

then what perceptions are they and how is the pigeon attempting
to control them?

Good question. Unfortunately, it can't be answered by this data.

My simple model proposes that the pigeon is attempting to
minimize the time to next grain access by switching keys
whenever the current key has not "paid off" in some
time. It's an ecoli-like control system

No it's not. It's a regular control system (no random output)
that is controlling (or trying to control) time between grain
deliveries. When the perceived interval exceeds "some time"
(the reference interval) it switches keys. This is a deterministic,
not a probabilistic, control strategy.

Bill Powers (990519.0315 MDT) explained how an E. coli model
might work:

The E. coli model would suggest that the time between switches
is some function of the difference between actual food intake
and desired or needed food intake.

I suppose it would be fun to develop this E. coli model of
concurrent VI behavior. But it still seems to me like we're
going about things ass-backwards. We're making up a model to
explain how an animal acts when it _can't_ systematically
control food intake before we know how and what an animal
controls when it _can_ systematically control food intake.

But I'll play along.

Best

Rick

···

--
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken

[From Bruce Abbott (990519.1240 EST)]

Rick Marken (990519.0830) --

Bruce Abbott (990518.2315 EST)

My simple model proposes that the pigeon is attempting to
minimize the time to next grain access by switching keys
whenever the current key has not "paid off" in some
time. It's an ecoli-like control system

No it's not. It's a regular control system (no random output)
that is controlling (or trying to control) time between grain
deliveries. When the perceived interval exceeds "some time"
(the reference interval) it switches keys. This is a deterministic,
not a probabilistic, control strategy.

Bill Powers (990510.1146 MDT) --

Hey, it should use the e. coli strategy! The "tumbles" here are switches
from one key to another. If there is no food, switch to the next choice; if
there is food, delay the switch to the next choice -- all the while pecking
away at a constant rate at whichever key is the current choice. That will
lead to delivering more pecks on the side that produces the most
reinforcement.

Regards,

Bruce

[From Chris Cherpas (990519.1000 PT)]

Rick Marken (990519.0830)--

...This seems to rule out the possibility that
matching is a side effect of controlling
for overall rate of reward in these experiments.

This is what I found with pigeons: no
change in overall rate of food even though
relative switching rate varied systematically
with relative schedule changes.

We're making up a model to explain how an animal
acts when it _can't_ systematically control food
intake before we know how and what an animal
controls when it _can_ systematically control food
intake.

This is what Skinner was more or less studying (not
in terms of control, of course) for his dissertation,
before he became fascinated with schedules.

I agree with you Rick that a project of simulating
concurrent VIs before having a model for FR1 is backwards,
but it beats having this discussion with no simulations.

Best regards,
cc

[From Bruce Abbott (990519.1415 EST)]

Rick Marken (990519.1120) --

Bill Powers (990510.1146 MDT)

Hey, it should use the e. coli strategy! The "tumbles" here are
switches from one key to another. If there is no food, switch
to the next choice; if there is food, delay the switch to the
next choice -- all the while pecking away at a constant rate at
whichever key is the current choice. That will lead to delivering
more pecks on the side that produces the most reinforcement.

Right. So you see that your model doesn't use the e. coli
strategy. You know what the e. coli strategy is, right? If
not, check my papers on this in _Mind Readings_.

My simulation simply implements the above. As Bill describes it as "the e.
coli strategy," I am at a loss as to how you can insist that Bill's
description of it as "the e. coli strategy" is correct and yet my
simulation, which does exactly as Bill suggested, does not use the e. coli
strategy. Methinks thou art contradicting thyself. Enlighten me.

Regards,

Bruce