The Opposition

[Abbott_Bruce]

[From Bruce Abbott (990510.1945 EST)]

Bill Powers (990510.1146 MDT) --

Hmm. Thinking out loud.

Apparent matching _could_ be explained by saying that animals basically
select the best key and use it, but as the total payoff declines, they
begin more and more often trying the other key. Actually, with concurrent
schedules I think that there is always a net gain from trying the other key
(this just occurred to me). An occasional press on the less productive key
does not substantially reduce the delivery rate from pressing the best key.
In fact, it need not reduce it at all unless the animal uses the other key
too often. The schedules are set up, according to Bruce, so that once the
interval runs out, the key is primed to produce a reinforcer on the next
peck, so anytime between the start of an interval and its termination, the
other key can be pecked without any penalty at all.

As a simple example, assume that one schedule is VI 30-s and the other is VI
60-s. By switching between keys often enough, our pigeon could collect
every reinforcer programmed on both schedules, netting itself an average
reinforcement rate of 3 reinforcers per minute (2/min on VI 30-s plus 1/min
on VI 60-s). So in this case the pigeon can increase reinforcement rate 50%
by switching between keys, as opposed to staying with the key offering the
higher payoff rate (VI 30-s).

This means that pressing the best key exclusively is NEVER the best
strategy, under the schedules as defined. The animal should peck one key
for a short time, and if that doesn't produce a bit of food immediately, it
should peck the other key for a short time, and so on back and forth as
quickly as possible. This will produce what looks like a tendency to
matching, because the animal will give up on the less productive key
without receiving a reward sooner than on the more productive one.

Hmmmm. This needs to be spelled out. Why would the animal give up "sooner"
on the less productive key? Maybe it switches keys if the current key
hasn't paid off for some relatively fixed period of time. Because there are
more "excessively long" intervals on the longer VI schedule, switching from
the longer schedule (less productive key) to the shorter one (more
productive key) would take place more often than the reverse. The pigeon is
"giving up" on a key after the same lapse of time without reinforcement,
regardless of schedule; it's not giving up on the less productive key
sooner, it's giving up on the less productive key more often.

Hmm again.

Hey, it should use the e. coli strategy! The "tumbles" here are switches
from one key to another. If there is no food, switch to the next choice; if
there is food, delay the switch to the next choice -- all the while pecking
away at a constant rate at whichever key is the current choice. That will
lead to delivering more pecks on the side that produces the most
reinforcement.

I think you mean "if there is no food _by some interval_, switch." If so,
that sounds reasonable.

We can generalize: one of the "keys" could be "no key" -- i.e. going away
from the keys and pecking at different places in the cage. Of course the
yield there is zero, so there will be a quick switch to the next place, and
a return to pecking on keys. Since the keys do usually provide food on the
first peck after a long delay, the time spend on keys as opposed to away
from them will increase. It should be easy to set the parameters of an e.
coli model to get the best match of model behavior to the data.

Yes, except that some things a pigeon does other than key-pecking may help
to control other variables; to that extent those other behaviors will
compete with keypecking. Herrnstein allowed for this possibility in his
equation for single-key responding: P = kr/(r + ro), where k is a constant
and ro is the rate of reinforcement for "other behavior," scaled in
grain-access-equivalent units.

I expect that by the time I get back from Boston, three people will have
this model working.

Hmmm. I wonder which three? . . . Anyone care to draw straws?

Regards,

Bruce

[Marc_Abrams5]

Hi Dick,

Nice to see you posting on the net again. Hope you don't mind a neophyte
jumping into this.

[From Dick Robertson,990510.1221CDT]

I understand that. What I want to ask is whether this matching law says,

and

has confirmation for, subjects (people or animals) choosing alternative

options

for getting payoffs in proportion to the relative proportions of the

payoff,

instead of simply devoting all their efforts to the option that has the

best

payoff? I don't undersatnd why it is so hard for me to get an answer to

this

question. It seems a simple one to me. What am I missing?

I equate your statement "best payoff" to mean "optimizing for ...". I don't
think this is an accurate description of behavior from a PCT perspective. I
think we have multiple goals and behaviors taking place at the same time,
and which "behavior" is "most important" changes continuously.
Without trying to be pedantic, a bit of an example if I may. I want to go to
the store and buy a few items of food. While I am walking to the store ( I
live in NYC so we do a lot of walking to our ;ocal food markets ) I am
controlling for not falling down. I am avoiding potholes, and various other
sundry disturbances. I am also listening to my walkman and humming and
whistling the songs. I stop at the corner for a light and look to see that
no cars are coming. ( In NYC the rule is whoever gets to that spot first,
meaning car or pedestrian, wins )I am looking out for cars, while I
listen to my music and try to stay erect ( not fall ). If I see no cars I
may run across the street. After I cross the street, I run into a friend. We
start talking about things and I realize that my ballgame will be on soon,
So I disengage from our conversation. On, it goes. What would a "best
payoff" mean with this series of events? Can we really talk about our
behavior as a "singular" occurance?

> >What interested me in the original report from Jeff's friend was the
> >apparent implication of the "matching law" that research had

established

> >that: given two levers, one that would deliver a food pellet twice as
> >often as the second, the rat would divide his efforts to push the first
> >twice as often as the second, rather than the more "expectable" (to the
> >lay observer at least) act of devoting all his efforts to the lever

that

> >gave the best payoff.

How can we begin to talk about "payoffs" when we don't know what the rat is
actually controlling for. How do we know that the rat is simply pressing the
bar because the rat thinks it's fun to presss the bar. Did the experiment
have a second bar that gave no pellets?

> >If that IS what the matching law says, I would find that somewhat

> >interesting. (And BTW in the original post there was some reference to
> >people in shopping malls, suggesting that the M-Law has some real-life
> >relevance.) I would wonder what the explanation might be, what could

be

> >surmised about how the subject, person or rat, controls for getting
> >rewarded that would make such a strategy fruitful. If I have got the
> >wrong idea, I would like to know what the right one is.

I'm with you on this.

[Dick_Robertson]

[From Dick Robertson,990511.0707CDT]

Bill Powers wrote:

[From Bill Powers (990510.1146 MDT)]

Dick Robertson,990510.1221CDT--

>What I want to ask is whether this matching law says, and
>has confirmation for, subjects (people or animals) choosing alternative
options
>for getting payoffs in proportion to the relative proportions of the payoff,
>instead of simply devoting all their efforts to the option that has the best
>payoff?

Yes, as I understand it that is the claim: the choice is supposedly
proportional to the payoff (sort of). You have to understand, however, that
the payoff is VERY SMALL, so the animal may well have reason to continue
searching for a better payoff by trying other keys. Pressing the key with
the higher payoff exclusively would still not produce subsistence-level
food intake. I suspect that with much higher payoffs, animals would tend
much more to pick the best key and ignore the other one. But that's just my
guess.

OK That makes sense. That sounds like something that could be tested. BTW Bruce
Gregory mentioned an old study of gambling behavior that seems to show the same
thing.

Hmm. Thinking out loud.

Apparent matching _could_ be explained by saying that animals basically
select the best key and use it, but as the total payoff declines, they
begin more and more often trying the other key. Actually, with concurrent
schedules I think that there is always a net gain from trying the other key
(this just occurred to me). An occasional press on the less productive key
does not substantially reduce the delivery rate from pressing the best key.

This means that pressing the best key exclusively is NEVER the best
strategy, under the schedules as defined. The animal should peck one key
for a short time, and if that doesn't produce a bit of food immediately, it
should peck the other key for a short time, and so on back and forth as
quickly as possible. This will produce what looks like a tendency to
matching, because the animal will give up on the less productive key
without receiving a reward sooner than on the more productive one.

Hmm again.

Hey, it should use the e. coli strategy! The "tumbles" here are switches
from one key to another. If there is no food, switch to the next choice; if
there is food, delay the switch to the next choice -- all the while pecking
away at a constant rate at whichever key is the current choice. That will
lead to delivering more pecks on the side that produces the most
reinforcement.
Best, Bill P.

This is getting better and better. Have a great in Boston.

Best, Dick R.

[Dick_Robertson]

[From Dick Robertson,990511.CDT]

Marc S Abrams wrote:

Hi Dick,

Nice to see you posting on the net again. Hope you don't mind a neophyte
jumping into this.

Thank, Mark

> [From Dick Robertson,990510.1221CDT]

> I understand that. What I want to ask is whether this matching law says,
>and has confirmation for, subjects (people or animals) choosing alternative
>options for getting payoffs in proportion to the relative proportions of the

>payoff, instead of simply devoting all their efforts to the option that has
the
>best payoff?

I equate your statement "best payoff" to mean "optimizing for ...". I don't
think this is an accurate description of behavior from a PCT perspective. I
think we have multiple goals and behaviors taking place at the same time,
and which "behavior" is "most important" changes continuously.
Without trying to be pedantic, a bit of an example if I may. I want to go to
the store and buy a few items of food. While I am walking to the store ( I
live in NYC so we do a lot of walking to our ;ocal food markets ) I am
controlling for not falling down. I am avoiding potholes, and various other
sundry disturbances. I am also listening to my walkman and humming and
whistling the songs. I stop at the corner for a light and look to see that
no cars are coming. ( In NYC the rule is whoever gets to that spot first,
meaning car or pedestrian, wins )I am looking out for cars, while I
listen to my music and try to stay erect ( not fall ). If I see no cars I
may run across the street. After I cross the street, I run into a friend. We
start talking about things and I realize that my ballgame will be on soon,
So I disengage from our conversation. On, it goes. What would a "best
payoff" mean with this series of events? Can we really talk about our
behavior as a "singular" occurance?

This is true, but I don't see these various activities going on in the hierarchy
at the same time as the same as apportioning one's efforts among a set of
alternatives for the same CV.

> > >What interested me in the original report from Jeff's friend was the
> > >apparent implication of the "matching law" that research had
established
> > >that: given two levers, one that would deliver a food pellet twice as
> > >often as the second, the rat would divide his efforts to push the first
> > >twice as often as the second, rather than the more "expectable" (to the
> > >lay observer at least) act of devoting all his efforts to the lever
that
> > >gave the best payoff.

How can we begin to talk about "payoffs" when we don't know what the rat is
actually controlling for. How do we know that the rat is simply pressing the
bar because the rat thinks it's fun to presss the bar. Did the experiment
have a second bar that gave no pellets?

See Bill's thoughts on this. Maybe we are going to see some fruitful
experimentation.

I'm with you on this.

Great. Best, Dick R.

[R_Marken]

[From Richard Marken (990511.0810)]

Me:

could you give a quick explanation of why the "matching"
phenomenon is such a big deal for behaviorists?

Bruce Abbott (990510.1240 EST)]

There is a number of reasons.... it can characterize animals'
foraging in a "patchy" environment (one in which the stuff
being foraged for occurs in relatively dense patches separated
by relatively large expanses of little or none).

Since matching only occurs on VI schedules, does this mean that
behaviorists have learned that a "patchy" environment is
equivalent to a VI schedule? Is this actually true? Isn't a
patchy environment more like a variable ratio schedule?

Second, the reason or reasons why the matching relation emerges
(when it does emerge) is/are not understood, and several
competing explanations have been proposed

What are these competing explanations? Hasn't this research been
going on for decades? Haven't all but one of the explanations
been rejected yet?

The present inability to arrive at concensus as to the
underlying mechanism(s) highlights the fact that there's
still a lot that cannot be readily understood from the
perspective of traditional reinforcement theory

I agree with you there. Actually, it seems to highlight the
fact that reinforcement theory is wrong. But I guess it's
hard to give up such a comfortable habit.

Best

Rick

···

---
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken

[Abbott_Bruce]

[Bruce Abbott (990511.1250 EST)]

Richard Marken (990511.0810) --

Me:

could you give a quick explanation of why the "matching"
phenomenon is such a big deal for behaviorists?

Bruce Abbott (990510.1240 EST)]

There is a number of reasons.... it can characterize animals'
foraging in a "patchy" environment (one in which the stuff
being foraged for occurs in relatively dense patches separated
by relatively large expanses of little or none).

Since matching only occurs on VI schedules, does this mean that
behaviorists have learned that a "patchy" environment is
equivalent to a VI schedule? Is this actually true? Isn't a
patchy environment more like a variable ratio schedule?

No. While you are depleting one patch, new goodies keep growing on other,
previously depleted patches that are not currently being visited. This is
more like the VI schedule: the longer you are away from it, the greater the
liklihood that a visit will pay off.

Second, the reason or reasons why the matching relation emerges
(when it does emerge) is/are not understood, and several
competing explanations have been proposed

What are these competing explanations?

I'm not quite up to the minute on this, but the basic conflict has been
between "molar" theories (which assume that animals are sensitive to the
rate of events over time) and "molecular" theories (which assume that
matching emerges as a by-product of immediate, individual consequences of
behavior). Many of these views assume that by behaving as it does on these
schedules, the animal maximizes (or at least "satisfices," as Simon would
have said) some sort of gain or gain/loss (although the animal may not
actually be intending to do so).

Hasn't this research been
going on for decades?

Yes.

Haven't all but one of the explanations

been rejected yet?

No, but is hasn't been for lack of trying . . .

The present inability to arrive at concensus as to the
underlying mechanism(s) highlights the fact that there's
still a lot that cannot be readily understood from the
perspective of traditional reinforcement theory

I agree with you there. Actually, it seems to highlight the
fact that reinforcement theory is wrong. But I guess it's
hard to give up such a comfortable habit.

I agree with me there, too. But having difficulty pinning down what is
going on in this situation is about as much a challenge to standard
reinforcement theory as having difficulty discovering what all the CVs are
is a challenge to PCT. In either case one is more likely to assume that the
problem is with the specific application of the theory (model) rather than
with the theory.

Anyway, at this point it's a bit premature to knock some other model when
you don't have anything better to offer in its place (i.e., talk is cheap).
Far more impressive would be to come up with a PCT model that blows away the
reinforcement competition in this arena.

Regards,

Bruce

[Marc_Abrams5]

from [ Marc Abrams (990511.1241) ]

[From Dick Robertson,990511.CDT]

> > [From Dick Robertson,990510.1221CDT]

This is true, but I don't see these various activities going on in the

hierarchy

at the same time as the same as apportioning one's efforts among a set of
alternatives for the same CV.

Ok, Many activities could be involved in the control of one CV. One activity
could also be involved in controlling many CV's. My first question is, how
do we know if we are dealing with the former or the latter. My second
question would be, how do I know that I have answered the first question
correctly.

I am concerned about this because I think that we ( meaning those of us who
believe in the PCT model ) are easily "trapped" into trying to "explain"
behavior in conventional ways. To try and explain a persons "behavior" as a
_singular_ act is _not_ ( at least from my view ) PCTish. Of course the
next question becomes, Ok _which_ behavior is the "important one", or which
act carriied the "most intent".

Doing the test only answers one of these questions.( If it is done
correctly )And that is, is the person controlling for something _you_
hypothesisezed about. So even doing the test is something that we do for
_ourselves_ with regard to others. It does not tell us whether the CV we
have uncovered is in fact important ( and to what degree ) _relative to
other CV's that might be present simultaneously

I think we need to start looking at people from a PCT perspective. To say
someone _is_ a moron
is an innaccurate PCT statement. To say that someone behaved moronically at
a specific point or interval in time would be more accurate.

> > > >What interested me in the original report from Jeff's friend was

the

> > > >apparent implication of the "matching law" that research had

established

> > > >that: given two levers, one that would deliver a food pellet twice

as

> > > >often as the second, the rat would divide his efforts to push the

first

> > > >twice as often as the second, rather than the more "expectable" (to

the

> > > >lay observer at least) act of devoting all his efforts to the lever

that

> > > >gave the best payoff.

I think rat experiments are useless for understanding anything other then
the rats behavior. And as soon as we can communicate with them I believe
they will tell us the same

Why aren't we spending more time trying to devise tests that are meaningful
to humans and PCT?

Marc
Marc

[Chris_Cherpas]

[From Chris Cherpas (9905.1030 PT)]

Regarding concurrent schedules, it should be clear we
are not talking about a choice between two ratio schedules
(i.e., contingency based on the number of pecks on a key),
but two interval schedules (i.e., contingency based on the
first peck after a given interval of time), although opposing
ratio and interval schedules has been done.

I always used an explicit changeover key to peck,
with accompanying changes in the colors of lights,
to supposedly get a more accurate measure of "time
on each alternative," by the way. My grad research
fit into the program of research of Vaughan and
Herrnstein, showing that matching is not the result
of a process which maximizes total rate of food
obtained over a session, but instead reflected a
more local process, such that changes in local
rates within each alternative, and in relation to
such changes in the alternative(s), were tracked
fairly rapidly. This can lead to "suboptimal"
receipt of food on a global level if the contingencies
are rigged to oppose local versus global outcomes
(i.e., not just a typical concurrent schedules setup).

Bill Powers (990510.1146 MDT)--

Since the keys do usually provide food on the first peck
after a long delay, the time spend on keys as opposed to away
from them will increase. It should be easy to set the parameters of an e.
coli model to get the best match of model behavior to the data.

The e. coli approach sounds appropriate. I would advise looking
at the Vaughan & Herrnstein studies to potential modelers of the
problem.

Best regards,
cc

[R_Marken]

[From Rick Marken (990511.1250)]

Me:

Isn't a patchy environment more like a variable ratio schedule?

Bruce Abbott (990511.1250 EST)

No. While you are depleting one patch, new goodies keep growing
on other, previously depleted patches that are not currently
being visited.

But it's growing while you are eating; the delay has no effect
on your food getting, as it does in the VI schedule. And the
intervals involved are _much_ longer than the VI intervals
anyway. And how do they know that foraging animals are producing
matching anyway? It seems like it would be hard to see that
r1/P1=r2/P2 on two different patches, especially if you conceive
of this as r1/(r1+r2) = P1/(P1+P2). I think the notion that
matching is found in the wild is just wishful thinking.

But having difficulty pinning down what is going on in this
situation is about as much a challenge to standard reinforcement
theory as having difficulty discovering what all the CVs are
is a challenge to PCT.

The difficulty is not discovering CVs; the difficulty is
getting researchers (like you) to have the goal of discovering
them.

Anyway, at this point it's a bit premature to knock some
other model when you don't have anything better to offer in
its place

We have something much better to offer in the place of
reinforcement theory: control theory.

Far more impressive would be to come up with a PCT model that
blows away the reinforcement competition in this arena.

You know the problem with doing this: the data are lousy. As
you know, Bill came up with PCT models of shock avoidance
and ratio schedule data that blew away the comptetition; but
then it turned out that the data were artifactual. It makes
no sense to compare models in terms of their ability to
account for lousy data. The first step in comparing PCT to
alternative models of behavior must be to get good data.

Best

Rick

···

--
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken

[Abbott_Bruce]

[From Bruce Abbott (990511.1520 EST)]

Rick Marken (990511.1250) --

Me:

Isn't a patchy environment more like a variable ratio schedule?

Bruce Abbott (990511.1250 EST)

No. While you are depleting one patch, new goodies keep growing
on other, previously depleted patches that are not currently
being visited.

But it's growing while you are eating; the delay has no effect
on your food getting, as it does in the VI schedule. And the
intervals involved are _much_ longer than the VI intervals
anyway.

Consider foraging for something mobile, like prey animals. There are small
groups of them here and there, separated by spaces. You hunt in one area
until the prey become scarce there, then you move on until you find another
group. Meanwhile, other groups of prey are moving back into the depleted
area to take advantage of the fact that if offers plenty of the food they
like to eat (the same reason the first group was there before you depleted
it). This certainly works much more like VI than your suggested variable
ratio model does.

And how do they know that foraging animals are producing

matching anyway? It seems like it would be hard to see that
r1/P1=r2/P2 on two different patches, especially if you conceive
of this as r1/(r1+r2) = P1/(P1+P2). I think the notion that
matching is found in the wild is just wishful thinking.

Animals have been observed in the wild after their environment was gridded
off into squares. Time spent in each patch is recorded along with some
measure of the amount of food available there. At least that's how I
remember one such study being described to me.

More precise measures come from studies involving laboratory analogs of
foraging in a patchy environment.

But having difficulty pinning down what is going on in this
situation is about as much a challenge to standard reinforcement
theory as having difficulty discovering what all the CVs are
is a challenge to PCT.

The difficulty is not discovering CVs; the difficulty is
getting researchers (like you) to have the goal of discovering
them.

That misses the point entirely. The point was, that if difficulty in
determining the precise nature of a CV were encountered in a PCT study, it
would not occasion abandonment of PCT now, would it? Yet you appear to
believe that others pursuing different theoretical views should behave
differently -- at the first difficulty they should simply assume that the
whole superstructure is incorrect rather than that they simply do not yet
understand how to model the situation correctly under the current view.

Anyway, at this point it's a bit premature to knock some
other model when you don't have anything better to offer in
its place

We have something much better to offer in the place of
reinforcement theory: control theory.

I think so, too, but you have yet to demonstrate that. The model Bill
proposed for the VI data predicted exactly the opposite of what I observed
to happen. That doesn't necessarily mean that PCT is wrong, but it was no
shining moment for PCT either. Those results suggest that neither you nor I
nor Bill yet know how to properly model rats or pigeons working for food on
VI schedules. That sort of result makes PCT a tough sell to operant
conditioning researchers.

Far more impressive would be to come up with a PCT model that
blows away the reinforcement competition in this arena.

You know the problem with doing this: the data are lousy.

That's not the problem. If you came up with a control model that correctly
predicted when matching would and would not be expected to occur, there are
plenty of good data out there on which those predictions could be tested.

As
you know, Bill came up with PCT models of shock avoidance
and ratio schedule data that blew away the comptetition; but
then it turned out that the data were artifactual.

I don't recall anyone showing that the avoidance data were artifactual. And
as far as I know, they are not. In the ratio data, the changes in average
response rate with ratio size were artifactual, but the running response
rates and pause lengths that went into those average response rates were not.

It makes
no sense to compare models in terms of their ability to
account for lousy data. The first step in comparing PCT to
alternative models of behavior must be to get good data.

See above. I think that the first step is the one Bill encouraged us to
take -- come up with a control model, based on plausible assumptions, and
see how it behaves. That, at least, would be a start.

Regards,

Bruce

[R_Marken]

[From Rick Marken (990511.1520)]

Me:

We have something much better to offer in the place of
reinforcement theory: control theory.

Bruce Abbott (990511.1520 EST)--

I think so, too, but you have yet to demonstrate that.

Actually, I think I have demonstrated it (to some extent)
with the "Control of Consequences" papers in _Mind Readings_
and the "Selection of Consequences" models at

http://home.earthlink.net/~rmarken/demos.html

I know you don't think these constitute such a demonstration.
But I think they constitute a better demonstration of the
relative merits of control theory than anything you have
provided so far, which seems to be precisely nothing. If you
want better demonstrations than mine, then I suggest you take
your own advice and

come up with a control model [of operant behavior]... and see
how it behaves.

You're the expert on operant conditioning. You've also had four
years on CSGNet to learn how to write a control model (and a
reinforcement model) and show that control theory has something
better to offer in place of reinforcement theory. Why not
take some time away from defending behaviorism to show how a
control model handles some operant data better than does a
reinforcement model?

Best

Rick

···

--
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken