Proposal for educating EAB about PCT

[Chris Cherpas (970818.0921)]

Rick Marken (970818.0830)--

Here is a modest proposal for a paper I would like to see
written by someone (like Bruce Abbott, perhaps) who is a
recognized expert in both operant behavior analysis and PCT.

I'm obviously not a recognized expert in PCT, and I've
only published a few articles in the EAB world, but I've been
gradually piecing together an article like this. Bruce
and I have talked about his doing something like this before
and Bruce wanted to get some experimental results before moving forward.
I don't think you need a new experiment to make the main points.

Rick Marken (970818.0830)--

And I'd submit it to the heartland of operant research: JEAB.

If the argument is essentially qualitative, containing
neither the results of experiments nor a theoretical model,
JEAB would probably not publish it, but "The Behavior Analyst"
might very well.

Bill Powers (970818.p0415 MDT)--

Suppose we write a program that is organized to maintain one
"environmental" variable at a reference level. However, under different
conditions, different outputs may be needed to affect this variable,
because some external agency keeps turning the available external
connections on and off. So this program contains some kind of auxiliary
routine that monitors to see if there's a large error in the control
system, and if there is, it starts switching through the available outputs
(activated by the same error signal). To keep this simple, suppose there
are only two paths, from outputs A and B to the environmental variable,
paths which may be connected or disconnected.

I don't want to repeat Bill's whole post here, but what he's done is
essentially set up what is studied in concurrect schedules in EAB.
In fact, the issue of going to extinction versus going to relatively
lower frequencies on alternative A than B, and varying this relative
rate parametrically is exactly what the matching is based on (the
extinction case is just a special case). It's been with one alternative
being "non-contingent" as well and you get the same result.

The "empirical" version of "reinforcement" which Bruce describes,
with only one controlled alternative source of food, is inadequate
for making the point. Maybe you think the deviations from perfect
matching complicates the matter too much, but the fact is that the
relativity inherent in a concurrent operant procedure is a nontrivial
property of even the "empirical" version of "reinforcement." Varying
feeding outside the chamber might get you there, but, I think "variable means
to common ends" is more cleanly viewed through the concurrent lens.
As a corollary, I would say that the concurrent set-up, and the
relativity perspective, provide a pretty good way to divorce
"reinforcement" from common sense "reward."

Ergo, Bill's "wasting" some of his arguments on a version of the
"reinforcement" concept that was displaced over a quarter of a century
ago. This isn't an ad hoc comment I throw in to defend EAB. I just
think you don't want to come off as arguing against an anachronism.

A more technical point, but I guess now I've got to say something
about it. What I'm calling the relativity property essential to
the "modern" definition of the concept of "reinforcement" was first
observed with aggregate (not across subjects, but aggregates of
bouts at the concurrent alternatives) doesn't give you the blow-by-blow
of what happens each time the grain hopper opens. However, there's
been enough work (e.g., Vaughan's adaptation of the Rescorla-Wagner
formulation in support of the theory of melioration) to make it safe
for most EABers to say that some kind of a local interpretation
is possible. Again, I say this not to support the concept itself,
but indicate that you probably wouldn't want to discount the
relativity property, per se, on the basis that matching, per se,
is at level of of aggregation that cannot be said to express the
general concept of "reinforcement."

Dennis Delprato (970818.1024)--

And how about this? It appears that some of the main clinical researchers
in the treatment of severe self-injurious behavior have discovered that
in many cases it is quite effective to provide "noncontingent reinforcers."
...

Likewise, the _essentially_ relative nature of the concept of "reinforcement"
provides the central point of Gene Heyman's December 1996 article in Behavioral
and Brain Sciences, volume 19, Number 4, "Resolving the contradictions of
addiction." I wouldn't say that everybody in EAB embraces the matching law,
but ever since matching got replicated (yes, with varying degrees of
deviation), there isn't an EABer of any sophistication who does see the
concept of "reinforcement" as _requiring_ the specification of alternative
sources of whatever it is that is common to the controlled variable and
what is called the "reinforcer."

Dennis Delprato (970818.1024)--

...That is, the client performs less self-abuse when they are not reinforced
with reinforcers.

I'm having difficulty parsing this in a way I can interpret unambiguously.

Also, while Bruce has been consistent in saying that the organism
has to be adequately food-deprived, I don't know if the audience
on this list gets how crucial this is in formulating the "reinforcement"
concept from the EAB view.

I've gotta go now...

Best regards,
cc

[From Bill Powers (970818.1349 MDT)]

Chris Cherpas (970818.0921)--

Rick Marken (970818.0830)--

And I'd submit it to the heartland of operant research: JEAB.

If the argument is essentially qualitative, containing
neither the results of experiments nor a theoretical model,
JEAB would probably not publish it, but "The Behavior Analyst"
might very well.

You're probably right. Bruce A. has a considerable amount of data, but it
doesn't cover all the possibilities. Maybe it would be enough to raise the
issues, if not settle them.

I don't want to repeat Bill's whole post here, but what he's done is
essentially set up what is studied in concurrent schedules in EAB.
In fact, the issue of going to extinction versus going to relatively
lower frequencies on alternative A than B, and varying this relative
rate parametrically is exactly what the matching is based on (the
extinction case is just a special case). It's been with one alternative
being "non-contingent" as well and you get the same result.

I see that now that you point it out. Of course in the model I proposed
there was never any reinforcement for behavior B.

The "empirical" version of "reinforcement" which Bruce describes,
with only one controlled alternative source of food, is inadequate
for making the point. Maybe you think the deviations from perfect
matching complicates the matter too much, but the fact is that the
relativity inherent in a concurrent operant procedure is a nontrivial
property of even the "empirical" version of "reinforcement."

I think that the matching approach, even though it seems to support my
position, is still based on a false proposition: that for _each_ schedule,
an increment of reinforcement produces an increment of behavior. If that is
wrong, then the entire derivation of the matching law is also wrong. This
would mean that there is a reason for deviations of observation from
prediction: the wrong matching law is being used, and in fact matching as
such is not occurring.

I realize that saying this to you is as much a red tablecloth as is telling
Bruce A. that there is no such thing as reinforcement. But consider this.
It used to be that behaviorists looked down on "normal" psychologists
because of all the elaborate statistical manipulations they went through
just to see if anything happened, while EABers could get even a single rat
to behave in the predicted way with no problem. But now, with the matching
approach, EABers are finding themselves doing exactly the same thing. I've
seen some of the matching results -- maybe not the best, but they were
published without apology. In my opinion, it took a pretty good imagination
to see the predicted curve as "fitting" the data. Any number of matching
formulae, with very different underlying forms, would have fit just as well.

When your theory begins to differ from your data to this extent, you can
get into all sorts of complicated explanations of what is going on, and
what could prove you wrong? Not the data! "Pure reason" begins to loom
larger, and simple deduction from the observations takes a minor role. The
discipline of experimentation falls away, and you can get away with any
damned thing. People start calculating a lot less, and talking a lot more.

Of course the study of switching from one behavior to another is important
in itself, whatever theory lies behind it. The process of switching, as I
tried to show in the computer model I described to Bruce A., is a
phenomenon in itself, superimposed on whatever the animals are doing when
engaged in any one task. It's a higher-level phenomenon. But why study it
from the standpoint of a completely artificial principle called "matching?"
If matching can be shown to produce a benefit for the organism, fine -- but
then we would expect the benefit itself to be the controlled variable, with
matching being a side-effect. I can't conceive of any practical reason for
which an animal would match its behavior distribution to the reinforcement
distribution -- just for esthetic purposes, or to make Herrnstein's ghost
happy?

Varying
feeding outside the chamber might get you there, but, I think "variable means
to common ends" is more cleanly viewed through the concurrent lens.
As a corollary, I would say that the concurrent set-up, and the
relativity perspective, provide a pretty good way to divorce
"reinforcement" from common sense "reward."

In a way, all setups are concurrent setups, although the alternative
behaviors aren't always recorded (except, occasionally, by a video camera).
But I truly think that matching is a red herring. There's no payoff there,
that's my gut feeling. Oh, you can go deeper and deeper into it, revealing
more and more new relationships and trying to fit them into the whole
scheme, but that is true of ANY randomly-picked "theory." I'll bet that the
closest anyone has come to finding a rationale for matching is to think up
some kind of "maximizing" principle (those are very popular). Maximizing is
a very clumsy substitute for control, because if a variable isn't at its
maximum, the shortfall contains no information about which way to go to
raise it. Control _minimizes_ error, but the error contains more than just
magnitude information; it indicates _which way_ we have to go to reach the
minimum. So control systems are always simpler to implement than maximizing
or minimizing systems.

Ergo, Bill's "wasting" some of his arguments on a version of the
"reinforcement" concept that was displaced over a quarter of a century
ago. This isn't an ad hoc comment I throw in to defend EAB. I just
think you don't want to come off as arguing against an anachronism.

No, definitely not. But evidently not all EABers go along with the matching
idea, and some still consider that reinforcement can be defined in the
simple situation Bruce describes. Actually, I think they _all_ rely on that
simple definition, whether they use it in practice or not. The matching law
was derived from that simple definition, no matter what kind of schedule
you're thinking about (the main differences between FR and VI ratios is
that the relationship of behavior to reinforcement in the latter is
nonlinear, and has a lot of noise thrown in to make analysis harder and
make it harder to prove that the predictions don't match the theory).

A more technical point, but I guess now I've got to say something
about it. What I'm calling the relativity property essential to
the "modern" definition of the concept of "reinforcement" was first
observed with aggregate (not across subjects, but aggregates of
bouts at the concurrent alternatives) doesn't give you the blow-by-blow
of what happens each time the grain hopper opens. However, there's
been enough work (e.g., Vaughan's adaptation of the Rescorla-Wagner
formulation in support of the theory of melioration) to make it safe
for most EABers to say that some kind of a local interpretation
is possible. Again, I say this not to support the concept itself,
but indicate that you probably wouldn't want to discount the
relativity property, per se, on the basis that matching, per se,
is at level of of aggregation that cannot be said to express the
general concept of "reinforcement."

I'm afraid I lost the thread in that last sentence. Maybe you did too. Come
again?

Also, while Bruce has been consistent in saying that the organism
has to be adequately food-deprived, I don't know if the audience
on this list gets how crucial this is in formulating the "reinforcement"
concept from the EAB view.

I certainly understand it from the PCT point of view, but what does
deprivation have to do with reinforcement in EAB theory? Isn't the
relationship just an unexplained fact? Maybe you'd better explain what the
importance is.

Best,

Bill P.