Some questions about reinforcement

[From Bill Powers (951210.2145 MST)]

Bruce Abbott, Chris Cherpas, Sam Saunders --

I have an idea about what we have to do to get somewhere about the
subject of reinforcement theory. I asked Bruce, at the end of my last
post, "What reason is there for assuming that a reinforcer has any
effect at all on the behavior that produces it?" That question has been
returning to my mind all evening, and I finally realized that it's
absolutely the key question when it comes to deciding whether an idea is
mush or science. But I would rather put aside pejoratives, and just
explore the question. I will expand the question and address it to all
three of you, and anyone else who wants to chime in..

ยทยทยท

----------------------------
Here is a rat on a simple fixed ratio schedule. At first it does not
press the bar. As time passes it presses the bar more frequently, and
after enough time has passed it is pressing the bar at a fairly steady
average rate.

We can see that pressing the bar causes a contact closure, and that for
every m contact closures, the apparatus delivers a pellet of food. The
rat pauses in its pressing to collect and eat the pellet, and it then
returns to pressing.
-----------------------------
So now let me amplify my question.

Question 1.

Every now and then, the apparatus delivers a pellet of food. It is said
that these deliveries are "contingent" on the pressing of the lever. I
ask, therefore, what you mean by "contingent," and how you would go
about proving that the deliveries are "contingent" on the contact
closures. There are two steps to this proof: first, defining
"contingent" so it has an unequivocal meaning, and second, proving (with
experimental evidence, not just logically or by appeal to common sense)
that the evidence fits the definition.

Question 2.

As time passes, we observe that both the frequency of pellet delivery
and the frequency of lever-pressing increase, and that eventually both
frequencies come to a steady-state average value. It is said that a
pellet delivery "increases the probability" of a press, and that after
the steady state is reached, the continuing deliveries of pellets
"maintain" a steady frequency of pressing. I ask first that the concepts
of "increasing the probability" of a press and "maintaining" the
behavior be defined, and then (as in question 1), that a proof be
offered giving experimental evidence that there is in fact an effect of
the pellet deliveries on a "probability of pressing" or an effect that
"maintains" the pressing.

What I expect will happen, if you take these questions seriously and try
to give competent answers to them, is that you will have little trouble
with question 1 (once the method of proof occurs to you), and a great
deal of trouble with question 2. Question 1 can be answered by normal
empirical procedures. If you have answered question 1 correctly,
question 2 can't be answered, or so I claim.
-----------------------------------------------------------------------
Best,

Bill P.

[From Bruce Abbott (951211.1155 EST)]

Bill Powers (951210.1725) --

Bruces, I have a simple question to ask you about reinforcement theory.
What reason is there for assuming that a reinforcer has any effect at
all on the behavior that produces it?

"Bruces"? Ah, given the way I talk, you've probably concluded that I'm a
multiple personality. That's O.K.: I understand. (;->

Bill Powers (951210.2145 MST) --

Question 1.

Every now and then, the apparatus delivers a pellet of food. It is said
that these deliveries are "contingent" on the pressing of the lever. I
ask, therefore, what you mean by "contingent," and how you would go
about proving that the deliveries are "contingent" on the contact
closures. There are two steps to this proof: first, defining
"contingent" so it has an unequivocal meaning, and second, proving (with
experimental evidence, not just logically or by appeal to common sense)
that the evidence fits the definition.

Step 1: Defining "contingent"

The dictionary definition will do: "6. Logic: depending upon some condition
or upon the truth of something else." (Webster's New Collegiate)

B is contingent upon A if A is a sufficient condition for the occurrence of B.

Step 2: Proof that food deliveries are contingent on contact-closure.

B (feeder operation) must be shown to depend on A (closure of the contact).
Experimental test: repeatedly close/open the contact, note whether or not
the feeder operates. If feeder operates during or immediately after some
contact closures, there is a contingency between contact-closure and feeder
operation.

Detecting a contingency is easier when A is both a sufficient AND a
necessary condition for B. In that case, A---->B and NOT A------>NOT B.
When A is only sufficient, B can occur in the absence of A. If B occurs
frequently, it may be difficult to tell whether B is contingent upon A,
because then B may often coincide with A by chance. Things get even more
difficult if there is a significant delay between A and B, especially if
that delay is variable.

In the ratio-schedule example, the contingency is between the Nth contact
closure since last feeder operation and feeder operation. To produce the
Nth contact closure, there must first be N-1 prior contact closures. One
could think of A as "N-1 closures since last feeder operation AND closure"
(a sufficient condition for feeder operation). Or, put another way, feeder
operation is contingent on a contact closure, but this contingency is
present only if N-1 closures have already occurred since last feeder
operation. To detect contingencies like this, one may have to sample for
quite some time: few contact closures will be followed by feeder operation.

Whether you wish to call A a sufficient condition for the occurence of B
depends on how fine-grain you wish to make the analysis. If the power to
the feeder is removed, then B will not occur even though A has occurred, and
the contingency is broken. A whole set of conditions has to be true before
B will be contingent on A, but we assume these at the outset, then there is
nothing improper about saying that B is contingent upon A under those
assumed conditions.

Question 2

As time passes, we observe that both the frequency of pellet delivery
and the frequency of lever-pressing increase, and that eventually both
frequencies come to a steady-state average value. It is said that a
pellet delivery "increases the probability" of a press, and that after
the steady state is reached, the continuing deliveries of pellets
"maintain" a steady frequency of pressing. I ask first that the concepts
of "increasing the probability" of a press and "maintaining" the
behavior be defined, and then (as in question 1), that a proof be
offered giving experimental evidence that there is in fact an effect of
the pellet deliveries on a "probability of pressing" or an effect that
"maintains" the pressing.

Step 1: Defining "increasing the probability"

In operant experiments of this type, "probability" is not measured directly,
but is assumed to be reflected in the rate of responding. A press with a
constant probability of occurrence over time would occur at a constant
average rate; as this probability increases, rate goes up. "Increasing the
probability of a press" means increasing its instantaneous probability of
occurrence, as indexed by an increase in its observed average rate of pressing.

Step 2: Defining "maintaining the behavior"

Maintaining the behavior means keeping its average instantaneous probability
of occurrence (as reflected in rate of responding) above its value as
observed in the absence of the contingency.

Step 3: Proof that there is an effect of pellet deliveries on the probability
        of pressing.

1. Establish the rate of pressing in the absence of a contingency between
    pressing and pellet delivery. In most operant situations this rate will
    be low (but not zero), indicating a low instantaneous response probability.

2. Establish the rate of pressing when the contingency between pressing and
    pellet delivery has been put into effect. If, over time, the rate of
    pressing increases over baseline, the instantaneous probability of a press
    has increased.

3. Repeat (1). If the increase in response rate observed in (2) is due to the
    contingency, then removing the contingency should allow press rate to fall
    back to baseline levels.

4. Repeat (2) and determine whether the effect observed in (2) is replicated.
    (3) and (4) may be repeated as necessary to establish the reliability of
    the observed changes in response rate through replication.

Step 4: Proof that pellet delivery maintains pressing

This has already been demonstrated in Step 3. If pressing is maintained
during the contingency phase at a rate consistently above the rate observed
in the absence of the contingency, the contingency is a necessary and
sufficient condition for maintaining the elevated press rate under the
conditions of the test.

What I expect will happen, if you take these questions seriously and try
to give competent answers to them, is that you will have little trouble
with question 1 (once the method of proof occurs to you), and a great
deal of trouble with question 2. Question 1 can be answered by normal
empirical procedures. If you have answered question 1 correctly,
question 2 can't be answered, or so I claim.

I must have missed something critical in your assumptions, as I seem to have
answered both questions by normal empirical procedures.

Regards,

Bruces (specifically, the EAB personality)

[From Bruce Abbott (951213.1630 EST)]

Samuel Saunders (951212:16:57:17 EST)

All that aside, I wanted to at least express an intent to respond to Bill
Power's's two questions. I will comment here on question 1, and reserve
question 2 for later comment.

Very nice answer, Sam. I like your notion of "looking backward in time" to
determine whether every feeder-operation was associated with a contact
closure: it simplifies assessment of the role of contact-closure per se in
the contingency. Nevertheless, as you know, it is still true that from a
more general perspective an operant contingency involving contact-closure
includes more than just the closure. In ratio schedules, for example,
feeder-operation depends on the _count_ of contact closures since last
feeder-operation reaching a certain value, and on interval schedules it
depends on the passage of a certain interval of time since last
feeder-operation (during which contact-closures are irrelevant) and then a
contact closure. Looking backward would easily establish that every feeder
operation was accompanied by a contact-closure, but it would also be
important to note whether every contact-closure was followed by feeder
operation. If not, one would want to look further to identify the other
necessary conditions: completion of N-1 contact-closures since last feeder
operation, for example. Stated in terms of conditional probabilities, both
p(B|A) and p(A|B) are of interest when assessing a contingency. Also
relevant is the fact that event B is _independent_ of event A if p(B|A) =
p(B), where p(B) is the overall probability of B.

By the way, I haven't received a single post from CSG-L for nearly 24 hrs
now. Either there's been a holiday called I wasn't told about or Net
Gridlock has set in again. Wait a minute -- a post! It's from Rick, dated
10:46 pm yesterday, or 1:46 am my time: nearly 15 hours ago. I wonder where
TODAY's posts are?

Regards,

Bruce