reinforcement as an empirical fact; follow-up

CSGnet Archive CSG1995
1

[From Bill Powers (951212.0600 MST)]

Bruce Abbott (951211.1830 EST) --

     Reinforcement as defined in EAB is an empirical fact, not a theory.

On reflection, this is what you might call a deduction from a set of
hypothetico-empirical facts. Hypothetico-empirical facts are facts that
make so much sense within a frame of reference, or are so useful in
supporting a conclusion that simply has to be correct, that there is no
actual need to observe or test them.

      An operant is a member of a class of activities having a common
     consequence (e.g., depressing a lever to the point of switch-
     closure). When this consequence is linked to another (the later is
     made "contingent" on the former), the frequency (or some other
     aspect, depending on the contingency) of the operant is sometimes
     observed to increase as a result. If so, the contingent
     consequence of the operant has been demonstrated to reinforce the
     operant.

It stands to reason that any consequence could be produced by a number
of different actions (hypothesis). Therefore if key-pressing occurs, it
must have been caused by one of the many actions that can cause
keypressing. If many actions COULD cause keypressing, then many actions
DO, on different occasions, cause keypressing (hypothetico-empirical
fact). There is an internal process that tends to emit such acts, but
the tendency to emit any one of the acts is normally low.

If reinforcement is made contingent on keypressing, then whichever act
currently causes keypressing also causes appearance of a reinforcer, and
the reinforcer causes the tendency to emit that act to increase, a
result called reinforcement. Since the appearance of the reinforcer
COULD HAVE CAUSED the increase in tendency to produce the act, and since
we did observe an increase in the frequency of that act, the reinforcer
DID CAUSE the increase in the frequency of the act. It is not necessary
to observe the causal link, or test to see if it exists, because the
causal link is a hypothetico-empirical fact. Hypothetico-empirical facts
do not need proof.

Now let's try a different set of hypothetico-empirical facts. As can be
observed, when a contingency is established between a consequence of
action and the appearance of a food pellet, nothing happens at first. In
fact, nothing can happen until that consequence occurs in the right
pattern (number of repetitions, timing, etc.) because the contingency by
itself can't cause the reinforcer to appear. And the critical
consequence can't occur until one of the actions that can cause it has
occurred.

It is possible that the tendencies to produce all of the actions that
can cause the consequence are mostly suppressed. Since we observe these
actions occurring only at low rates, it stands to reason that they must
be suppressed by something, which we can call a suppressor of action.
When one of those actions does occur and results in appearance of a
reinforcer, it is possible that the reinforcer acts on the suppressor of
action and decreases its suppression of the act that has just occurred.
Thus the reinforcement does not cause an increase in the tendency to
emit the act, but removes some of the suppression of that act by
weakening the connection between the suppressor of action and that act.
Reinforcement is therefore the wrong name for the effect of the
rewarding object. The correct name is "desuppressor." Since all of the
other actions that could have produced the desuppressor when the
contingency is in effect have been almost totally suppressed, the action
that does produce a desuppressor is suppressed by a smaller amount and
tends to occur more often. This creates even more desuppression, and
that action quickly becomes predominant, while the others that produce
the same effect remain suppressed. So we can say that the contingent
consequence of the operant has been demonstrated to desuppress the
operant.

···

----------------------------------------
There is another way to describe the phenomenon which doesn't assert
either a positive or a negative causal influence of the food pellet, and
relies only on observable causal links.

With the contingency absent, no action can cause a food pellet to
appear. So if there is an increase in the frequency of one of the
actions that can depress the lever, there will be no effect on delivery
of food pellets.

With the contingency present, if (and only if) there is an increase in
the frequency of one of the actions that can produce the critical
consequence, the frequency of delivery of food pellets necessarily
increases because of the contingency, which is an observable causal
path. If the frequency of the behavior that creates the consequence
continues to rise, the frequency of delivery of food pellets also
continues to rise as a result. If that behavior rises to a final level
and remains constant at that level, the rate of delivery of food pellets
also rises to a final level and remains constant at that level. If the
behavior rate fluctuates, the food delivery rate also fluctuates, always
in the way described by the form of the contingency.

There is one more general observation we can make. When the contingency
is not in effect, it is seldom observed that the operant in question is
produced significantly more often than other operants (other actions
with a different common effect). With the contingency in effect, the
same operant is often observed to occur at a higher frequency than
ordinarily seen. So the presence of the contingency is associated with
increases of the frequency of occurrance of a particular operant, and
thus also with effects that depend on the operant such as delivery of
food pellets. However, merely establishing a contingency is not enough
to predict an increase in frequency of the operant, so the contingency
itself has no causal relation to the frequency of the operant.

In this account, we use only true empirical facts, actual observations,
and no hypothetico-empirical facts. We propose no causal links that we
can't observe, while we take into account the one causal link that we
can observe in complete detail, the contingency. We have said everything
non-theoretical that can be said about the observations.

There is one proposition in particular that does not appear in this
factual account. Nowhere is it proposed that occurrances of food pellets
have any effect on the frequency of occurrance of the operant act. On
the contrary, the changes in frequency of the food deliveries are
treated as strictly dependent on changes in frequency of the operant
acts, as determined by the form of the contingency.
---------------------------------
Given this factual account of the observations, there is, of course,
nothing to prevent our proposing mechanisms that might exist inside the
organism which explain the changes in frequency of the operant act. We
might propose, for example, a positive feedback connection inside the
organism, such that when the food delivery rate increases, the average
rate of production of the operant act increases, thus further increasing
the food delivery rate. Or we could propose a negative feedback
connection inside the organism, with the frequency of the operant act
depending on the difference between delivered food rate and some
reference rate defined by a system constant. We could propose a system
that detects both actions and food deliveries, and goes through a
systematic or stochastic search process to find the act that produces
the greatest yield of food.

There is, in short, any number of theories that could be offered to
explain the changes in operant rates. With several theories of equal-
appearing plausibility, we then have to think of experimental
manipulations that should have different effects according to each
theory, and use them to eliminate the theories that predict incorrectly.
---------------------------------
You claim that reinforcement is an empirical fact. It is not. It is a
conclusion drawn from a mix of actual observations and imagined or
hypothetico-empirical facts. When you present a description based only
on observable facts, the causal role of reinforcers never appears.
-----------------------------------------------------------------------
Best,

Bill P.

2

[From Bruce Abbott (951212.1450 EST)]

Bill Powers (951212.0600 MST) --

    Bruce Abbott (951211.1830 EST)

    Reinforcement as defined in EAB is an empirical fact, not a theory.

On reflection, this is what you might call a deduction from a set of
hypothetico-empirical facts. Hypothetico-empirical facts are facts that
make so much sense within a frame of reference, or are so useful in
supporting a conclusion that simply has to be correct, that there is no
actual need to observe or test them.

So, an empirical observation is now to be recast as a
"hypothetical-empirical fact," unsupported by observation or test. So, what
is your evidence? We read on:

     An operant is a member of a class of activities having a common
    consequence (e.g., depressing a lever to the point of switch-
    closure). When this consequence is linked to another (the later is
    made "contingent" on the former), the frequency (or some other
    aspect, depending on the contingency) of the operant is sometimes
    observed to increase as a result. If so, the contingent
    consequence of the operant has been demonstrated to reinforce the
    operant.

It stands to reason that any consequence could be produced by a number
of different actions (hypothesis).

This is not hypothesis, but pure logic: it is true according to the rules of
logic. An hypothesis is something asserted as true but which needs to be
tested in order to determine its truth-status.

Therefore if key-pressing occurs, it
must have been caused by one of the many actions that can cause
keypressing.

Again, pure logic: no hypothesis here at all.

If many actions COULD cause keypressing, then many actions
DO, on different occasions, cause keypressing (hypothetico-empirical
fact).

THAT is an hypothesis. Also, it is an hypothesis that has been empirically
verified as true in specific cases such as lever-pressing. Thus, this is
not a "hypothetical-empirical fact" (no proof necessary because it seems to
follow logically), it is a real, live fact, established by observation.

There is an internal process that tends to emit such acts, but
the tendency to emit any one of the acts is normally low.

If an act occurs, some process must have produced it. That is an assumption
(not "a hypothetico-empirical fact). Except perhaps in quantum physics, it
is also the customary assumption in science that things do not "just
happen." That "the tendency to emit any one of a given set of acts in
normally low" is true of most experimenter-defined operants (by design) but
would be established through observation; this is an empirical fact,
confirmed by observation, and not a "hypothetico-empirical" fact.

If reinforcement is made contingent on keypressing, then whichever act
currently causes keypressing also causes appearance of a reinforcer,

Let's get back to the observables. If, e.g., grain-presentation is made
contingent on keypressing, then whichever act currently causes keypressing
causes appearance of the grain.

and the reinforcer causes the tendency to emit that act to increase, a
result called reinforcement.

and the grain-presentation causes the tendency to emit that act to increase,
a result called reinforcement, AND grain-presentation is a reinforcer.

Since the appearance of the reinforcer
COULD HAVE CAUSED the increase in tendency to produce the act, and since
we did observe an increase in the frequency of that act, the reinforcer
DID CAUSE the increase in the frequency of the act. It is not necessary
to observe the causal link, or test to see if it exists, because the
causal link is a hypothetico-empirical fact. Hypothetico-empirical facts
do not need proof.

Hold on there! Did you not read my description of the empirical tests
required to establish that a given consequence is acting as a reinforcer?
In case you don't recall it, here is a summary:

1. Observe rate of operant prior to establishing contingency between
    putative reinforcer and operant.

2. Put contingency into effect; observe rate of operant. If operant rate
    increases, consequence MAY be a reinforcer.

3. Remove contingency; observe rate of operant. If operant rate decreases,
    consequence is PROBABLY a reinforcer.

4. Repeat steps 2 and 3 several times to determine whether the changes
    observed in steps 2 and 3 are REPLICABLE. If they are, consequence IS
    a reinforcer.

Thus, reinforcement is established as an empirical fact.

Let's review. All the facts I asserted are empirically verified through
observation and replication, or are logical truisms; none are these strange
beasties you define as "hypothetico-emperical."

Now let's try a different set of hypothetico-empirical facts. As can be
observed, when a contingency is established between a consequence of
action and the appearance of a food pellet, nothing happens at first.

When you say that "nothing happens at first," you mean that the series of
acts required to produce the pellet is not immediately completed.

In
fact, nothing can happen until that consequence occurs in the right
pattern (number of repetitions, timing, etc.) because the contingency by
itself can't cause the reinforcer to appear.

We have to talk about two "consequences" here, don't we? (1) Completing the
response requirement produces food as a consequence. This is true by
assumption, because it is how the experimenter has arranged the contingency.
(2) Delivery of food as a consequence of lever-pressing MAY result in an
increase in lever-pressing. If so, then the increase in lever-pressing is a
consequence of the contingency between lever-pressing and food-delivery. I
believe you are referring to the second case.

So to rephrase your statement, food cannot be delivered until the response
requirement has been met. Again, this is true because the experimenter sets
things up that way, and can be demonstrated empirically.

And the critical
consequence can't occur until one of the actions that can cause it has
occurred.

This is a bit ambiguous, but I take it to be a statment that one of the set
of possible actions that could complete the response requirement must occur
before the response requirement can be completed. This is a logical truth.

It is possible that the tendencies to produce all of the actions that
can cause the consequence are mostly suppressed. Since we observe these
actions occurring only at low rates, it stands to reason that they must
be suppressed by something, which we can call a suppressor of action.

In EAB such things are known to exist (demonstrated empirically) and are
called "punishers." If the operant is observed to occur at a low rate, it
may be unreinforced or it may be reinforced AND punished, in which case
withdrawal of the punishment contingency would be followed by an increase in
the rate of the operant. Thus your statement "it stands to reason" is false
in that there are at least two (and possibly more) reasons why the operant
may be observed to occur at a low rate. But go on.

When one of those actions does occur and results in appearance of a
reinforcer, it is possible that the reinforcer acts on the suppressor of
action and decreases its suppression of the act that has just occurred.

Sure, and situations such as you describe have been studied. The usual case
is that reinforcement and punishment effects are additive (but in opposite
directions), but other kinds of interaction are possible.

Thus the reinforcement does not cause an increase in the tendency to
emit the act, but removes some of the suppression of that act by
weakening the connection between the suppressor of action and that act.

Now you have left the realm of the empirical _fact_ of reinforcement and
entered the realm of reinforcement as a theoretical cause, by specifying a
"strengthening" or "weakening" effect on some kind of causal connection.

Reinforcement is therefore the wrong name for the effect of the
rewarding object. The correct name is "desuppressor." Since all of the
other actions that could have produced the desuppressor when the
contingency is in effect have been almost totally suppressed, the action
that does produce a desuppressor is suppressed by a smaller amount and
tends to occur more often. This creates even more desuppression, and
that action quickly becomes predominant, while the others that produce
the same effect remain suppressed. So we can say that the contingent
consequence of the operant has been demonstrated to desuppress the
operant.

Fine, I have no quarrel with your theoretical analysis, although it requires
some rather unusual and unsupported assumptions that violate the principle
of parsimony.

There is another way to describe the phenomenon which doesn't assert
either a positive or a negative causal influence of the food pellet, and
relies only on observable causal links.

With the contingency absent, no action can cause a food pellet to
appear. So if there is an increase in the frequency of one of the
actions that can depress the lever, there will be no effect on delivery
of food pellets.

True in the usual operant experiment.

With the contingency present, if (and only if) there is an increase in
the frequency of one of the actions that can produce the critical
consequence, the frequency of delivery of food pellets necessarily
increases because of the contingency, which is an observable causal
path.

Again true in the usual operant experiment. You read my post.

If the frequency of the behavior that creates the consequence
continues to rise, the frequency of delivery of food pellets also
continues to rise as a result. If that behavior rises to a final level
and remains constant at that level, the rate of delivery of food pellets
also rises to a final level and remains constant at that level. If the
behavior rate fluctuates, the food delivery rate also fluctuates, always
in the way described by the form of the contingency.

Yes, that is the nature of the contingency you described (although other
contingencies are possible).

There is one more general observation we can make. When the contingency
is not in effect, it is seldom observed that the operant in question is
produced significantly more often than other operants (other actions
with a different common effect).

That is probably true in most cases; I'll assume for the sake of the
argument that it is true in this case.

With the contingency in effect, the
same operant is often observed to occur at a higher frequency than
ordinarily seen.

Yes; if this is a reliable effect which can be demonstrated to be due to the
contingency, the consequence of the operant is a reinforcer and the increase
in responding due to the contingency is reinforcement.

So the presence of the contingency is associated with
increases of the frequency of occurrance of a particular operant, and
thus also with effects that depend on the operant such as delivery of
food pellets.

Yes.

However, merely establishing a contingency is not enough

to predict an increase in frequency of the operant, so the contingency
itself has no causal relation to the frequency of the operant.

To provide a physical analogy to your statement, you would assert that
merely establishing a contingency between pulling the trigger on a pistol
and the firing of the cartridge is not enough to predict the acceleration of
the bullet, so the pulling of the trigger has no causal relation to the
acceleration of the bullet.

This seems like a strange use of the phrase "no causal relation" to me.
Don't you mean that it is a cause, but not sufficient cause? If the powder
in the bullet is wet, the causal chain in broken. If the food is under
present conditions not an effective reinforcer (wet powder in the bullet),
the contingincy will be without effect. But that fact does not mean that
the contingency itself has no causal relation. You are asserting that the
release of the trigger per se has no causal relation to the acceleration of
the bullet because there are other links in the causal chain that may be
broken. What you mean to say (I think!) is that by itself it is not sufficient.

In this account, we use only true empirical facts, actual observations,
and no hypothetico-empirical facts. We propose no causal links that we
can't observe, while we take into account the one causal link that we
can observe in complete detail, the contingency. We have said everything
non-theoretical that can be said about the observations.

Yes. My account was similarly empirical and stuck only to the observations
and to statements that must be true according to the rules of logic. It,
too, avoided these "hypothetico-empirical" facts, as I demonstrated and
counter to your assertion otherwise.

There is one proposition in particular that does not appear in this
factual account. Nowhere is it proposed that occurrances of food pellets
have any effect on the frequency of occurrance of the operant act. On
the contrary, the changes in frequency of the food deliveries are
treated as strictly dependent on changes in frequency of the operant
acts, as determined by the form of the contingency.

Yes, but that effect COULD have been included, because that fact also can be
demonstrated empirically.

Given this factual account of the observations, there is, of course,
nothing to prevent our proposing mechanisms that might exist inside the
organism which explain the changes in frequency of the operant act. We
might propose, for example, a positive feedback connection inside the
organism, such that when the food delivery rate increases, the average
rate of production of the operant act increases, thus further increasing
the food delivery rate. Or we could propose a negative feedback
connection inside the organism, with the frequency of the operant act
depending on the difference between delivered food rate and some
reference rate defined by a system constant. We could propose a system
that detects both actions and food deliveries, and goes through a
systematic or stochastic search process to find the act that produces
the greatest yield of food.

Yes, now you're talking!

There is, in short, any number of theories that could be offered to
explain the changes in operant rates. With several theories of equal-
appearing plausibility, we then have to think of experimental
manipulations that should have different effects according to each
theory, and use them to eliminate the theories that predict incorrectly.

I agree. In fact, I've been working toward that end, although I think you
have yet to recognize it. Instead, you think I'm trying to prove the
correctness of reinforcement theory, or at least that's how it looks from here.

You claim that reinforcement is an empirical fact. It is not. It is a
conclusion drawn from a mix of actual observations and imagined or
hypothetico-empirical facts. When you present a description based only
on observable facts, the causal role of reinforcers never appears.

Your conclusion is false. My conclusion that reinforcement is an empirical
fact depends on observations which have established that the observed
changes labeled as "reinforcement" do occur as a result of a contingency
between a given operant and a given consequence of that operant. There are
no "hypothetico-empirical facts" of any kind in this set of observations.

When you present a description that leaves out the crucial observations, the
role of reinforcers never appears. Because I detailed these observations to
you in my post answering your challenge about contingencies, I must assume
that you were aware of them. But of course, to bring them in would have
destroyed your argument.

Reinforcement, the observed increase in the liklihood of a given operant
owing to the contingency between that operant and a particular consequence,
is an empirical _fact_. Period. Reinforcement, the theory, well, it's only
a theory, and not a very good one at that. I'm looking for a structural
model to replace it. Instead of trying to prove to me that reinforcement
theory is wrong (a waste of time, you're preaching to the choir), how about
helping me develop that model? I could sure use the help.

Regards,

Bruce

3

[From Kent McClelland (951213.1000 CST)]

Bruce Abbott (951212.1450 EST) responding to Bill Powers (951212.0600 MST) says,

So, an empirical observation is now to be recast as a
"hypothetical-empirical fact," unsupported by observation or test. So, what
is your evidence? We read on:

. . .

You claim that reinforcement is an empirical fact. It is not. It is a
conclusion drawn from a mix of actual observations and imagined or
hypothetico-empirical facts. When you present a description based only
on observable facts, the causal role of reinforcers never appears.

Your conclusion is false. My conclusion that reinforcement is an empirical
fact depends on observations which have established that the observed
changes labeled as "reinforcement" do occur as a result of a contingency
between a given operant and a given consequence of that operant. There are
no "hypothetico-empirical facts" of any kind in this set of observations.

When you present a description that leaves out the crucial observations, the
role of reinforcers never appears. Because I detailed these observations to
you in my post answering your challenge about contingencies, I must assume
that you were aware of them. But of course, to bring them in would have
destroyed your argument.

Reinforcement, the observed increase in the liklihood of a given operant
owing to the contingency between that operant and a particular consequence,
is an empirical _fact_. Period. Reinforcement, the theory, well, it's only
a theory, and not a very good one at that. . .

Evidently, your disagreement revolves around the definition of what
constitutes a fact (as Bill Powers implies in his later post (951212.1530
MST)). It seems to me that it might help to keep in mind that facts, like
other perceptions, are just perceptions. Facts are, of course,
"collectively controlled perceptions", because they have been agreed upon
as true by a relevant group of individuals who have been able to persuade
each other that they all see the things in question the same way, the way
they collectively deem as factual. Establishing the facts of the case
means coming to agreement over what "really" happened, that is, agreeing
upon a set of verbally communicable reference standards that can be used by
everyone participating in the collective action of perceptual control to
control without undo error their own perception of the alleged event or
phenomenon.

Mere observation by any individual, then, cannot establish facts. Rather
they must be socially constructed and maintained by a group of cooperating
fact-knowers, say a group of scientists (or, I shudder to think, a group of
lawyers!). While agreement in most cases increases the probability that
the fact as stated corresponds in some way with the aspect of "boss
reality" it purports to describe, social groups can and have been wrong in
their determination of facts, including perhaps even EAB scientists (and I
suppose, though only hypothetically of course, PCT researchers). So the
conclusion of my gratuitous sociological observation here is that
scientists can't start from facts to build their theories, because science
consists of establishing those facts in the first place. (It takes a good
theory to construct a robust fact!)

Probably you knew all that already. Sorry for the interruption!

Kent