"Wrong" relation of reinforcement to behavior

[From Bill Powers (941115.0830 MST)]

I haven't received the post, but Rick Marken (941114.2100) cites Bruce
Abbott (941114.1630 EST) as saying

It has been recognized since at least the 1970s that reinforcement is
"relative," (c.f. Baum, Reinforcement as Situation Transition). For
example, given that a rat is receiving 20 food pellets per hour, a
response that would increase this rate to 30 pellets per hour would be
reinforced, whereas one that cut this rate to 10 would be punished.

This is a generalization based on the theory that an increase in
reinforcement produces an increment in behavior, and a decrease produces
a decrement. There is a certain subset of experiments in which this
relationship is seen. However, there is another and far larger subset in
which the opposite relationship can be seen: an increase in
reinforcement produces a decrement in behavior, and a decrease produces
an increment.

Baum has it backward for most behaviors. In the normal range of control,
anything (including an increase in the animal's behavior rate) that
increases reinforcement rate will decrease the behavior rate, and
anything that decreases reinforcement rate will increase the behavior
rate. The only time we see the relationship Baum describes is under
extreme schedules corresponding to the left side of Fig. 7.18 in
Staddon. This, unfortunately, is the standard condition under which many
operant conditioning experiments are conducted, in order to avoid
"satiation" effects. In fig. 7.18, we can see that "satiation" effects
predominate over 3/4 of the range of behavior rates and reinforcement
rates. Under the conditions on the left side of 7.18, animals' control
systems are not operating normally; there is reason to think that the
animals are not even engaged in the primary task all of the time,
spending more and more time looking elsewhere for reinforcement as the
schedule becomes less and less forthcoming. This may in fact be the
whole explanation of why the overall average behavior rate does not keep
increasing, or at least remain at the maximum possible level, as
reinforcement rate declines completely to zero or until "extinction"
(reorganization) sets in.

In the normal range of control there is an essential addition to the
basic concept of reinforcement that is needed.

The basic concept says that behavior rate is controlled by reinforcement
rate. The addition is that behavior rate is _varied_ (not controlled) by
the _difference between_ the reinforcement rate actually being received
and the reinforcement rate that the behaving system considers
satisfactory. The result is that in the normal range of control, the
organism is controlling the reinforcement rate, not vice versa. The
controlling factor is the level of reinforcer internally specified by
the organism as satisfactory.

In operant conditioning terms, the desired reinforcement rate is the
level of reinforcement at which the organism appears totally "satiated."
That is, it is the rate of reinforcement just large enough to reduce the
behavior rate to zero. This is the reference _level_ (an observable
quantity) as formally defined in PCT, which is explained by a model in
which there is an internal reference _signal_ against which a perceptual
_signal_ representing the current rate of reinforcement is being
continually compared. See Glossary in B:CP under "reference level."

Satiation does not mean that the organism is unable to absorb any more
reinforcement. It means that the organism is receiving a satisfactory
amount of reinforcement, as the organism sees it. If control is two-way,
the organism will act to correct deviations of the rate of reinforcement
from the satisfactory level whether the deviation be above or below that
level. If control is one-way, the organism will act to bring the
reinforcement rate up to the satisfactory level (as nearly as possible),
but will do nothing to resist being presented with levels higher than

In the case of ingested food reinforcers, the control is two-way: a
normal rat will act to resist overfeeding just as it resists
underfeeding. Force-feeding a rat immediately turns off the behavior by
which it provides its own food for itself. This fact has been observed,
if not understood, in EAB. "Noncontingent reinforcement reduces operant
behavior." In obesity experiments, the two-way control of the
reinforcement rate (or its effects on body weight) by the behavior of
rats is well documented. There are many other published experiments in
which the "wrong" relationship of behavior to reinforcement is seen, but
generally the point of such experiments was to investigate some other
phenomenon, and this one has simply not been noticed (or at least
commented upon).



Bill P.