Illusion of control; reinf theory; control defs; abstractions

[From Bill Powers (950606.0000 MDT)]

Bruce Abbott (950605.1210 EST) --

     Earlier research had shown that, in rats at least, shock that was
     controllable was less aversive and had milder physiological effects
     than otherwise equivalent shock that was uncontrollable.

Problem: what was the difference between shocks that were controllable
and shocks that were uncontrollable, when no matter what the rats did
the shocks occurred with the same intensity and according to an
unalterable schedule? It seems to me that you took great pains to assure
that the rats did NOT have any control over the shocks in the
experimental condition -- if, that is, your precautions actually
succeeded, so the rats' actions had no effect on their experiences of
the shocks.

     Earlier research had shown that, in rats at least, shock that was
     controllable was less aversive and had milder physiological effects
     than otherwise equivalent shock that was uncontrollable.

In those studies, how did "controllable" shock differ from "equivalent
uncontrollable" shock? The only sure way to measure the total intensity
of shocks would be to use an integrator to measure the total flow of
current through the rat over the time the circuit was supposed to be
turned on. An uncontrollable shock would be one that was measured to
deliver a fixed total charge through the rat regardless of the rat's
actions. Is that what these studies measured?

     But what if just HAVING control is itself a controlled perception?
     Maintaining that perception might have been "worth" a little extra
     effort. My study was designed to find out.

If the rats' behavior had no effect on the experienced shocks, then the
rats did not actually have any control, did they? The only evidence you
could possibly adduce to support the idea that a shock was controllable
was that the rat's actions had an effect on it that reduced the average
experienced shock rate, duration, or intensity. If the actions have no
such effect, then the only correct conclusion is that one has no
control.

     There is an apparently common-sensical idea that people who are
     experiencing stress in their lives should be given more control
     over the sorts of things that cause the stress, and that this
     control (even if illusory) will diminish the impact of those
     events. My research (and a little thought) suggests that this idea
     is not necessarily true.

I know; Langer has promoted these concepts, and Bill Glasser has decided
that her ideas are more germane to his "Control Theory" than mine are.
But the concept of illusory control is very hard to test if it does not
involve actual control -- that is, actual behavior-dependent mitigation
of the effects of stressful stimuli. If you create an illusion of
control without giving actual control, the illusion will break down
unless you arrange for the stimuli to be reduced whenever appropriate
behavior occurs. And in that case, what is the difference between
"illusory" and "real" control?

In our experiments where we replayed the cursor positions with the
handle disconnected, the "illusion of control" was sustained only as
long as the handle movements remained apparently correlated with changes
in cursor position. As soon as a cursor movement occurred which was
visibly different from the handle movement that supposedly caused it,
the illusion was destroyed. Of course the illusion could be indefinitely
sustained if an observer in the background was watching the subject's
handle movements and using another handle to make the cursor move in
synchronism with the subject's handle movements -- but then the control
would be real, if somewhat degraded. The observer would simply become
part of the control loop.

The only other way to disguise illusory control would be to make the
connection between action and result very uncertain, so it would be
difficult to judge whether behavior was having an effect. But in that
case, even real control would be hard to detect because the best
achievable control would be very poor. It would not be surprising if
subjects saw little to choose between have very poor and uncertain
control and actually having no control under conditions where it would
be hard to detect the lack of control even if you were looking for it.

     Having the perception of control will probably reduce the
     stressfulness of the situation if your experience suggests that
     this control will generally lead to an objectively better outcome
     than if you lacked control. For example, you may feel more relaxed
     when YOU drive the car than when your teenage son, who has had two
     weeks of driving experience, is behind the wheel.

Poor example, because both you and your teenage son actually have
control of the car when driving. What is stressful about riding with an
inexperienced driver is seeing all the errors that are not being
corrected. And there are differences in driving style even among good
drivers: how far before a stop sign do you start to apply brake
pressure, when approaching at 50 mph?

You get the perception of control when deviations in the controlled
variable are apparently corrected by your actions. While "real" control
may be difficult to verify in some circumstances, lack of control is
easy to verify: there is no effect of action on the controlled variable.

     However, having the perception of control will probably _increase_
     the stressfulness of the situation if your experience suggests that
     this control will generally lead to an objectively _worse_ outcome.
     For example, you might feel much more comfortable allowing the
     pilot to land the 737 you are flying in than taking the wheel
     yourself.

Why would having the perception of control increase the stressfulness of
the situation if you actually had control? The reason that you prefer to
let the pilot land the plane is your knowledge that merely sitting in
the pilot's seat and holding the control wheel would NOT give you
control over the airplane. You don't know how to fly it, particularly
how to land it. Even if you started with the illusion of control (and
were thus not bothered by the prospect of landing the plane), you would
soon find that it is an illusion, because the plane would not do what
you want it to do. Being made responsible for landing the airplane, or
thinking you can fly it, does not give you control of the airplane.

     In my study the rats were able to compare the objective outcome
     under controllable- and uncontrollable-shock conditions and
     determine that they were the same.

What they discovered was that in either situation, the outcome was
uncontrollable. I do not think there were any "controllable" shock
conditions.

···

---------------------------------
     One difficulty is that there _is_ such a thing as reinforcement--by
     definition. So when you tell a reinforcement theorist that there
     "ain't no such thing," you're bound to be met with a bit of
     skepticism.

You just have to separate what is observed from what is imagined. What
is observed is that animals will produce actions that produce food,
water, etc., and that given practice they will do this more and more
reliably. This way of putting it gives us a complete description of what
is observable, without ever mentioning "reinforcement" as a thing or a
process.

We can easily verify the "forward" link; we can show that the
consequences of behavior follow lawfully from the behavior. The laws are
simply those of physical science and engineering. We can also easily
verify that as the behavior gradually changes, the consequences change,
too. On these observations there will be no disagreement between
behaviorists and PCTers.

What is imagined is that the food, water, etc. are causing the behavior
of the organism to change as we see it changing. This "reverse link,"
however, is not observable or verifiable. We find that the only possible
verification reduces to a restatement of the observable "forward"
relationships.

The transition to PCT comes when we propose a model for the reverse
relationships, the ones connecting the consequence of behavior back to
the behavior via the organism. Doing this using correct mathematics
leads inevitably to the equations of a closed-loop system, and the
transition to a systems analysis of behavior rather than a cause-effect
analysis. We find that the so-called reinforcer is just a variable in a
closed loop of relationships. We learn to distinguish performance
(operation of a closed loop with fixed parameters) from learning or
adaptation (processes that alter the parameters of a given closed-loop
system).

That's one line of argument.
--------------------------------------------
     If I food-deprive a rat for a few hours and then give the rat the
     opportunity to earn a bit of food by pressing a lever, the rat
     learns to press the lever. The frequency of lever-pressing will
     increase over time and will be maintained at some higher level so
     long as the rat remains hungry and the contingency between lever-
     pressing and pellet delivery remains in effect.

What is important is that the food-deprived rat will learn to do
whatever is required to reduce the food deficit. By stopping at the
lever-pressing, you are only noticing one point in a closed loop. You
have to carry the analysis all the way around the loop, wherever you
started. If you started with a reduction in food intake, you have to
follow the process all the way around to an effect on food intake; the
effect is to act against the deficit. If you do the equivalent of
pressing the lever now and then, independently of the rat, you can
predict what will happen by tracing events all the way around to the
lever-depressions again: assuming that the system is inside the normal
control range, the rat will reduce its own lever-pressing, acting
against the externally-induced increase in rate of depression of the
lever. Disturbances applied anywhere in the loop are resisted.

     The observed increase in responding is _by definition_
     reinforcement.

But notice that if you artificially elevate the rate of delivery of
food-pellets, the most likely effect is a _reduction_ in the rat's rate
of lever-pressing, because (going the rest of the way around the loop),
the change in food delivery rate is opposed by the change in the
behavior rate. This assumes, of course, that the system is inside the
normal control range, and not at an extreme of deprivation where the
relationships reverse.

You have still never explicitly commented on this problem: that you get
the assumed relationship between increases of behavior and increases in
reinforcement only for the most extreme degrees of deprivation. Is this
something that behaviorists have simply agreed to keep quiet about? Some
of them certainly haven't agreed to this.

     Thus the argument is not whether there is or is not a phenomenon
     which has been labeled "reinforcement," but how this phenomenon is
     to be explained.

But in fact, over most of the possible range of schedules, the
relationship observed is that an increase in reinforcement rate leads to
a decrease in behavior rate. So I would say that there is a serious
problem with assuming that there is no argument over the existence of
the phenomenon. I can make a good case that the phenomenon is largely an
artifact of experimental conditions. Standard experimental conditions
have been adjusted until the theoretically expected relationship is
observed.

     Reinforcement theory holds that certain sensory consequences of
     responding cause changes to occur in the nervous system that make
     it more likely that the response will occur again. PCT holds that
     responding increases because error between a controlled perception
     and its reference level generates output that tends to reduce the
     error; given the environmental feedback function, reduction of
     error requires an increase in (lever-pressing) output.
     Reinforcement theory makes reinforcement the central explanatory
     principle for behavior change; PCT makes it a side-effect of
     control. The argument is not about the objective phenomenon of
     reinforcement but its theoretical significance.

But PCT predicts that any externally-induced change in the rate of
reinforcement will result in an _opposite_ change in the rate of
behavior, while reinforcement theory predicts that the change in rate of
behavior will be _in the same direction_ as the change in rate of
reinforcement. This is a very clear and unequivocal contradiction.

We can find copious data in the published literature to support both
predictions. When we examine the data to see how this can possibly be
true, we find that the data supporting each side were taken under very
different experimental conditions. The PCT prediction is supported
clearly by data taken under conditions of mild to large degrees of
deprivation, but generally those degrees of deprivation that could still
allow life to continue if extended indefinitely. The reinforcement-
theory prediction is supported under conditions of extreme deprivation,
under which the animal could not survive.

The data I know about show that the transition (for food reinforcement)
occurs at a rate about 1/3 of the normal free-feeding food intake. A
human being who consumes about 3000 calories per day would just be on
the boundary between the two conditions on a diet providing 1000
calories per day. At this level of input, reinforcement theorists would
say that "satiation" has occurred, because further increases in food
intake would fail to increase the behavior rate (under the same kinds of
ratio schedules used to take the rat data). To get a clear demonstration
of the positive relationship between reinforcement and behavior, the
food intake would have to be in the range of perhaps 100 to 800 calories
per day. In other words, human subjects would be starving to death.

Starvation is warded off in operant conditioning experiments by
adjusting the food intake between experiments. Reducing a rat's weight
to a stable 80 percent of free-feeding weight requires, I would guess,
lowering average daily food intake by less than 10 percent. In obesity
experiments, it has been found that rats with hypothalamic lesions, who
may stabilize their free-feeding weight at 300% of preoperative weight,
eat only about 5% to 10% more than before, after they reach equilibrium.

Therefore a 70% reduction in food reinforcement rate, if sustained,
would undoubtedly be lethal. And _at least_ this degree of deprivation
during an experiment is required in order to show a clear positive
relationship between behavior and reinforcement.

So the very existence of reinforcement as a phenomenon is in question.
-----------------------------------------------------------------------
Bruce Abbott (950605.1545 EST) --

Interesting quote from _Design of feedback control systems_, especially
this:

     Such an open-loop control system has the advantage of simplicity,
     but its performance is highly dependent upon the properties of the
     plant, which may vary with time.

The alleged "simplicity" depends to an extreme degree on the simplicity
of the plant. If the plant has even a single time integration in it, the
open-loop system has to be provided with computing capacity to calculate
the inverse of the plant response to inputs, so that the reference
signal can be passed through a computation that puts the inverse of the
plant dynamics into the controller. For even modestly complex plants,
calculation of the required inverses can soak up huge amounts of
computing power and time. In general, a feedback model will accomplish
the same result in a far simpler and faster way requiring very much less
computation.

     [Bill P. will be happy to see this next one included:]

     4. Increased speed of response and bandwidth. Feedback may be
     used to increase the range of frequencies over which a system will
     respond and to make it respond more desirably. A satellite booster
     rocket, for example, has aerodynamics resembling those of a giant
     broomstick. It may, with feedback, behave with beauty and grace.

You bet I'm happy to see it. So how did this "feedback is too slow" shit
get such a foothold in the literature?
-----------------------------------------------------------------------
Bill Leach (last few days or a week) --

I haven't been acknowledging your great posts much, but the reason is
that I get tired of saying yes, yes, yes.

One thing, though, that I must acknowledge: your comments on Bruce
Buchanan's latest offerings. It would be very easy to sit back and let
you stick your neck out while I avoid the flak from saying the same
things. But I agree totally with your assessment of Buchanan's abstract
wanderings. With respect to any kind of knowledge I'm interested in, or
anything I'm interested in doing, they are useless. Thanks for saying it
so clearly.
----------------------------------------------------------------------
Best to all,

Bill P.