perpetual motion

CSGnet Archive CSG1995
1

[From Bruce Abbott (951204.1550 EST)]

Bill Powers (951202.2350 MST) --

    Bruce Abbott (951201.2105 EST)

    Ah, you cut me to the quick! I have indeed commented: check your
    archive. I don't remember which post, but my reply was given the
    last time you brought up the topic. As I recall, you asked me
    about it in the form of a test. After reading my reply, you said I
    passed. Remember?

I didn't ask you about the implications for the reinforcement
explanation.

O.K., my mistake. You DON'T want me to describe what control systems theory
implies about reinforcement.

You accept that actions vary as disturbances are applied to
the controlled variable; you definitely passed that question. But if
that's true, and if reinforcers are controlled variables, what does this
imply about the statement that reinforcement increases the probability
of the response that produces the reinforcement?

Hey, isn't this the question you just said you didn't ask? Now I'm really
confused . . . (:-/

With a varying
independent disturbance acting directly on the controlled variable, the
"responses" of the system will be varying over the whole possible range,
yet the system will continue to produce essentially the same amount of
"reinforcer." When no disturbances are present, as in almost all op-cond
experiments, a false impression is created that a particular response
always goes with a particular reinforcer. So a false generalization is
created: the idea that a particular consequence of a response is what is
bringing the response into being and maintaining it. Applying a
disturbance reveals the general case, and in the general case that
generalization is wrong. In the general case, there is no particular
relationship between reinforcement and a change in the probability of
any particular response. This can be experimentally demonstrated.

Yes, I recognize that, have for a long time. You don't think I do?

I hope you're not going to be like that amateur inventor. Sure, the same
value of the controlled variable can go with completely different
directions and amounts of response. Now back to the maintenance of
responses by their reinforcing consequences ...

Now wait a minute. Let's back up a bit. First, I don't define a reinforcer
as a value of the controlled variable (and neither, I believe, do you).
Second, I have noted that what traditional reinforcement theory defines as a
reinforcer appears to act as such only when the contingent event called the
reinforcer has the effect of reducing error in a controlled perception.
When the effect of a disturbance is such that the contingent delivery of the
so-called reinforcer would not reduce this error, it would not act as a
reinforcer; in fact if its effect were now to _increase_ the error, it would
actually appear to _suppress_ the behavior that produced it.

Although the concept of reinforcement becomes superfluous in this context,
one may still wish to show how a PCT analysis can account for the results of
EAB research. One way to really get the attention of folks who buy into the
reinforcement concept is to show that PCT resolves a longstanding difficulty
in reinforcement theory by showing how the ability of a contingent event to
"reinforce" (maintain) a response can be predicted from a knowledge of the
effect of the event on the relevant perceptual variable and the state of the
error signal. This is what I meant when I said that PCT is fundamental: the
apparent effect of "reinforcers" on steady-state behavior is a byproduct of
control-system operation, so long as disturbances do not interfere. I
thought I was making my position clear about this and am very surprised to
find you questioning by understanding of the situation. As I feared, you
appear to be mistaking the message (standard reinforcement-theory
interpretation) for the opinions of the messenger. Maybe that's why you
keep accusing ME of sloppy thinking, when all I'm doing is describing an
application of someone else's theory.

Now if you'll excuse me, I've got a perpetual motion machine to attend to.
If I don't drain some of the fuel out of the tank every once in a while, it
overflows and makes a mess. (;->

Regards,

Bruce

2

[From Bill Powers (951204.1520 MST)]

Bruce Abbott (951204.1550 EST) --

I didn't ask you about the implications for the reinforcement
explanation.

     O.K., my mistake. You DON'T want me to describe what control
     systems theory implies about reinforcement.

I meant I didn't ask you about it _then_, during the "test". I'd be
happy to hear what you have to say about it _now_.

     Now wait a minute. Let's back up a bit. First, I don't define a
     reinforcer as a value of the controlled variable (and neither, I
     believe, do you).

This you'll have to explain. When an organism acts in order to get food
for itself, it is at least trying to control the food intake (as the
external observer would see it) or its own state of nutrition or
fullness (as the organism sees it). Its means of doing this is through
the scheduling apparatus, and of course schedules can be set up that
make controlling the reinforcer very difficult and arduous. But the
basic organization is that of a control system, and the quantity being
controlled is what EABers call the reinforcer. So why wouldn't I call
the reinforcer the value of the controlled variable?

     When the effect of a disturbance is such that the contingent
     delivery of the so-called reinforcer would not reduce this error,
     it would not act as a reinforcer; in fact if its effect were now to
     _increase_ the error, it would actually appear to _suppress_ the
     behavior that produced it.

I don't think you've quite got the picture.

Actually, the behavior would be suppressed if the error were reduced.
Suppose the animal, with a reference level of 10 grams of food eaten per
hour, is pressing the bar and receiving 9.9 grams of food per hour. If
you then start adding a few extra pellets, say 1 gram per hour, the rate
of pressing will _fall_ until it is delivering 8.91 grams per hour and
the disturbance is supplying 1, for a total of 9.91. The intake is 0.1%
higher, the error is a 10% smaller, and the rate of pressing is 10%
lower. This is for a control system with a loop gain of 100 (counting
the schedule); for a system with lower loop gain, the numbers will be
somewhat different, but the relationships will be the same.

The behavior under normal conditions maintains the controlled variable a
little below the reference level. The remaining error is exactly what is
required to sustain that amount of behavior. If a disturbance tends to
_increase_ the value of the controlled variable (working in the same
direction as the behavior), the error will become smaller and the
behavior will decrease. A helping disturbance is therefore resisted just
as strongly as an opposing disturbance, through rapid relaxation of the
system's own efforts. Only if the disturbance were large enough all by
itself to drive the controlled variable higher than its reference level
would an increase in error call for more behavior; now the behavior
would be reversed, and would be working actively to reduce the
controlled variable (if a reversal could exist). An overfed cat simply
reverses its ingestion: it finds your shoe and throws up into it.

What EABers call a reinforcer is simply a physical quantity. In control
theory, the same physical quantity is called a controlled variable. It
may be true that EABers would have to rename the physical quantity if it
seemed to have a different effect on behavior, but it would still be the
same quantity and would still be called a controlled variable in PCT. Is
that what you're saying?

     Now if you'll excuse me, I've got a perpetual motion machine to
     attend to. If I don't drain some of the fuel out of the tank every
     once in a while, it overflows and makes a mess. (;->

Gee, maybe I should lend you my time-reversal machine. It cleans up
those messes as if they'd never happened.

···

-----------------------------------------------------------------------
Best,

Bill P.

3

[From Bruce Abbott (951204.2250 EST)]

Bill Powers (951204.1520 MST) --

    Bruce Abbott (951204.1550 EST)

    Now wait a minute. Let's back up a bit. First, I don't define a
    reinforcer as a value of the controlled variable (and neither, I
    believe, do you).

This you'll have to explain. When an organism acts in order to get food
for itself, it is at least trying to control the food intake (as the
external observer would see it) or its own state of nutrition or
fullness (as the organism sees it). Its means of doing this is through
the scheduling apparatus, and of course schedules can be set up that
make controlling the reinforcer very difficult and arduous. But the
basic organization is that of a control system, and the quantity being
controlled is what EABers call the reinforcer. So why wouldn't I call
the reinforcer the value of the controlled variable?

REINFORCEMENT: [Positive]: Anything that reduces intrinsic error.
  Powers (1973, p. 287)

I don't see anything about a reinforcer being THE value (whatever that is)
of a controlled variable. Did I miss it? [NOTE: Technically speaking, you
have defined a reinforcer (the consequence), not reinforcement, which is the
process.]

My definition is slightly broader than yours: I would eliminate the word
"intrinsic."

    When the effect of a disturbance is such that the contingent
    delivery of the so-called reinforcer would not reduce this error,
    it would not act as a reinforcer; in fact if its effect were now to
    _increase_ the error, it would actually appear to _suppress_ the
    behavior that produced it.

I don't think you've quite got the picture.

I'll try to explain what I had in mind later, when I'm a bit more rested.
Judging from your reply, we're not on quite the same wavelength here.

What EABers call a reinforcer is simply a physical quantity. In control
theory, the same physical quantity is called a controlled variable. It
may be true that EABers would have to rename the physical quantity if it
seemed to have a different effect on behavior, but it would still be the
same quantity and would still be called a controlled variable in PCT. Is
that what you're saying?

When you said that a reinforcer was THE value of the controlled variable, I
thought you were referring to the current value of the controlled varable
within the control system (what other value is there?). I hold to the view
that it is something that produces a particular _change_ in the value of
that variable, in the direction that reduces error.

    Now if you'll excuse me, I've got a perpetual motion machine to
    attend to. If I don't drain some of the fuel out of the tank every
    once in a while, it overflows and makes a mess. (;->

Gee, maybe I should lend you my time-reversal machine. It cleans up
those messes as if they'd never happened.

Please send it along right away -- I need to go back and rewrite some of my
earlier posts in the light of your comments so that I will not have had to
do all this reexplaining. That will have been quite a time saver!

By the way, my perpetual motion machine also works on a time-reversal
principle. As the flywheel spins, the machine draws energy from the
surrounding environment (a great benefit in the summertime as it helps to
keep the room cool), sucks in carbon dioxide and water vapor,
endothermically reacts these to produce gasoline and oxygen, expels the
oxygen into the air, and pumps the gasoline (unleaded of course!) into that
storage tank I have to keep emptying.

Regards,

Bruce

4

<[Bill Leach 951205.22:28 U.S. Eastern Time Zone]

[Bill Powers (951204.1520 MST)]

Actually, the behavior would be suppressed if the error were reduced.
Suppose the animal, with a reference level of 10 grams of food eaten per
hour, is pressing the bar and receiving 9.9 grams of food per hour. ...

Correct

The behavior under normal conditions maintains the controlled variable a
little below the reference level. The remaining error is exactly what is
required to sustain that amount of behavior. If a disturbance tends ...

Corrent

A helping disturbance is therefore resisted just as strongly as an
opposing disturbance, through rapid relaxation of the system's own
efforts. Only if the disturbance were large enough all by itself to
drive the controlled variable higher than its reference level would an
increase in error call for more behavior; now the behavior would be
reversed, and would be working actively to reduce the controlled
variable (if a reversal could exist). An overfed cat simply reverses its
ingestion: it finds your shoe and throws up into it.

Not Correct! This makes an assumption that is often being made without
proving by using the TEST and is, I think in this case quite unlikely to
be true.

The controlled perception(s) that result in what we observe as "eating"
are quite likely NOT the same perception(s) that will "resist a
disturbance that provides too much nutrient or too much gross food".

Even the "bi-directional" control observed in typical muscle control is,
we know, hardly "mirror image" control loops. I believe that some animal
(besides humans) experiments have shown that under the "right"
circumstances the animal can quite literally "eat itself to death" (and
this is not the "lets force enough sachrine into this rat ...).

-bill