Fred's Home Heating System

CSGnet Archive CSG2000
1

[From Bruce Abbott (2000.11.23.1105)]

Once upon a time there lived a man who liked to invent and build things for
his own use. He lived to a good old age but eventually he passed away and
his house was put on the market. The house was purchased and it didn't take
Fred, the new owner, long to realize that the furnace system in his new home
was no ordinary system, and he wondered how he would get it fixed if the
thing ever failed. Something of a tinkerer himself, Fred began to study it
carefully, hoping to gain an understanding of its operating principles.

It turned out to be quite a different system from the usual. When the
temperature of the house was below a certain value, a relay closed
momentarily and this activated a mechanism that squirted a small amount of
coal dust into the fire box of the furnace, where a spark ignited the dust.
This produced a flash of heat, which was detected by a sensor in the fire
box. When the coal dust had burned, the sensor saw that there was no fire
and triggered the relay to close again, producing another flash of heat, and
this continued at a rapid rate, machine-gun fashion. All the heat being
generated warmed a heat-exchanger, which warmed the air in the bonnet, and a
fan then distributed the warmed air through the ductwork to the various
rooms of the house. The house eventually warmed up to the required value,
and this altered the circuit so that the disappearance of heat from the
burning coal dust no longer initiated another cycle of relay closure and
injector operation. The furnace shut down and stayed in this state until
the house cooled sufficiently so that the lack of heat in the fire box once
again closed the relay, and began the cycle anew.

Sometimes the coal dust failed to squirt from the injection mechanism. If
this happened, there would be no flash, the sensor would detect that there
was no fire, and command the injector to operate again. The injector wasn't
always reliable, but usually after three or four commands it would work
again. However, if no fire occurred after a fairly large number of
injection cycles, a stepper mechanism would operate and select another
injector (there were several injectors installed). If this new injector
failed to produce heat within a few cycles, then the stepper operated again.
This continued until heat was detected again.

Fred explained the system's operation as follows. The little bursts of heat
produced when the coal dust ignited, he said, were serving as reinforcers
for relay action. However, these reinforcers became effective only after the
house had cooled below a particular value; cooling the house thus served as
an "establishing operation" for reinforcer effectiveness. The system
generates responses, Fred stated, in the form of the pulses sent to the
currently selected injector. Those responses repeat as long as they
continue to be reinforced (by heat flashes) at least once every few
responses. However, if reinforcement is not delivered after a number of
responses, this behavior extinguishes and the system begins to vary its
actions (the stepper operates), selecting new responses (commanding
different injectors) until reinforcement once again occurs. As the room
heats up, the reinforcers lose their effectiveness (satiation) and
responding stops.

Fred's neighbor Bill, a heating and air-conditioning expert, heard about
Fred's heating system and his rather unconventional explanation of its
operating principles. I rather imagine we'll be hearing a different
explanation from Bill any minute now.

Bruce A.

2

[From Rick Marken (2000.11.23.0930)]

Bruce Abbott (2000.11.23.1105) --

I don't think it's possible to do quality science by cute story.
I find it much more satisfying to do science by modeling and
experiment. So when you get a chance could you tell me what you
think of my proposed experiment [Rick Marken (2000.11.22.1330)]
that distinguishes the reinforcement and control theory models
of learning.

Thanks

Rick

···

--
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: marken@mindreadings.com
mindreadings.com

3

[From Bruce Abbott (2000.11.23.1305 EST)]

Rick Marken (2000.11.23.0930) --

Bruce Abbott (2000.11.23.1105)

I don't think it's possible to do quality science by cute story.
I find it much more satisfying to do science by modeling and
experiment. So when you get a chance could you tell me what you
think of my proposed experiment [Rick Marken (2000.11.22.1330)]
that distinguishes the reinforcement and control theory models
of learning.

Well, Rick, you're right. It is not possible to do quality science by cute
story. Cute stories sometimes do, however, serve other worthwhile purposes.
If you really think about the implications of this particular cute story,
you might come to understand why your proposed experiment would not provide
a test of reinforcement theory. So do us all a favor -- think about it. I
mean _really_ think about it. Deeply. Thus far, I don't think you have.

Bruce A.

4

[From Bill Powers (2000.11.23.1130 MST)]

Bruce Abbott (2000.11.23.1105)--

Once upon a time there lived a man who liked to invent and build things for
his own use. He lived to a good old age but eventually he passed away and
his house was put on the market. The house was purchased and it didn't take
Fred, the new owner, long to realize that the furnace system in his new home
was no ordinary system, and he wondered how he would get it fixed if the
thing ever failed.

I thknk I've figured out who that former owner was -- a fellow named R.
Goldberg. I hope he kept his day job.

Best,

Bill P.

5

[From Rick Marken (2000.11.23.1100)]

Bruce Abbott (2000.11.23.1305 EST)--

If you really think about the implications of this particular
cute story, you might come to understand why your proposed
experiment would not provide a test of reinforcement theory.
So do us all a favor -- think about it. I mean _really_ think
about it. Deeply. Thus far, I don't think you have.

I've thought about it about as deeply as I can. If you don't
think my proposal is an appropriate test of the difference
between reinforcement and control theories of learning, it
would help if you could explain what you would change about the
experiment to make it a better test.

Thanks

Rick

···

--

Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: marken@mindreadings.com
mindreadings.com

6

[From Bruce Abbott (2000.11.23.2105 EST)]

Bill Powers (2000.11.23.1130 MST)

Bruce Abbott (2000.11.23.1105)

Once upon a time there lived a man who liked to invent and build things for
his own use. He lived to a good old age but eventually he passed away and
his house was put on the market. The house was purchased and it didn't take
Fred, the new owner, long to realize that the furnace system in his new home
was no ordinary system, and he wondered how he would get it fixed if the
thing ever failed.

I thknk I've figured out who that former owner was -- a fellow named R.
Goldberg. I hope he kept his day job.

Me, too. However, once you're done laughing, I hope you'll give the system
a careful analysis. For starters, given my description of the mechanism, do
you agree that if actually built it would work as I described?

Bruce A.

7

[From Bruce Abbott (2000.11.24.750 EST)]

Rick Marken (2000.11.23.1100)]

Bruce Abbott (2000.11.23.1305 EST)--

> If you really think about the implications of this particular
> cute story, you might come to understand why your proposed
> experiment would not provide a test of reinforcement theory.
> So do us all a favor -- think about it. I mean _really_ think
> about it. Deeply. Thus far, I don't think you have.

I've thought about it about as deeply as I can. If you don't
think my proposal is an appropriate test of the difference
between reinforcement and control theories of learning, it
would help if you could explain what you would change about the
experiment to make it a better test.

It would help even more if you understood _why_ your current proposal is
_not_ an appropriate test.

Bruce A.

8

[From Bill Powers (2000.11.24.0839 MST)]

Bruce Abbott (2000.11.23.2105 EST)--

... once you're done laughing, I hope you'll give the system

a careful analysis. For starters, given my description of the mechanism, do
you agree that if actually built it would work as I described?

Well, sure, because you've said _what_ it does without specifying fully
_how_ it does it. I can design a car that will rise into the air when I
pull back on the steering wheel -- but if I don't say how that is going to
work, I've left out the most important part of the design, haven't I?

Your proposal was:

When the
temperature of the house was below a certain value, a relay closed
momentarily and this activated a mechanism that squirted a small amount of
coal dust into the fire box of the furnace, where a spark ignited the dust.
This produced a flash of heat, which was detected by a sensor in the fire
box. When the coal dust had burned, the sensor saw that there was no fire
and triggered the relay to close again, producing another flash of heat, and
this continued at a rapid rate, machine-gun fashion. All the heat being
generated warmed a heat-exchanger, which warmed the air in the bonnet, and a
fan then distributed the warmed air through the ductwork to the various
rooms of the house. The house eventually warmed up to the required value,
and this altered the circuit so that the disappearance of heat from the
burning coal dust no longer initiated another cycle of relay closure and
injector operation. The furnace shut down and stayed in this state until
the house cooled sufficiently so that the lack of heat in the fire box once
again closed the relay, and began the cycle anew.

What's missing here is any statement about how the warming of the house
"alters the circuit so that the disappearance of heat from the burning coal
dust no longer initiated another cycle of relay closure and injector
operation." You wave your arms and it happens that way: abracadabra. Also
there are some problems with the tendency of accumulated coal dust to
explode violently rather than just burning when ignited, and how to feed
coal dust to an injector so it can be emitted in uniform quantities with
the right dispersion in sufficient amounts to heat a house, and so on. But
these are all quibbles, because this is not a serious design for a real
system. It's an allegory, supposedly an analogy to a rat pressing a lever
repeatedly to provide jolts of food for itself, with some more global
effect of eating shutting down the trigger mechanism when the effect
reaches some reference level. Alternatively, the shortfall of the global
effect below the reference level provides the signal that activates the
eating, which is just another way of saying the same thing.

I can see this as a sketch of a possible control-system model of a rat's
behavior. It would not be a good model for reinforcement, because the
global effect of eating _reduces_ the eating rather than increasing it. And
anyway this is a performance model, not a learning model, so it doesn't
really apply to reinforcement.

If you had some other analysis in mind, perhaps you'd better just say what
it is. Do you have any comments about my comments about the role of
disturbances in distinguishing the control model from the behaviorist's model?

Best,

Bill P.

9

[From Rick Marken (2000.11.24.0830)]

Bruce Abbott (2000.11.24.750 EST)

It would help even more if you understood _why_ your current
proposal is _not_ an appropriate test.

Of course it would help. But obviously I _don't_ understand why
my proposal is not an appropriate test. If I _did_ understand
why, then I wouldn't have proposed it. I'm currently under
the impression that my experiment provides an excellent test
that discriminates reinforcement from control theory. I'm hoping
that you will disabuse me of that misconception (if necessary)
and describe what would be an appropriate test, either by making
changes to my proposal or by making an entirely new proposal.
Just as a reminder, here is a general description of my proposed
experiment:

The organism learns to reproduce some specific consequence under
conditions where a specific disturbance requires specific action
variations in order to reproduce it. Once the organism has
learned the action variations that reproduce the consequence, a
new disturbance is introduced so that _different_ action variations
are required to reproduce the consequence.

Thanks

Rick

···

--
Richard S. Marken Phone or Fax: 310 474-0313
MindReadings.com mailto: marken@mindreadings.com
www.mindreadings.com

10

[From Bruce Abbott (2000.11.24.1410 EST)]

Bill Powers (2000.11.24.0839 MST) --

What's missing here is any statement about how the warming of the house
"alters the circuit so that the disappearance of heat from the burning coal
dust no longer initiated another cycle of relay closure and injector
operation." You wave your arms and it happens that way: abracadabra.

Yes, that's true, but you and I are both clever enough to figure out how do
to it; it isn't as if we have to appeal to magic.

Also
there are some problems with the tendency of accumulated coal dust to
explode violently rather than just burning when ignited, and how to feed
coal dust to an injector so it can be emitted in uniform quantities with
the right dispersion in sufficient amounts to heat a house, and so on. But
these are all quibbles, because this is not a serious design for a real
system. It's an allegory, supposedly an analogy to a rat pressing a lever
repeatedly to provide jolts of food for itself, with some more global
effect of eating shutting down the trigger mechanism when the effect
reaches some reference level. Alternatively, the shortfall of the global
effect below the reference level provides the signal that activates the
eating, which is just another way of saying the same thing.

I'm glad you said that. For a moment there I thought you were going to opt
out on a technicality . . .

I can see this as a sketch of a possible control-system model of a rat's
behavior. It would not be a good model for reinforcement, because the
global effect of eating _reduces_ the eating rather than increasing it.

If you think that, then you need to be made aware that you really don't
understand how EABers think of reinforcement. I'll explain presently.

And
anyway this is a performance model, not a learning model, so it doesn't
really apply to reinforcement.

Incorrect again -- it _is_ a learning model.

The learning takes place based on the system's experience with producing
heat via commands to the injectors. When no heat occurs after many attempts
(commands), the stepper operates and selects a new injector. This is
analogous to the response failing to produce a food pellet. [Please note:
Operation of the stepper is an admittedly oversimplified representation of
the process whereby the subject abandons the response and begins to produce
new ones, but good enough, I think, for the present purpose of
illustration.] If the new responses (commands to the new injector) are
followed by reinforcement (heat), this output remains selected, but if,
after a few tries, no heat occurs, the selector operates again and another
response is tried.

So, what keeps a given "response" selected? Continued "reinforcement."
What happens if "reinforcement" ceases? The response is abandoned
("extinction"). What determines which response occurs? Two things do: (1)
which responses have been "tried," and (2), which, of those tried, produces
the reinforcer. Thus, because a response is followed by reinforcement, it
is selected for repetition in the future. Continued reinforcement of that
response assures that it will be maintained; failure to be reinforced
assures that it will be abandoned. This is what Skinner called "selection
by consequences."

As for reinforcement weakening the response in the long run (responses stop
when room temperature reaches the set point), this is not contrary to
reinforcement theory. Empirically, some things become reinforcing only
under certain conditions. If repeated delivery of the reinforcer has a
side-effect that removes those conditions, the reinforcer's ability to
reinforce diminishes to zero, and the behavior producing the reinforcer ceases.

Do you have any comments about my comments about the role of
disturbances in distinguishing the control model from the behaviorist's model?

Good question. At this point I'm not entirely sure. The problem for me is
that I can imagine how a reinforcement process could create a control
system. Also problematic is deciding what is learned during the acquisition
of the behavior.

Bruce A.

11

[From Bruce Gregory (2000.1124.1553)]

Bruce Abbott (2000.11.24.1410 EST)

Good question. At this point I'm not entirely sure. The problem for me is
that I can imagine how a reinforcement process could create a control
system. Also problematic is deciding what is learned during the acquisition
of the behavior.

I think the fundamental problem is that there is no working model of
reinforcement. Reinforcement is a just-so story. No way to refute it and no
way to discriminate between it and control theory. It's like the "will of
God". It explains everything and therefore nothing. To demonstrate that I
am full of hot air, why don't you just describe an experiment that _would_
allow us to reject reinforcement theory, as Rick has been asking for?

BG

12

[From Rick Marken (2000.11.24.1310)]

Bruce Abbott (2000.11.24.1410 EST)--

If the new responses (commands to the new injector) are followed
by reinforcement (heat), this output remains selected, but if,
after a few tries, no heat occurs, the selector operates again
and another response is tried.

In the rat bar press situation what is the "response" selected by
reinforcement: 1) the neural "commands" sent to the muscles 2) the
changes in muscle tension resulting from these "commands" 3) the
forces on the bar resulting from the muscle tensions 4) the change
in the position of the bar resulting from these forces 5) all of
the above 6) none of the above?

The answer might help me understand what is wrong with my experi-
mental test of reinforcement theory.

Best

Rick

···

---
Richard S. Marken Phone or Fax: 310 474-0313
MindReadings.com mailto: marken@mindreadings.com
www.mindreadings.com

13

[From Bill Powers (2000.11.25.0116 MST)]

Bruce Abbott (2000.11.24.1410 EST)--

... you need to be made aware that you really don't
understand how EABers think of reinforcement.

I'm basing my comments on Fred's explanation. Perhaps you would want to
modify it, but here it is:

Fred explained the system's operation as follows. The little bursts of heat
produced when the coal dust ignited, he said, were serving as reinforcers
for relay action. However, these reinforcers became effective only after the
house had cooled below a particular value; cooling the house thus served as
an "establishing operation" for reinforcer effectiveness. The system
generates responses, Fred stated, in the form of the pulses sent to the
currently selected injector. Those responses repeat as long as they
continue to be reinforced (by heat flashes) at least once every few
responses. However, if reinforcement is not delivered after a number of
responses, this behavior extinguishes and the system begins to vary its
actions (the stepper operates), selecting new responses (commanding
different injectors) until reinforcement once again occurs. As the room
heats up, the reinforcers lose their effectiveness (satiation) and
responding stops.

These are theoretical interpretations interspersed with observations of
what happens. The use of terms like "response," "reinforcement," and
"extinction" introduce Fred's theory about what is happening. Let's see if
we can separate the interpretation from the description.

What is observed is that activations of the igniter (not "responses")
generate heat flashes. We observe a physical mechanism by which the puffs
of coal dust and electric sparks cause a flash of heat. These flashes,
repeated, are seen to warm the house. When the temperature of the house
rises to some specific level, the activation of the igniter ceases. If the
igniter fails, when repeatedly activated, to cause a flash, nothing changes
immediately, but if the flash fails to occur for some time, causing the
room temperature to drop significantly, a new behavior occurs: new injector
nozzles (and ignitors, I guess) are rotated into place. This behavior
continues until the house temperature begins to rise again because a
working nozzle is in place. Then we revert to the previous behavior: a
(small) decrease in temperature results in activations of the igniter, and
an increase is followed by cessation of the repeated activations.

I believe this is an unbiased description of what is observed, favoring
neither reinforcement theory nor control theory. You may want to correct it
if I have misrepresented your intention, but I'll go on.

Now Fred's embellishments of this bare description:

1. Something is "responding" to some unnamed stimulus by sending impulses
to the igniter. This unnamed stimulus is not itself observable; all we
observe is that there are impulses sent to the igniter, and that they cause
flashes of heat.

2. The fact that the impulses continue to be emitted is interpreted as
meaning that the flashes of heat are "reinforcing" the responses to the
unnamed stimuli. This reinforcing action can't be observed; all we see is
that the igniter continues to operate.

3. When the nozzle quits working, the "responding" is no longer
"reinforced," so it "extinguishes." This does not imply, however, that
impulses cease to be sent to the igniter. If they were, changing nozzles
would do no good: there would never be another flash of heat even with a
working nozzle in place.

4. When reinforcement ceases," ... the system begins to vary its
actions (the stepper operates), selecting new responses (commanding
different injectors) until reinforcement once again occurs." Of course it
is not reinforcement that is observed to occur, but flashes of heat. Their
reinforcing effect is imagined. The system varies its actions until flashes
of heat occur, and then (when the house starts to warm up) the variations
cease.

5. Note that since the emission of impulses must continue if a flash of
heat is to occur, what is reinforced becomes uncertain. The only observable
result of reappearance of flashes of heat is that new nozzles cease to be
rotated into place. Emission of impulses must already have been occurring,
and it continues as long as the room temperature is low enough.

6. "As the room heats up, reinforcers lose their effectiveness (satiation)
and responding stops." Satiation is introduced as an explanation of why the
igniter ceases to be operated when the temperature rises to a certain
level. What is observed is not a loss of effectiveness of the reinforcers,
of course, but cessation of igniter operations.

I think it's clear that Fred's description of what is happening is not a
description, but a theory that invokes unobservable factors and influences
(as all theories do). A purely factual description does not require the use
of terms like reinforcement, response, and extinction: ordinary English
will do it (for an English-speaker).

Best,

Bill P.

14

[From Bruce Gregory (2000.1125.0642)]

Bill Powers (2000.11.25.0116 MST)

I think it's clear that Fred's description of what is happening is not a
description, but a theory that invokes unobservable factors and influences
(as all theories do). A purely factual description does not require the use
of terms like reinforcement, response, and extinction: ordinary English
will do it (for an English-speaker).

Very nice. I've performed a similar "deconstruction" of a typical EAB
experiment. It's fun, so I won't spoil it for anyone who wants to give it a
try. Suffice it to say, the experimenter arranges things so that his
behavior is properly "reinforced".

BG

15

[From Bruce Abbott (2000.11.25.1125 EST)]

Bill Powers (2000.11.25.0116 MST) --

I see you're receiving congratulations on your reply even before I've had a
chance to respond to it. That's disappointing, as it shows that at least
some CSGneters have their minds already made up and have taken up
cheerleading rather than serious consideration of my argument. But for
those who have managed to stay open to the argument, here is my return salvo:

Bruce Abbott (2000.11.24.1410 EST)

... you need to be made aware that you really don't
understand how EABers think of reinforcement.

I'm basing my comments on Fred's explanation. Perhaps you would want to
modify it, but here it is:

Fred explained the system's operation as follows. The little bursts of heat
produced when the coal dust ignited, he said, were serving as reinforcers
for relay action. However, these reinforcers became effective only after the
house had cooled below a particular value; cooling the house thus served as
an "establishing operation" for reinforcer effectiveness. The system
generates responses, Fred stated, in the form of the pulses sent to the
currently selected injector. Those responses repeat as long as they
continue to be reinforced (by heat flashes) at least once every few
responses. However, if reinforcement is not delivered after a number of
responses, this behavior extinguishes and the system begins to vary its
actions (the stepper operates), selecting new responses (commanding
different injectors) until reinforcement once again occurs. As the room
heats up, the reinforcers lose their effectiveness (satiation) and
responding stops.

These are theoretical interpretations interspersed with observations of
what happens. The use of terms like "response," "reinforcement," and
"extinction" introduce Fred's theory about what is happening. Let's see if
we can separate the interpretation from the description.

What is observed is that activations of the igniter (not "responses")
generate heat flashes. We observe a physical mechanism by which the puffs
of coal dust and electric sparks cause a flash of heat. These flashes,
repeated, are seen to warm the house. When the temperature of the house
rises to some specific level, the activation of the igniter ceases. If the
igniter fails, when repeatedly activated, to cause a flash, nothing changes
immediately, but if the flash fails to occur for some time, causing the
room temperature to drop significantly, a new behavior occurs: new injector
nozzles (and ignitors, I guess) are rotated into place. This behavior
continues until the house temperature begins to rise again because a
working nozzle is in place. Then we revert to the previous behavior: a
(small) decrease in temperature results in activations of the igniter, and
an increase is followed by cessation of the repeated activations.

I believe this is an unbiased description of what is observed, favoring
neither reinforcement theory nor control theory. You may want to correct it
if I have misrepresented your intention, but I'll go on.

I agree that this is a description of what Fred would have observed just by
watching the system in action. But Fred did not arrive at his description
of the system's operation just by watching the system work. (Remember, I
said that Fred was something of a tinkerer himself.) This is extremely
important (see below).

Now Fred's embellishments of this bare description:

1. Something is "responding" to some unnamed stimulus by sending impulses
to the igniter. This unnamed stimulus is not itself observable; all we
observe is that there are impulses sent to the igniter, and that they cause
flashes of heat.

Fred did not like the term "response" because it implied a stimulus to
produce it. I should have used Fred's substitute term "operant." That term
defines behaviors in terms of their observable consequences in the
environment, and does not suggest a stimulus cause. (I'll substitute
"operant" for "response" in the remainder of this post.) The operant in
this analogy is sending an impulse to a particular injector.

2. The fact that the impulses continue to be emitted is interpreted as
meaning that the flashes of heat are "reinforcing" the responses to the
unnamed stimuli. This reinforcing action can't be observed; all we see is
that the igniter continues to operate.

Why do you assume that Fred was not allowed to tinker with the machine --
that he must simply observe the system during normal operation, without
being allowed to perform any tests that might help him to discover the
system's basic operating principles?

A careful experimenter, Fred tried covering the sensor in the firebox.
After a few more injection cycles, the system stopped firing the currently
selected injector, the stepper operated, and impulses were sent to the new
injector. After a few more impulses were delivered to the new injector, the
stepper operated again -- and this continued as long as the sensor was
covered. As soon as the sensor was uncovered and a flash occurred, this
reselection stopped, and the currently selected injector was repeatedly sent
impulses. Fred concluded that receiving the heat-flash stimulus at least
once following every few impulses is a necessary condition for maintaining
the current operant -- sending impulses to the presently selected injector.

3. When the nozzle quits working, the "responding" is no longer
"reinforced," so it "extinguishes." This does not imply, however, that
impulses cease to be sent to the igniter. If they were, changing nozzles
would do no good: there would never be another flash of heat even with a
working nozzle in place.

Let's not destroy the analogy by redefining what Fred called operants.
Operants are impulses to a particular injector, not just impulses period.
Sending impulses to a particular injector ceased. If you extinguish a
flame, it ceases to exist. If you stop sending impulses to an injector,
impulses to that injector cease to exist. The term "extinction" simply
denotes what is observed -- the frequency of the operant declines (in this
case to zero).

4. When reinforcement ceases," ... the system begins to vary its
actions (the stepper operates), selecting new responses (commanding
different injectors) until reinforcement once again occurs." Of course it
is not reinforcement that is observed to occur, but flashes of heat. Their
reinforcing effect is imagined.

Not so! The reinforcing effect is not imagined, it is observed. Fred
demonstrated that the flashes of heat had to be received by the sensor or
the current operant would cease to occur. When those flashes reappeared,
the currently selected operant continued to be repeated. This is a matter
of observation, not imagination.

5. Note that since the emission of impulses must continue if a flash of
heat is to occur, what is reinforced becomes uncertain. The only observable
result of reappearance of flashes of heat is that new nozzles cease to be
rotated into place. Emission of impulses must already have been occurring,
and it continues as long as the room temperature is low enough.

Again, impulses are not analogous to operants in this story. Impulses are
just activity that can operate on the environment in various ways: press
lever A, press lever B, touch nose to upper left front corner of chamber.

What is reinforced is whatever operant is followed by the flash. There is
nothing uncertain about it.

6. "As the room heats up, reinforcers lose their effectiveness (satiation)
and responding stops." Satiation is introduced as an explanation of why the
igniter ceases to be operated when the temperature rises to a certain
level. What is observed is not a loss of effectiveness of the reinforcers,
of course, but cessation of igniter operations.

Flashes following operants no longer keep the current operant selected.
Here my analogy breaks down a bit as a real organism is more complicated
than the furnace analogy would suggest. In the real organism, once
sufficient food had been taken in, the delivery of food pellets for
lever-pressing would no longer keep the rat returning to the lever, and
other activities with different consequences would take the place of
lever-pressing. One would observe that this particular consequence was no
longer keeping lever-press behavior going. One observes that this cessation
of the effectiveness of this consequence is related to the amount of food
the animal has consumed.

In the furnace analogy, the system only has a limited "repertoire"
consisting of sending impulses to one or another of several injectors, so it
has no choice but to continue selecting alternative routes to the same
environmental consequence (heat flashes), which is not the case with the
real organism.

I think it's clear that Fred's description of what is happening is not a
description, but a theory that invokes unobservable factors and influences
(as all theories do). A purely factual description does not require the use
of terms like reinforcement, response, and extinction: ordinary English
will do it (for an English-speaker).

If all you have been arguing about is whether there is such a thing as
theory-free observation, we could have saved everyone's time. No, there
isn't. But what happened to trying to understand how the terms of Fred's
explanation relate to the system's components and their operation? That was
supposed to be the topic under discussion. I'm not going to let you wriggle
out of it by diverting the debate to a different topic.

Come on, Bill, it isn't all that difficult. The model I've presented could
hardly be clearer in demonstrating how the consequences of certain behaviors
relate to the selection of those behaviors for repetition over others, and
the continued repetition of those behaviors (maintenance) so long as those
consequences continue to follow those behaviors and so long as those
consequences continue to be desired. And guess what, the process I've
described comes down to two control loops, one nested inside the other.

Bruce A.

16

[From Bill Powers (2000.11.25.1354 MST)]

Bruce Abbott (2000.11.25.1125 EST)--

I'm basing my comments on Fred's explanation. Perhaps you would want to
modify it, but here it is:

....

I agree that this is a description of what Fred would have observed just by
watching the system in action. But Fred did not arrive at his description
of the system's operation just by watching the system work. (Remember, I
said that Fred was something of a tinkerer himself.) This is extremely
important (see below).

Well, it can't be all _that_ important, since you failed to mention these
experiments. Perhaps it's time to rewrite the exposition, this time
including all the information that is important to the interpretation. I
find it hard to keep in mind the whole argument when it's presented in
fragments, so I'd appreciate your consolidating all the pieces into a
revised argument. I'll wait for that, then we can go on from there.

Best,

Bill P.

17

[From Bill Powers (2000.11.25.1400 MST)]

Bruce Abbott (2000.11.25.1125 EST)--

Afterthought: could you include, in your revised description, Fred's
observations of what happens to the house temperature when the igniter
ceases to fire properly?

Thanks,

Bill P.

18

[From Bruce Abbott (2000.11.25.1800 EST)]

Bill Powers (2000.11.25.1354 MST) --

Bruce Abbott (2000.11.25.1125 EST)

I'm basing my comments on Fred's explanation. Perhaps you would want to
modify it, but here it is:

....

I agree that this is a description of what Fred would have observed just by
watching the system in action. But Fred did not arrive at his description
of the system's operation just by watching the system work. (Remember, I
said that Fred was something of a tinkerer himself.) This is extremely
important (see below).

Well, it can't be all _that_ important, since you failed to mention these
experiments.

It is important, but I assumed that you were familiar with Fred's methods.

Perhaps it's time to rewrite the exposition, this time
including all the information that is important to the interpretation. I
find it hard to keep in mind the whole argument when it's presented in
fragments, so I'd appreciate your consolidating all the pieces into a
revised argument. I'll wait for that, then we can go on from there.

Yes sir. Perhaps it would be best if I began by identifying the various
elements:

Commands to different injectors represent different operants that the system
could perform. The term "operant" refers to the fact that the command
"operates" on the environment to produce some observable change -- in this
case, injector actuation. The rat's lever-press is an operant in the same
sense.

The stepper represents a mechanism that can be actuated to select a new operant.

The flash of heat in the furnace represents the reinforcing event. (I
define "reinforcement" below.)

The lowering of the house's interior temperature below a set-point
represents a condition necessary if the heat-flashes are to function as
reinforcers. When the temperature reaches the set-point the system "looses
interest" in producing them. That is, it fails to issue commands to produce
them, fails to check for heat flashes, fails to actuate the stepper. (This
is not a perfect analogy, but I don't want to make the example overly complex.)

"Reinforcement" in this context is keeping the currently selected injector
selected. Having experimented with the furnace, Fred knows that if the heat
flashes fail to reach the sensor in the firebox, then after several cycles
of injector operation the system will actuate the stepper and select a new
operant. (He has demonstrated this fact by experimentally covering the
sensor and observing how the system's behavior changes as a result.) So
what is maintaining the current operant (that is, keeping up the repeated
commanding of the same injector) is the at least occasional sensing of the
heat flash immediately after an injector command. Or to use Fred's terms,
the occasional reinforcement of the operant is what maintains the operant.

As Fred observed, when the operant fails to be reinforced, then after a few
cycles the stepper operates and selects a new operant (sends commands to a
new injector). The old operant ceases to occur, or to use Fred's term, it
has "extinguished." This is simply a name for the observed change in the
frequency of the operant that occurs when reinforcement of the operant ceases.

When the house-temperature rises to set-point, the system stops generating
operants and the flashes of heat are no longer produced. It remains in this
state until a fall in room temperature causes the system to begin generating
operants again.

Bill Powers (2000.11.25.1400 MST)]

Afterthought: could you include, in your revised description, Fred's
observations of what happens to the house temperature when the igniter
ceases to fire properly?

That depends of course on the rate of heat-loss from the house and the rate
of heat-production in the furnace. The latter will depend on how much heat
each flash produces and on the rate at which flashes continue while the
igniter "ceases to fire properly." If the problem is severe enough to bring
heat production below the rate of heat loss from the house, the house
temperature will fall.

O.K., your turn.

Bruce A.

19

[From Bill Powers (2000.11.26.01255 MST)]

Bruce Abbott (2000.11.25.1800 EST)--

Commands to different injectors represent different operants that the system
could perform. The term "operant" refers to the fact that the command
"operates" on the environment to produce some observable change -- in this
case, injector actuation. The rat's lever-press is an operant in the same
sense.

...

The stepper represents a mechanism that can be actuated to select a new

operant.

By "selecting a new operant" do you mean selecting a new kind of
environmental change to produce? As I understand it, a particular operant
would already include all possible means of producing a particular
environmental effect, so the only way to produce a _different_ operant
would be to cause a different environmental effect.

I seem to recall that an operant is not any particular act, but only the
class of all acts that can produce the same observable change in the
environment. Thus any manner of depressing a lever from using a paw-press
to sitting on the lever would be called the same operant, as long as the
lever goes down. Lighting a fire in a fireplace on a cold day and closing a
window would be the same operant in that they both can raise the
temperature of the room. And changing from one injector to another would
not change the operant, since they all are supposed to produce heat
flashes. Is this correct?

The flash of heat in the furnace represents the reinforcing event. (I
define "reinforcement" below.)

What is reinforced by this reinforcing event? Is it the emission of an
impulse to an igniter? I'll assume that in the following.

The lowering of the house's interior temperature below a set-point
represents a condition necessary if the heat-flashes are to function as
reinforcer.

How is the temperature lowered significantly below the set-point? Unless
the outside temperature falls far below the design limits, the only way for
this to happen is for the igniter to fail (this is why I asked about the
effect of igniter failure on house temperature). But in that case, there
would be _two_ effects of igniter failure, according to Fred. The first
would be cessation of reinforcement of emission of impulses, and the second
would be a drop in house temperature.

When the temperature reaches the set-point the system "loses interest" in
producing them. That is, it fails to issue commands to produce
them, fails to check for heat flashes, fails to actuate the stepper. (This
is not a perfect analogy, but I don't want to make the example overly

complex.)

"Reinforcement" in this context is keeping the currently selected injector
selected.

Wait a minute: I assumed it was the emission of impulses that was
reinforced. Was this wrong, then?

Having experimented with the furnace, Fred knows that if the heat
flashes fail to reach the sensor in the firebox, then after several cycles
of injector operation the system will actuate the stepper and select a new
operant.

What does the sensor in the firebox sense? Is it the heat flashes? Yes, I
see that that is what Fred originally said. It was said that when the
sensor of heat flashes failed to detect fire, it triggered the relay that
injected another puff of coal dust and ignited it. But now you're saying
that failure of the sensor to detect a heat flash for some time will cause
the stepper to operate and select another injector. What determines which
effect will occur,and what keeps both effects from occurring at the same time?

You say "If the heat flashes fail to reach the sensor in the firebox ...",
which seems to imply that the heat flashes are occurring, but fail to reach
the sensor, which is different from saying that the sensor fails to detect
heat flashes because they are not occurring.

Now I'm getting thoroughly confused. I thought a heat flash was a
reinforcer that kept the system emitting impulses to the igniter. But it
also seems to be said that the _lack_ of a heat flash is a stimulus that
triggers the relay that causes impulses to be sent to the igniter,
"machine-gun fashion." The actual generation of impulses is simply a
response to the (negative) stimulus of cessation of the fire, according to
one statement. But you've denied that this "operant" is a response to a
stimulus, in another statement.

(He has demonstrated this fact by experimentally covering the
sensor and observing how the system's behavior changes as a result.)

As Fred observed, when the operant fails to be reinforced, then after a few
cycles the stepper operates and selects a new operant (sends commands to a
new injector). The old operant ceases to occur, or to use Fred's term, it
has "extinguished." This is simply a name for the observed change in the
frequency of the operant that occurs when reinforcement of the operant

ceases.

When the house-temperature rises to set-point, the system stops generating
operants and the flashes of heat are no longer produced. It remains in this
state until a fall in room temperature causes the system to begin generating
operants again.

But in fact, when the commands are issued the heat-flashes do occur, even
though the sensor can't detect them. The house should warm up as usual, and
this should shut down the issuing of commands and therefore the heat
flashes. So the temperature control loop would continue to operate as
before. Covering the sensor shows that the sensor loop is unnecessary for
temperature control. All that's neccessary is for a drop in temperature to
produce repeated heat flashes. An abnormally large temperature error could
result in actuating the stepper motor.

Clearly, this system needs to be simulated. There are apparent
contradictions and unmentioned effects that will probably occur as it is
currently described. I know that your explanation seems perfectly clear to
you, but it has only confused me. I don't know what the reinforcer
reinforces, or whether the sensor produces stimuli that trigger responses,
or produces reinforcements, or both. The role of the enabling condition is
ambiguous, or so it seems to me right now. It's not at all clear what Fred
expects to observe when he covers the sensor but allows commands to go on
producing heat flashes.

A simulation is really the only way to see the actual implications of the
descriptions you have given. How about it?

Best,

Bill P.

20

[From Bruce Abbott (2000.11.26.1015 EST)]

Bill Powers (2000.11.26.01255 MST) --

Clearly, this system needs to be simulated. There are apparent
contradictions and unmentioned effects that will probably occur as it is
currently described. I know that your explanation seems perfectly clear to
you, but it has only confused me. I don't know what the reinforcer
reinforces, or whether the sensor produces stimuli that trigger responses,
or produces reinforcements, or both. The role of the enabling condition is
ambiguous, or so it seems to me right now. It's not at all clear what Fred
expects to observe when he covers the sensor but allows commands to go on
producing heat flashes.

I started to write a detailed reply to your questions but found that I was
going to have do a lot of additional explaining to answer an excellent
question you raise, which is how we define an operant. I'm sorry I wasn't
able to write more clearly; evidently it can be as difficult to follow this
sort of analysis as it is to properly understand how a simple control system
really works. In both cases, once you do understand it, it seems perfectly
clear and you wonder how you could have been so confused about it earlier.

A simulation is really the only way to see the actual implications of the
descriptions you have given. How about it?

Yes, that does seem to be the way to go. I hate to leave things hanging
like this, but I won't be able to get to it right away as I'm now facing
having to get other things done that I can't put off any longer. I had not
anticipated spending as much time on this as I have already.

Best wishes,

Bruce A.