What's a reinforcer?

[Bruce_Abbott]

[From Bruce Abbott (970923.2015 EST)]

Bill Powers (970923.1043 MDT) --

Bruce Abbott (970922.2000 EST)

A controlled perception is a variable, like the position of a cursor. It
would not be correct to call the cursor itself the controlled variable if
what is being controlled about the cursor is its position relative to a
target. The cursor is only an object, not a variable. By the same logic,
it would not be correct to call a food pellet -- an object -- the
controlled variable when what is being controlled about the pellet is its
rate of presentation, or its current state of availability (available/not
available). To do so only invites confusion.

The confusion is in the idea that a food pellet is not a variable. All
perceptions are variables in particular states.

A particular state of a variable is not a variable.

What makes the food pellet
what it is is the set of states of all variable attributes associated with
it: size, color, odor, position, shape, weight, and so forth. When you have
specified all the variable attibutes and their values, you have defined the
food pellet as it is perceived. Some of these attributes can be changed by
behavior; some, perceptually, interact (the distance of the food pellet
from the rat affects the intensity of its odor, for example).

Yes, and it is perceptually different if held in the paw, or placed in the
mouth and tasted. None of which has any relevance to the question of
whether the pellet per se, being delivered as a consequence of the action of
a control system, is a controlled variable, or simply a constant being added
to or otherwise affecting the value of the CV.

Perhaps what you have in mind is that the reinforcer is some particular
state of the controlled variable, e.g., cursor on-target or pellet
present. That is, the reinforcer is "what is wanted." But a particular
state of a controlled variable is not the same as a controlled variable,
so in this definition too, a reinforcer (a particular state of a CV)
cannot be a controlled variable. Dr. Marken loses either way.

What Rick has in mind, and what I have in mind, is that there is nothing
reinforcing about the food pellet.

That's O.K. by me, but again irrelevant to the issue. We are asking what is
this thing EABers call a reinforcer, not whether it actually "reinforces"
anything. In particular, we are asking what its role is within a
functioning control system.

The so-called reinforcer is JUST some
attribute of the food pellet, such as its perceived position or taste or
smell, that is under control by the organism.

So there is some attribute of the food pellet (such as its perceived
position) that the rat is controlling by pressing the lever. I agree. But
wait: If the rat's current reference for this attribute is in-the-food-cup,
and the current state of this attribute is in-the-feeder, then the rat will
press the lever in order to bring this attribute of the food pellet to a
perceptual state matching the reference. It presses the lever rather than
doing something else (or nothing at all) because it has learned that this
action will move the pellet into the cup. If the pellet is already in the
food cup, no action is required and we will not observe a lever-press. So
what is motivating ("reinforcing") the lever-press is not the attribute
(where the pellet is), but rather the desire on the part of the rat to
change the pellet's attribute from in-the-feeder to in-the-cup. The rat
presses the lever in order to bring the CV to its reference state, or as
close to that state as one lever-press will produce. In other words, it
presses the lever to reduce (to zero if possible) the currently existing
error between the CV and its reference state.

Thus, a reinforcer must be that which reduces error between what I have
and what I want, or in other words, between the current state of a CV and
its reference value.

This is not a general definition. If anything that reduces the difference
between a CV and its reference value is a reinforcer, then a change in the
reference value toward the perceived value of the CV is a reinforcer. Under
this definition, it makes no difference what reduces the error; there is
nothing in this definition to say that noncontingent food pellets would not
be reinforcing -- they would be just as effective in reducing error as
pellets produced by behavior. Obviously, the reduction in error per se
can't be the only criterion for defining a reinforcer.

Good point, but see my reply to Bruce Gregory on this issue [Bruce Abbott
(970923.1850 EST)]. There must be a particular relationship established
between action, reinforcer, and reduction of error if the reinforcer is to
function as such.

The food pellets consumed change from being merely food pellets to being
reinforcers only when they are produced as a consequence of behavior.

Yes.

Their
error-reducing effects are not what make the difference. If we want to find
the difference, we have to try to see what is different about the situation
when the food pellets are and are not produced by behavior. That is the
only difference that makes any difference.

Incorrect. Both are necessary: the action of the control system must
produce the reinforcer, and the effect of the reinforcer must be to reduce
the error between the CV and its reference state.

If you're trying to define a reinforcer in terms of PCT, you have to find
some unique role that the reinforcer plays that fits your definition under
all circumstances. It is necessary, but not sufficient, that the reinforcer
reduce error. It is necessary, but not sufficient, that the reinforcer be
produced by behavior.

Now you seem to be agreeing with me. In the previous paragraph you said
that the only difference that makes any difference is whether or not the
behavior produces the pellets. You can't have it both ways.

So is it necessary AND sufficient that the reinforcer
be produced by behavior AND reduce error? No, not yet, because there are
other conditions that have to be satisfied.

When you're speaking of Phase 1, food that appears because of behavior will
be followed by more of the behavior that produced it. We will observe a
concurrent increase in the number of behaviors of the required kind and the
number of deliveries of food pellets. So the necessary and sufficient
conditions are met and reinforcement does take place.

Yes, I agree.

During phases 2 and 3, however, we find that while food produced by
behavior does reduce error, any change in the contingency that results in
_more_ food being produced leads to _less_ behavior, and vice versa. So now
the purported reinforcing effect of the food has reversed. Where in Phase 1
an increase in food production went with an increase in behavior, now an
increase in food production goes with a decrease in behavior.

I disagree, because the comparison of rate of responding across schedules,
pellet sizes, etc. is not the appropriate comparison to be making.
Responding continues under _each_ of these new contingencies (rather than
extinguishing) so long as the putative reinforcer continues to be produced
by the operant and continues to reduce the error in the relevant control system.

Thus we have to conclude that food is reinforcing only under phase 1.

However, there are other explanations, such as that when food appears
during phase 1 it does not increase the freqauency with which more of the
same behavior will occur; it simply stops the continuing search for food in
different places.

Certainly. But in either case, the operant will be observed to occur more
often (up to some limit) and other behaviors will be observed to occur less
often (as more time is taken up performing the operant. Even so, I would
expect that the changes expected under these two different scenarios can be
distinguished and thus provide an experimental test betweent them.

At the start of your post, you disputed my claim that reinforcers are not
controlled variables, but rather serve to reduce error between CV and
reference when made contingent on the system's actions. At the end you seem
to be agreeing with my position. Did I misunderstand something, or did you
change your mind?

Regards,

Bruce

[Bill_Powers1]

[From Bill Powers (970924.0920 MDT)]

Bruce Abbott (970923.2015 EST)--

The confusion is in the idea that a food pellet is not a variable. All
perceptions are variables in particular states.

A particular state of a variable is not a variable.

It's a value of the variable. This is getting pretty silly.

To say that something is a variable isn't to say that it is necessarily
varying. The sides of a rectangle are the variables in the equation for
area, a = x*y, but this doesn't imply that x and y are changing -- only
that they could have different values in different situations.

You seem to be taking an almost deliberately non-physical, non-mathematical
approach here.

We are asking what is
this thing EABers call a reinforcer, not whether it actually "reinforces"
anything. In particular, we are asking what its role is within a
functioning control system.

I think I understand what you're trying to do here by identifying the food
pellets with reinforcers but not with a CV. Basically you're trying to say
that the food pellet is the visible correlate of the real CV, which for the
control system you're thinking of is nutrient level. When you say that the
food pellet isn't the controlled variable, you're saying it's not the
controlled variable you have in mind. By setting it up this way, you can
say that EABers have simply identified one variable in the loop, something
about the delivery of food pellets, as the reinforcing factor, and that it
is the error in the nutrient-level control system that produces the bar
pressing.

The problem is that something about the delivery of food pellets is
probably ALSO a controlled variable. There is evidence to show that
directly interfering with the delivery of food pellets will produce an
immediate change in behavior, before the nutrient-level control system
could react. The appearance is that the nutrient-level control system does
NOT produce behavior itself, but acts by raising and lowering the reference
level for the pellet-intake control system. It is only the pellet-intake
control system that acts through behavioral systems to make pellets appear.
I've made this point before, and you need to take it seriously. In a
hierarchical control system, the higher levels do NOT produce motor
behavior. They act ONLY by adjusting the reference signals for lower
systems, and these reference signals specify inputs, not outputs. The lower
systems alone have access to the behavioral outputs.

So there is some attribute of the food pellet (such as its perceived
position) that the rat is controlling by pressing the lever. I agree. But
wait: If the rat's current reference for this attribute is in-the-food-cup,
and the current state of this attribute is in-the-feeder, then the rat will
press the lever in order to bring this attribute of the food pellet to a
perceptual state matching the reference. It presses the lever rather than
doing something else (or nothing at all) because it has learned that this
action will move the pellet into the cup.

Not quite. In more neutral language, it begins to press the lever more
frequently, resulting in more frequent appearance of food in the cup. We
_call_ this change of behavior "learning." In fact we don't know what
brings the rat to press more frequently, and thus to produce more food. All
we know is that when it is possible for more behavior to produce more food,
more food will be produced by an increase in behavior.

The attribute is not "in-the-feeder." It's "food present." If food is not
present, press the lever until it is present. Where it comes from is
unknown to the rat, and irrelevant. I doubt that a rat can see that it
needs to get that food hidden up there in the feeder down here into the dish.

If the pellet is already in the
food cup, no action is required and we will not observe a lever-press. So
what is motivating ("reinforcing") the lever-press is not the attribute
(where the pellet is), but rather the desire on the part of the rat to
change the pellet's attribute from in-the-feeder to in-the-cup. The rat
presses the lever in order to bring the CV to its reference state, or as
close to that state as one lever-press will produce. In other words, it
presses the lever to reduce (to zero if possible) the currently existing
error between the CV and its reference state.

Fine. Then the presence of the pellet in the cup is a controlled variable.
The animal does not press for another pellet until the current pellet has
been removed from the cup (by any agency, including another control system
in the rat). Whenever there is no food in the cup, the rat behaves so as to
bring food back into the cup. It is THIS system that has to be reorganized
to produce the effective lever-pressing actions.

The food pellets consumed change from being merely food pellets to being
reinforcers only when they are produced as a consequence of behavior.

Yes.

Their
error-reducing effects are not what make the difference. If we want to find
the difference, we have to try to see what is different about the situation
when the food pellets are and are not produced by behavior. That is the
only difference that makes any difference.

Incorrect. Both are necessary: the action of the control system must
produce the reinforcer, and the effect of the reinforcer must be to reduce
the error between the CV and its reference state.

But the action of the "reinforcer" (the food pellet) is _always_ to reduce
the error, whether it is produced by the behavior or not. So the special
property of reinforcingness is a matter of a change in the ability of the
food pellet to reduce error.

Whatever produces the food pellet is not a property of the food pellet, but
of something else -- the apparatus. Neither is the fact that behavior
triggers the apparatus a property of the food pellet. In your terms, the
ONLY property the food pellet has is to reduce error, and as should be the
case, it always has that property, whatever brings the food pellet to the
dish. Nothing about the effects of the food pellet changes when the cause
of its appearance changes. So the mere fact that behavior caused the food
to appear can't possibly account for its purported reinforcing properties.

If you're trying to define a reinforcer in terms of PCT, you have to find
some unique role that the reinforcer plays that fits your definition under
all circumstances. It is necessary, but not sufficient, that the reinforcer
reduce error. It is necessary, but not sufficient, that the reinforcer be
produced by behavior.

Now you seem to be agreeing with me. In the previous paragraph you said
that the only difference that makes any difference is whether or not the
behavior produces the pellets. You can't have it both ways.

I'm saying that the _concept of reinforcement as held in EAB_ requires that
the source of the food pellets makes all the difference. But changing the
source has no effect at all on how the food pellets affect the organism;
their actual properties are not influenced at all. They always reduce the
error, so that condition is always satisfied. There is no physical change
in the food pellets, and hence no change in their physical effects on the
organism. The only way it can be argued that changing the source of the
food pellets changes their effects on the organism is to propose that they
have a NON-PHYSICAL effect on the organism that DOES change according to
their source.

When you're speaking of Phase 1, food that appears because of behavior will
be followed by more of the behavior that produced it. We will observe a
concurrent increase in the number of behaviors of the required kind and the
number of deliveries of food pellets. So the necessary and sufficient
conditions are met and reinforcement does take place.

Yes, I agree.

During phases 2 and 3, however, we find that while food produced by
behavior does reduce error, any change in the contingency that results in
_more_ food being produced leads to _less_ behavior, and vice versa. So now
the purported reinforcing effect of the food has reversed. Where in Phase 1
an increase in food production went with an increase in behavior, now an
increase in food production goes with a decrease in behavior.

I disagree, because the comparison of rate of responding across schedules,
pellet sizes, etc. is not the appropriate comparison to be making.
Responding continues under _each_ of these new contingencies (rather than
extinguishing) so long as the putative reinforcer continues to be produced
by the operant and continues to reduce the error in the relevant control

system.

When you say it's not the "appropriate" comparison to be making, all you
mean is that this comparison doesn't lead to the conclusion you want. But
it's a perfectly appropriate comparison: it's there to be made, and
therefore it _must_ be made if you're not trying to force the analysis to a
foregone conclusion. You can't just ignore the fact that an increase in
reinforcement goes with a decrease in behavior. Changing the schedule is a
legitimate way to test the relationship between reinforcement and behavior,
and in fact the only way to change it. The fact that this test contradicts
the statement that increasing reinforcement goes with increasing behavior
is indisputable. You can't ignore it just because it creates a
contradiction. To ignore it is to admit that you want to select only the
observations that support a predetermined conclusion.

At the start of your post, you disputed my claim that reinforcers are not
controlled variables, but rather serve to reduce error between CV and
reference when made contingent on the system's actions. At the end you seem
to be agreeing with my position. Did I misunderstand something, or did you
change your mind?

At the end I was taking your point of view, even though I don't accept it.
According to your point of view, I conclude that reinforcers are
reinforcing only during phase 1, because it is only then that we see an
increase in behvior-dependent reinforcement apparently resulting in an
increase in behavior. If we recall that reinforcement depends on behavior,
that is a trivial observation, because the apparatus sees to it that an
increase in behavior _must_ produce an increase in reinforcement. The only
way to see any other relation is to alter the schedule so the same behavior
produces more or less reinforcement, and that can be done only in phases 2
and 3. When that is done we see the opposite relationship: less
reinforcement leads to more behavior. In order to see the phase-1 relation,
we have to go all the way to extinction: no reinforcement, and ultimately
no behavior (or only the background rate). But that is not what happens
during phases 2 and 3.

Best,

Bill P.

[Bruce_Abbott]

[From Bruce Abbott (970925.1030 EST)]

Bill Powers (970924.0920 MDT) --

Bruce Abbott (970923.2015 EST)

The confusion is in the idea that a food pellet is not a variable. All
perceptions are variables in particular states.

A particular state of a variable is not a variable.

It's a value of the variable. This is getting pretty silly.

My point was that a thing like a food pellet is not a variable, nor are
particular fixed attributes of that thing, such as its size, weight, and
composition. If it is not a variable, then it cannot be a controlled
variable. Thus, when you say that a food pellet is a variable, you are
either flat out wrong or being fast and free with your word usage. You do
not mean that the food pellet is a controlled variable (or indeed a variable
of any sort); you mean that some aspect of the rat's perception of the
pellet is a controlled variable.

Your claim is that a reinforcer is a controlled variable. If the reinforcer
is the pellet, and the pellet itself is not a controlled variable, then a
reinforcer is not a controlled variable. My claim is that the reinforcer is
some entity having attributes that allow it to affect the value of the
controlled variable in a particular way. This claim does not make any
strange assertions that a food pellet is a variable, controlled or
otherwise; it asserts only that the pellet can in some way affect the value
of the controlled variable. Example: a quarter is not a variable. Amount
of money on hand is a variable. Adding a quarter to my collection of
quarters, dollars, etc. increases the amount of money I have on hand.
Contributing a quarter affects the value of the variable "money on hand" but
the quarter itself is a fixed amount of money, not a variable amount.

You seem to be taking an almost deliberately non-physical, non-mathematical
approach here.

I am? If D is the amount of money I have in dollars and Q is a quarter, and
you contribute a quarter, then the amount of money I have is now

  Dnew = Dold + Q.

D is here varying; Q is constant. Now what is nonphysical and
nonmathematical about this?

We are asking what is
this thing EABers call a reinforcer, not whether it actually "reinforces"
anything. In particular, we are asking what its role is within a
functioning control system.

I think I understand what you're trying to do here by identifying the food
pellets with reinforcers but not with a CV. Basically you're trying to say
that the food pellet is the visible correlate of the real CV, which for the
control system you're thinking of is nutrient level. When you say that the
food pellet isn't the controlled variable, you're saying it's not the
controlled variable you have in mind. By setting it up this way, you can
say that EABers have simply identified one variable in the loop, something
about the delivery of food pellets, as the reinforcing factor, and that it
is the error in the nutrient-level control system that produces the bar
pressing.

No, I am thinking about how food pellets affect the CV, whether the CV is
"availability of food" or "nutrient level," or several other controlled
variables in between. True, I have error in the nutrient-level control
system in mind as the ultimate reason for the lever press, but we are
dealing here with a chain involving a lever press, delivery of the pellet
into the cup, approach to the cup, retrieval of the pellet into the mouth,
chewing, swallowing, and digesting, the latter finally supplying additional
nutrients to the bloodstream and tissues and thereby reducing the error
between nutrient level and its reference. Each of the behaviors in the
chain serves to control some variable related to the perception of pellet,
and each controlled variable must be brought to a particular state before
the next control system can effectively act. The pellet cannot be digested
until it is in the stomach. The pellet cannot be in the stomach until it
has been swallowed. It cannot be swallowed until it has been chewed. It
cannot be chewed until it is in the mouth. It cannot be in the mouth until
it is available in the food cup. It cannot be available in the food cup
until the lever has been pressed. The whole series is ultimately held
together by the effect of the nutrient content of the pellet (essentially
constant) on nutrient level (in this simplified model; I recognize that the
system is far more complex than this). Each particular behavioral act in
the sequence would be selected for its ability to bring about a change in
the state of a perception related to the pellet from its current state to a
goal state. The action does this through its consequence on the
environment, which affects the state of the relevant controlled variable via
the environment function.

It is only the pellet-intake
control system that acts through behavioral systems to make pellets appear.
I've made this point before, and you need to take it seriously. In a
hierarchical control system, the higher levels do NOT produce motor
behavior. They act ONLY by adjusting the reference signals for lower
systems, and these reference signals specify inputs, not outputs. The lower
systems alone have access to the behavioral outputs.

I am fully aware of these facts and do take them seriously; they are the
model on which my analysis is based. On what ground do you assert that I
ignore them?

This is getting lengthly. I'll stop here and find out where we are from
your perspective.

Regards,

Bruce

[Bruce_Gregory9]

[From Bruce Gregory (970925.1210 EDT)]

Bruce Abbott (970925.1030 EST)

My point was that a thing like a food pellet is not a variable, nor are
particular fixed attributes of that thing, such as its size, weight, and
composition. If it is not a variable, then it cannot be a controlled
variable. Thus, when you say that a food pellet is a variable, you are
either flat out wrong or being fast and free with your word usage. You do
not mean that the food pellet is a controlled variable (or indeed a variable
of any sort); you mean that some aspect of the rat's perception of the
pellet is a controlled variable.

I'm in your camp on this one. When it comes to the utility of
the notion of a reinforcer, I'm on the other side.

Sancho

[Bruce_Abbott]

[From Bruce Abbott (970925.1200 EST)]

Bruce Gregory (970925.1210 EDT) --

Bruce Abbott (970925.1030 EST)

My point was that a thing like a food pellet is not a variable, nor are
particular fixed attributes of that thing, such as its size, weight, and
composition. If it is not a variable, then it cannot be a controlled
variable. Thus, when you say that a food pellet is a variable, you are
either flat out wrong or being fast and free with your word usage. You do
not mean that the food pellet is a controlled variable (or indeed a variable
of any sort); you mean that some aspect of the rat's perception of the
pellet is a controlled variable.

I'm in your camp on this one. When it comes to the utility of
the notion of a reinforcer, I'm on the other side.

Sancho:

Nice to have _somebody_ agree with me (about reinforcers) but I'm
_astonished_ you think that the notion of a reinforcer has some utility!

Regards,

Bruce

[Bill_Powers1]

[From Bill Powers (970925.1144 MDT)]

Bruce Abbott (970925.1030 EST)--

My point was that a thing like a food pellet is not a variable, nor are
particular fixed attributes of that thing, such as its size, weight, and
composition. If it is not a variable, then it cannot be a controlled
variable.

That much is true: we can't control the pellet-ness of the pellet. But we
can control _something about_ the pellet, like position or number or
frequency of occurrance. We can control anything about the pellet that our
actions can influence, and that has to be a variable. If it were not a
variable, we could not influence it. I don't want to quibble over this;
it's obvious to me that a rat could control the rate of delivery of food
pellets, and if that's not obvious to you there's nothing I can do about
it. Let's skip to the main issue.

My claim is that the reinforcer is
some entity having attributes that allow it to affect the value of the
controlled variable in a particular way. This claim does not make any
strange assertions that a food pellet is a variable, controlled or
otherwise; it asserts only that the pellet can in some way affect the value
of the controlled variable.

I'm sorry, but this is NOT your claim. Your claim is that the attributes of
the reinforcer affect the value of the controlled variable in a particular
way _only if the reinforcer is produced by behavior_, and not otherwise.
Your attempt to find a physical explanation of reinforcement falls apart at
this point. If this were a _physical_ explanation, it would make no
difference what caused the food pellet to appear. The only attribute of the
food pellet that has an effect on the CV you define is its nutrient
content, and that is not changed when the source of the food pellets
changes. Yet the reinforcing value of the food pellet _does_ change.

Your claim is not just that the reinforcer reduces error in a control
system. That much we can account for by the nutrient content of the
pellets. Your claim is that the reinforcer has an added, nonphysical,
property that can cause behavior to _increase_, and that this property
appears only when the food pellet is produced by behavior. We know that
this is NOT the effect of the nutrients in the food pellet; their effect is
always to _reduce_ behavior if there is a control system. Even a single
pellet will reduce the error and thus the behavior, if only by a small
amount, and it will do so no matter where the food pellet came from. It
will never increase the behavior. So the food pellet must have some special
reinforcing property _in addition_ to its nutrient value, a special
property that can cause the rate of behavior to increase rather than
decrease, and only if the food pellet is caused to appear by the behavior.
Being produced by behavior alters the reinforcing value of the food pellet.

I don't see how you can get around this.

Best,

Bill P.

[Bruce_Abbott]

[From Bruce Abbott (970925.2205 EST)]

Bill Powers (970925.1144 MDT) --

Bruce Abbott (970925.1030 EST)

My point was that a thing like a food pellet is not a variable, nor are
particular fixed attributes of that thing, such as its size, weight, and
composition. If it is not a variable, then it cannot be a controlled
variable.

That much is true: we can't control the pellet-ness of the pellet. But we
can control _something about_ the pellet, like position or number or
frequency of occurrance. We can control anything about the pellet that our
actions can influence, and that has to be a variable. If it were not a
variable, we could not influence it. I don't want to quibble over this;
it's obvious to me that a rat could control the rate of delivery of food
pellets, and if that's not obvious to you there's nothing I can do about
it. Let's skip to the main issue.

That much is obvious to me, too. It is also obvious that rate of food
delivery is not a food pellet; so if rate of food delivery is the controlled
variable, then the pellet is not the CV as you claimed it is.

My claim is that the reinforcer is
some entity having attributes that allow it to affect the value of the
controlled variable in a particular way. This claim does not make any
strange assertions that a food pellet is a variable, controlled or
otherwise; it asserts only that the pellet can in some way affect the value
of the controlled variable.

I'm sorry, but this is NOT your claim. Your claim is that the attributes of
the reinforcer affect the value of the controlled variable in a particular
way _only if the reinforcer is produced by behavior_, and not otherwise.

Well I'm sorry, too, because if you think that this is my claim, then you
have misunderstood my claim. Certain attributes of the reinforcer can
affect the value of the controlled variable regardless of how they are
applied to it. Placing the pellet directly into the rat's stomach will have
the same effect on the rat's nutrient level as allowing the rat to pick up
the pellet and eat it. Dropping the pellet into the food cup will have the
same effect on the availability of the pellet for the rat's consumption as
delivering it to the food cup contingent on the rat's lever-press. I have
not claimed otherwise.

Your attempt to find a physical explanation of reinforcement falls apart at
this point.

How so?

If this were a _physical_ explanation, it would make no
difference what caused the food pellet to appear.

It would make no difference to _what_?

The only attribute of the
food pellet that has an effect on the CV you define is its nutrient
content, and that is not changed when the source of the food pellets
changes. Yet the reinforcing value of the food pellet _does_ change.

No, the reinforcing value of the pellet does not change. The rat becomes
more likely to reproduce those elements of its behavior that consistently
precede the delivery of the pellet. We do not have enough information as
yet to specify the physiological mechanism through which this change occurs,
but it is not hard to imagine a simple physical mechanism with the required
functional properties. One proposal I have read (somewhere) had it that the
mechanism varied the output connections of the active control system at a
rate proportional to error in systems involving certain "essential"
variables; by reducing error in these systems the reinforcer would prevent
the currently established connection from being "reorganized" away. You
will probably say that this is a rather far-fetched mechanism, but it is a
mechanism that could conceivably work, thus demonstrating that no
"nonphysical" agencies are needed to account for the effect.

When the pellet is not made contingent on the lever-press, the rat may be
doing just about anything when the pellet arrives; as this is unlikely to be
lever pressing, that particular behavior will not constitute the selected
action that remains selected. [Note: there are no doubt other possible
physical mechanisms that would produce a similar result; I use this one only
for illustration.]

Your claim is not just that the reinforcer reduces error in a control
system. That much we can account for by the nutrient content of the
pellets. Your claim is that the reinforcer has an added, nonphysical,
property that can cause behavior to _increase_, and that this property
appears only when the food pellet is produced by behavior.

No, I have not made any such claim. Where have I said anything about a
nonphysical property?

We know that
this is NOT the effect of the nutrients in the food pellet; their effect is
always to _reduce_ behavior if there is a control system. Even a single
pellet will reduce the error and thus the behavior, if only by a small
amount, and it will do so no matter where the food pellet came from. It
will never increase the behavior. So the food pellet must have some special
reinforcing property _in addition_ to its nutrient value, a special
property that can cause the rate of behavior to increase rather than
decrease, and only if the food pellet is caused to appear by the behavior.
Being produced by behavior alters the reinforcing value of the food pellet.

You are imagining some completely different view from any that I am
entertaining, Bill.

I don't see how you can get around this.

I don't see that I have to -- it isn't my view.

Regards,

Bruce

[Bill_Powers1]

[From Bill Powers (970926.0552 MDT)]

Bruce Abbott (970925.2205 EST)--

You are imagining some completely different view from any that I am
entertaining, Bill.

I don't see how you can get around this.

I don't see that I have to -- it isn't my view.

I'm having difficulty again distinguishing your view from the EAB view to
which you propose to build a bridge. Since you agree with me that the only
effect of the food pellet is to increase the CV, which I stipulate is the
nutrient level for the sake of the argument, and since increasing the CV
toward its reference level must cause behavior to decrease, it seems to me
that the only phenomenon explained by this model, so far, is a decrease in
behavior apparently caused by an increase of food intake.

But is that the phenomenon that is called "reinforcement" in EAB?

Best,

Bill P.

[Bruce_Abbott]

[From Bruce Abbott (970926.0840 EST)]

Bill Powers (970926.0552 MDT)

Bruce Abbott (970925.2205 EST)

You are imagining some completely different view from any that I am
entertaining, Bill.

I don't see how you can get around this.

I don't see that I have to -- it isn't my view.

I'm having difficulty again distinguishing your view from the EAB view to
which you propose to build a bridge. Since you agree with me that the only
effect of the food pellet is to increase the CV, which I stipulate is the
nutrient level for the sake of the argument, and since increasing the CV
toward its reference level must cause behavior to decrease, it seems to me
that the only phenomenon explained by this model, so far, is a decrease in
behavior apparently caused by an increase of food intake.

When you say that "the only effect of the food pellet is to increase the
CV," you are talking _direct effect_, right? What about _indirect_ effects?
Do any changes take place in the functional organization of the rat's
nervous system as a result, directly or indirectly, of the increased CV that
is the direct effect of the food pellet? Are any changes _prevented_ that
would otherwise have occurred?

But is that the phenomenon that is called "reinforcement" in EAB?

No. That is the phenomenon that is called "satiation" in EAB. The use of
small pellets and the low delivery rates seen on most schedules tend to
miminize the magnitude of this phenomenon over the course of a typical session.

Regards,

Bruce

[Bill_Powers1]

[From Bill Powers (970926.0-805 MDT)]

Bruce Abbott (970926.0840 EST)--

When you say that "the only effect of the food pellet is to increase the
CV," you are talking _direct effect_, right? What about _indirect_ effects?
Do any changes take place in the functional organization of the rat's
nervous system as a result, directly or indirectly, of the increased CV that
is the direct effect of the food pellet? Are any changes _prevented_ that
would otherwise have occurred?

I don't know. If you're proposing that these effects occur, then you have
to add something to your control system model that's not in it now, because
all that model can do right now is to decrease the behavior when the CV
increases.

But is that the phenomenon that is called "reinforcement" in EAB?

No. That is the phenomenon that is called "satiation" in EAB. The use of
small pellets and the low delivery rates seen on most schedules tend to
miminize the magnitude of this phenomenon over the course of a typical

session.

Do you mean that "satiation" occurs upon ingestion of the very first food
pellet? That is what happens in the control model you're proposing. The
greatest amount of behavior occurs when no food at all has been delivered.
Every pellet of food after that decreases the amount of behavior. You can't
give a small enough amount of food to reverse this relationship. That is
how the control model you're using behaves.

If that's not what happens during the phenomenon of reinforcement, you're
not using the right model.

Best,

Bill P.

[Bruce_Abbott]

[From Bruce Abbott (970926.1040 EST)]

Bill Powers (970926.0805 MDT) --

Bruce Abbott (970926.0840 EST)

When you say that "the only effect of the food pellet is to increase the
CV," you are talking _direct effect_, right? What about _indirect_ effects?
Do any changes take place in the functional organization of the rat's
nervous system as a result, directly or indirectly, of the increased CV that
is the direct effect of the food pellet? Are any changes _prevented_ that
would otherwise have occurred?

I don't know. If you're proposing that these effects occur, then you have
to add something to your control system model that's not in it now, because
all that model can do right now is to decrease the behavior when the CV
increases.

What about in _your_ model? Does the pellet have any _indirect_ effects on
the organization of the system in question?

But is that the phenomenon that is called "reinforcement" in EAB?

No. That is the phenomenon that is called "satiation" in EAB. The use of
small pellets and the low delivery rates seen on most schedules tend to
miminize the magnitude of this phenomenon over the course of a typical

session.

Do you mean that "satiation" occurs upon ingestion of the very first food
pellet? That is what happens in the control model you're proposing. The
greatest amount of behavior occurs when no food at all has been delivered.
Every pellet of food after that decreases the amount of behavior. You can't
give a small enough amount of food to reverse this relationship. That is
how the control model you're using behaves.

The term can refer to the end-point of the process (reduction of motivation
to zero), but satiation as a variable admits of degrees. If we think of
satiation in terms of reduction of error between the CV and its reference,
then each pellet tends to contribute to satiation. However, metabolic
output has the opposite effect, and the level of satiation will be
determined by the contribution of these two rates to nutrient level. If
pellet intake rate is small enough and metabolic rate is high enough, we may
not observe any satiation, or even an increase in the level of deprivation.
(Satiation and deprivation are just opposite sides of the same coin.)

If that's not what happens during the phenomenon of reinforcement, you're
not using the right model.

I am not attempting to present anything other than a control model. I am
attempting to relate the operation of this control model to the observations
which have been described previously in the language of reinforcement. But
for some reason, you apparently believe that I am attempting to offer an
alternative, reinforcement-based mechanism as a substitute for a control
mechanism.

Now, if a control model is not the right model, what do you propose as an
alternative?

Regards,

Bruce

[Bill_Powers1]

[From Bill Powers (970926.109 MDT)]

Bruce Abbott (970926.1040 EST)--

Do any changes take place in the functional organization of the rat's
nervous system as a result, directly or indirectly, of the increased CV

that

is the direct effect of the food pellet? Are any changes _prevented_ that
would otherwise have occurred?

I don't know. If you're proposing that these effects occur, then you have
to add something to your control system model that's not in it now, because
all that model can do right now is to decrease the behavior when the CV
increases.

What about in _your_ model? Does the pellet have any _indirect_ effects on
the organization of the system in question?

No, the pellets themselves have only sensory and nutritive effects in my
model, such as it is. Everything following those initial effects depends on
how the organism deals with them. In my model there are several parallel
control systems that control the same variable by different means. The
"search" system controls for some amount of input higher than the
background rate (phase 1). It is turned off when behavior produces more
than this amount of input, and the behavioral system then produces enough
behavior to satisfy the reference level for rate of food intake. The
reference level for rate of food intake is set by a system one or more
levels higher that is concerned with maintaining the level of nutrition. In
the middle of this process there is some sort of tuning procedure, for a
naive animal, which may be a reorganizing process or a more narrowly
focused version of the search system. I've never tried a working model of
this arrangement, so I can't say what problems there might be with it.

But is that the phenomenon that is called "reinforcement" in EAB?

No. That is the phenomenon that is called "satiation" in EAB. The use of
small pellets and the low delivery rates seen on most schedules tend to
miminize the magnitude of this phenomenon over the course of a typical

session.

Do you mean that "satiation" occurs upon ingestion of the very first food
pellet? That is what happens in the control model you're proposing. The
greatest amount of behavior occurs when no food at all has been delivered.
Every pellet of food after that decreases the amount of behavior. You can't
give a small enough amount of food to reverse this relationship. That is
how the control model you're using behaves.

The term can refer to the end-point of the process (reduction of motivation
to zero), but satiation as a variable admits of degrees. If we think of
satiation in terms of reduction of error between the CV and its reference,
then each pellet tends to contribute to satiation.

Right. That is what the model you're using does. The behavior is maximum
when there is no food intake. It drops off as the level of food intake
increases, no matter what the metabolic drain is. The end-point is set by
the rate of gain and the rate of loss, but that doesn't change the basic
relationship. An increase in food intake reduces the behavior rate under
all conditions. I'm speaking, of course, of the model you're using, not
necessarily about what is actually observed.

However, metabolic
output has the opposite effect, and the level of satiation will be
determined by the contribution of these two rates to nutrient level. If
pellet intake rate is small enough and metabolic rate is high enough, we may
not observe any satiation, or even an increase in the level of deprivation.
(Satiation and deprivation are just opposite sides of the same coin.)

The fact that you can't observe something because the effect is too small
to see is irrelevant to the relationship that exists. If you feed the
organism one molecule of food, its behavior rate will drop by some quite
small amount, probably far less than the noise in the measurements, but
there will still be a drop -- if the control model you're using actually
describes the organism.

If that's not what happens during the phenomenon of reinforcement, you're
not using the right model.

I am not attempting to present anything other than a control model. I am
attempting to relate the operation of this control model to the observations
which have been described previously in the language of reinforcement. But
for some reason, you apparently believe that I am attempting to offer an
alternative, reinforcement-based mechanism as a substitute for a control
mechanism.

No, I'm simply pointing out that the control model you're using doesn't
behave like the observations described in the language of reinforcement.
What is (purportedly) observed is that when behavior begins to produce
food, the behavior rate increases. But there is no condition under which
the model you're using will behave that way. If you deprive your model of
food, it will be behaving at the maximum possible rate. Extinction will
never occur. When you connect the feedback loop, the behavior will
immediately decline, arriving finally at an equilbrium rate that depends on
the rate of metabolic losses. Write the model, run it, and see.

Now, if a control model is not the right model, what do you propose as an
alternative?

I have already proposed one. Feel free to try it out, if you wish.

Best,

Bill P.

[Bruce_Abbott]

[From Bruce Abbott (970926.1315 EST)]

Bill Powers (970926.109 MDT) --

Bruce Abbott (970926.1040 EST

What about in _your_ model? Does the pellet have any _indirect_ effects on
the organization of the system in question?

No, the pellets themselves have only sensory and nutritive effects in my
model, such as it is. Everything following those initial effects depends on
how the organism deals with them.

This "everything following" is what I call "indirect effects." "How the
organism deals with them" refers to the state of the organism at the time
the direct effects occur; this state determines, via the relevant
mechanisms, what the "indirect effects" will be.

For example, about two posts ago I brought up the indirect effect of
reorganization, using your proposed reorganization process (involving
essential variables) as an example. The direct effect of the food pellet
was to increase the nutrient level, thus reducing the error that we are
assuming exists at the beginning between nutrient level and its reference.
An indirect effect, via this reduction of error in the nutrient-control
system, is to slow or halt the reselection of output connections. (I've
been hinting for several weeks now that another reorganization mechanism not
involving essential variables might make more sense here, but for
illustration your current proposal will do.) When both the direct
(nutrient-increasing) and indirect (error-reducing; reorganization halting)
effects of the pellets are included, the observed changes in lever-press
frequency are not difficult to account for in a purely mechanistic way.

In my model there are several parallel
control systems that control the same variable by different means. The
"search" system controls for some amount of input higher than the
background rate (phase 1). It is turned off when behavior produces more
than this amount of input, and the behavioral system then produces enough
behavior to satisfy the reference level for rate of food intake. The
reference level for rate of food intake is set by a system one or more
levels higher that is concerned with maintaining the level of nutrition. In
the middle of this process there is some sort of tuning procedure, for a
naive animal, which may be a reorganizing process or a more narrowly
focused version of the search system. I've never tried a working model of
this arrangement, so I can't say what problems there might be with it.

Yes, I was surprised when you offered this new proposal earlier in our
discussion; it is much closer to what I have had in mind as a reorganization
process than the "essential variables" proposal, which to my mind suffers
from a serious difficulty in that systems often seem to reorganize without
there being any serious or persistent error in what might logically be
called an essential variable, and before there would be sufficient reduction
in such error (if it did exist) to keep the selected output link from being
reorganized away. Perhaps we should try out your new proposal and see how
viable it is. It doesn't seem to me that we are very far apart in our
conceptions here.

Do you mean that "satiation" occurs upon ingestion of the very first food
pellet? That is what happens in the control model you're proposing. The
greatest amount of behavior occurs when no food at all has been delivered.
Every pellet of food after that decreases the amount of behavior. You can't
give a small enough amount of food to reverse this relationship. That is
how the control model you're using behaves.

The term can refer to the end-point of the process (reduction of motivation
to zero), but satiation as a variable admits of degrees. If we think of
satiation in terms of reduction of error between the CV and its reference,
then each pellet tends to contribute to satiation.

Right. That is what the model you're using does. The behavior is maximum
when there is no food intake. It drops off as the level of food intake
increases, no matter what the metabolic drain is. The end-point is set by
the rate of gain and the rate of loss, but that doesn't change the basic
relationship. An increase in food intake reduces the behavior rate under
all conditions. I'm speaking, of course, of the model you're using, not
necessarily about what is actually observed.

When you speak of "the model you're using," you evidently mean the
restricted portion of the model (a simple control system) I've been focusing
on thus far in attempting to win your agreement that the reinforcer is
something that has a specific effect on the CV and is not the CV itself.
Now you are taking this portion to be the entire model. Of course, what you
say above is correct in the model as thus far developed (although I have
already presented an elaboration of it by adding your reorganization system,
in order to illustrate how reinforcement (Phase 1) might be accounted for
without inventing nonphysical effects of the pellet).

However, metabolic
output has the opposite effect, and the level of satiation will be
determined by the contribution of these two rates to nutrient level. If
pellet intake rate is small enough and metabolic rate is high enough, we may
not observe any satiation, or even an increase in the level of deprivation.
(Satiation and deprivation are just opposite sides of the same coin.)

The fact that you can't observe something because the effect is too small
to see is irrelevant to the relationship that exists. If you feed the
organism one molecule of food, its behavior rate will drop by some quite
small amount, probably far less than the noise in the measurements, but
there will still be a drop -- if the control model you're using actually
describes the organism.

I did not say that the relationship was irrelevant; I only stated that
satiation will not be observed if the rate of intake equals or exceeds the
rate of drain. In that case there will be no observed drop in output, since
the error in the nutrient-control system will be steady or increasing.

No, I'm simply pointing out that the control model you're using doesn't
behave like the observations described in the language of reinforcement.
What is (purportedly) observed is that when behavior begins to produce
food, the behavior rate increases. But there is no condition under which
the model you're using will behave that way. If you deprive your model of
food, it will be behaving at the maximum possible rate. Extinction will
never occur. When you connect the feedback loop, the behavior will
immediately decline, arriving finally at an equilbrium rate that depends on
the rate of metabolic losses. Write the model, run it, and see.

I don't need to write the model, run it, and see. I know how this portion
of the model behaves, and it is exactly as you describe. Why have you
chosen to ignore my earlier exposition in which I added a reorganizing
system? Reinforcement is all about learning.

Regards,

Bruce

[Bill_Powers1]

[From Bill Power (970926.1533 MDT)]

Bruce Abbott (970926.1315 EST)--

No, the pellets themselves have only sensory and nutritive effects in my
model, such as it is. Everything following those initial effects depends on
how the organism deals with them.

This "everything following" is what I call "indirect effects." "How the
organism deals with them" refers to the state of the organism at the time
the direct effects occur; this state determines, via the relevant
mechanisms, what the "indirect effects" will be.

I'm not used to speaking of "the effect" of a variable when the state of
the thing affected (as well as many other variables) determines what the
effect will be, and when the effect of the same state of the same variable
can change over a wide range. What is the effect of gasoline flow on the
speed of a car? This is very hard to answer in any simple way, because the
flow is not independent of the speed of the car.

For example, about two posts ago I brought up the indirect effect of
reorganization, using your proposed reorganization process (involving
essential variables) as an example. The direct effect of the food pellet
was to increase the nutrient level, thus reducing the error that we are
assuming exists at the beginning between nutrient level and its reference.
An indirect effect, via this reduction of error in the nutrient-control
system, is to slow or halt the reselection of output connections. (I've
been hinting for several weeks now that another reorganization mechanism not
involving essential variables might make more sense here, but for
illustration your current proposal will do.) When both the direct
(nutrient-increasing) and indirect (error-reducing; reorganization halting)
effects of the pellets are included, the observed changes in lever-press
frequency are not difficult to account for in a purely mechanistic way.

First, I agree with you that reorganization is probably not the appropriate
model in this case. A search strategy, innate or learned, is likely to be
more realistic.

What do you mean by a "purely mechanistic way?" We can speak of the effect
of an input quantity on a perceptual signal, because the perceptual signals
depends on the input quantity alone. But we can't speak of "the effect of
an input quantity on behavior" (in a control system), because the input
quantity _depends on_ the behavior at the same time the behavior depends on
it. You have to solve the system equations to see how the input quantity
and behavior are related, and when you do you find that neither one is an
independent variable; they both depend on two other variables, the
reference signal and the disturbance. The appearance that either one
"causes" the other is spurious.

So I repeat, what do you mean by "mechanistic?" Do you mean the old idea
that behavior must ultimately be traceable back to effects of the
environment on the organism?

... I was surprised when you offered this new proposal earlier in our
discussion; it is much closer to what I have had in mind as a reorganization
process than the "essential variables" proposal, which to my mind suffers
from a serious difficulty in that systems often seem to reorganize without
there being any serious or persistent error in what might logically be
called an essential variable, and before there would be sufficient reduction
in such error (if it did exist) to keep the selected output link from being
reorganized away. Perhaps we should try out your new proposal and see how
viable it is. It doesn't seem to me that we are very far apart in our
conceptions here.

Possibly so. Actually, in thinking about a reasonably experienced animal, I
try to save the "reorganization" hypothesis as a last resort (unless I'm
just feeling lazy). The random reorganizing process leaves behind it
systematic control processes that make further reorganization unnecessary,
so it's very likely that we will be seeing systematic processes most of the
time.

When you speak of "the model you're using," you evidently mean the
restricted portion of the model (a simple control system) I've been focusing
on thus far in attempting to win your agreement that the reinforcer is
something that has a specific effect on the CV and is not the CV itself.

I know you've been trying to convince me of that, but in case you haven't
noticed, I remain unconvinced. I think the rate of delivery of food pellets
is a controlled variable, and that the nutrient control system acts by
setting a reference level for delivery (or intake) rate of food pellets.

You propose that nutrient level is _the_ CV. To what is the error signal of
that control system connected? The logical connection is to a system that
provides food intake, but in that case the output of the nutrient control
system would not produce a behavior. It would be up to the food intake
control system to generate a behavior that would cause food intake.

Now you are taking this portion to be the entire model. Of course, what you
say above is correct in the model as thus far developed (although I have
already presented an elaboration of it by adding your reorganization system,
in order to illustrate how reinforcement (Phase 1) might be accounted for
without inventing nonphysical effects of the pellet).

All we have is "the model as thus far developed." If you want to develop it
farther, by all means do. What you've been saying about the reinforcing
effects of food pellets does not connect to the model as developed so far;
in fact, we get just the opposite of the right effect. So to get the right
effect, you have to add something to the model that will produce the right
effect.

I did not say that the relationship was irrelevant; I only stated that
satiation will not be observed if the rate of intake equals or exceeds the
rate of drain. In that case there will be no observed drop in output, since
the error in the nutrient-control system will be steady or increasing.

I thought satiation occurred exactly when the rate of intake equals the
rate of loss (the rate of loss is assumed proportional to nutrient level).

I don't need to write the model, run it, and see. I know how this portion
of the model behaves, and it is exactly as you describe. Why have you
chosen to ignore my earlier exposition in which I added a reorganizing
system? Reinforcement is all about learning.

If an animal already has learned a search control system that behaves as I
propose, then what we would observe as an apparent effect of reinforcement
does not involve learning at all. We're simply seeing the operation of a
system with fixed characteristics. It searches until it finds the food;
then it stops searching and uses that food source to satisfy its need for
food until the food is stops appearing. Then it starts searching again. The
food pellets it gets have no effect on changing this organization.

Best,

Bill P.

[Bruce_Abbott]

[From Bruce Abbott (970926.2030 EST)]

Bill Power (970926.1533 MDT)

Bruce Abbott (970926.1315 EST)

No, the pellets themselves have only sensory and nutritive effects in my
model, such as it is. Everything following those initial effects depends on
how the organism deals with them.

This "everything following" is what I call "indirect effects." "How the
organism deals with them" refers to the state of the organism at the time
the direct effects occur; this state determines, via the relevant
mechanisms, what the "indirect effects" will be.

I'm not used to speaking of "the effect" of a variable when the state of
the thing affected (as well as many other variables) determines what the
effect will be, and when the effect of the same state of the same variable
can change over a wide range. What is the effect of gasoline flow on the
speed of a car? This is very hard to answer in any simple way, because the
flow is not independent of the speed of the car.

I'm only using it here to distinguish direct influence from changes that
occur elsewhere in the system that flow from that influence. Otherwise, you
get to say things like "the _only_ effect of the food pellet is on nutrient
level" when in fact a whole cascade of other changes may follow, depending
on the state of the system at the time. This can lead to confusion. For
example, it is true that the "only effect" of igniting the gasoline/air
mixture in the cylinder is to release chemical energy in the form of heat,
but somewhere, somehow, that effect translates to driving the piston down in
its cylinder and thus propelling the car forward. But when you say "the
only effect," readers may assume that this is all that happens consequent on
that ignition. This makes the motion of the piston appear somewhat magical.

. . . When both the direct
(nutrient-increasing) and indirect (error-reducing; reorganization halting)
effects of the pellets are included, the observed changes in lever-press
frequency are not difficult to account for in a purely mechanistic way.

First, I agree with you that reorganization is probably not the appropriate
model in this case. A search strategy, innate or learned, is likely to be
more realistic.

I'm really glad to hear that. (:->

What do you mean by a "purely mechanistic way?" We can speak of the effect
of an input quantity on a perceptual signal, because the perceptual signals
depends on the input quantity alone. But we can't speak of "the effect of
an input quantity on behavior" (in a control system), because the input
quantity _depends on_ the behavior at the same time the behavior depends on
it. You have to solve the system equations to see how the input quantity
and behavior are related, and when you do you find that neither one is an
independent variable; they both depend on two other variables, the
reference signal and the disturbance. The appearance that either one
"causes" the other is spurious.

Yes, I know. I am not speaking of linear cause-and-effect. What I mean by
a "purely mechanistic way" is that one can specify a physical mechanism,
involving nothing but ordinary physical interactions, that will behave in
the required way.

So I repeat, what do you mean by "mechanistic?" Do you mean the old idea
that behavior must ultimately be traceable back to effects of the
environment on the organism?

No.

When you speak of "the model you're using," you evidently mean the
restricted portion of the model (a simple control system) I've been focusing
on thus far in attempting to win your agreement that the reinforcer is
something that has a specific effect on the CV and is not the CV itself.

I know you've been trying to convince me of that, but in case you haven't
noticed, I remain unconvinced. I think the rate of delivery of food pellets
is a controlled variable, and that the nutrient control system acts by
setting a reference level for delivery (or intake) rate of food pellets.

A reasonable model. But Bill, the rate of delivery of a food pellet is not
a reinforcer (although an increase in the rate of delivery might be). The
amount of money I have in my pocket is not a reinforcer (although an
increase in that amount might be). The amount of pain in my back muscles is
not a reinforcer (although a decrease in that amount certainly would be!).
Your CV simply does not correspond to the everyday notion of a reward, from
which the concept of a reinforcer was originally derived. Achieving a given
rate of pellet delivery might be rewarding, but not rate-of-delivery itself,
which could be anything from zero up.

You propose that nutrient level is _the_ CV. To what is the error signal of
that control system connected? The logical connection is to a system that
provides food intake, but in that case the output of the nutrient control
system would not produce a behavior. It would be up to the food intake
control system to generate a behavior that would cause food intake.

Do you recall my describing a whole chain of perceptions that have to be
controlled, in sequence, by the rat before its behavior can result in a
change in its nutrient level? To model how lever-pressing gets acquired, we
have to create a more complex model in which the output of the nutrient
control system sets appropriate references for each of these other systems,
and in just the right order to bring about the required sequence of
behavioral acts. To simplify the model considerably, we might just pretend
that lever-pressing just inputs a food pellet into the rat's stomach
directly, as a first approximation. But that still leaves us with a
two-level system. What gets changed via experience is what reference(s) the
output of the nutrient-control system gets connected to.

I thought satiation occurred exactly when the rate of intake equals the
rate of loss (the rate of loss is assumed proportional to nutrient level).

No, it occurs when the system reaches its reference level and thus no
further action occurs. When you've had enough, and don't want no more, you
are satiated. Beyond that point you will take action to _prevent_ more food
from entering your system.

I don't need to write the model, run it, and see. I know how this portion
of the model behaves, and it is exactly as you describe. Why have you
chosen to ignore my earlier exposition in which I added a reorganizing
system? Reinforcement is all about learning.

If an animal already has learned a search control system that behaves as I
propose, then what we would observe as an apparent effect of reinforcement
does not involve learning at all. We're simply seeing the operation of a
system with fixed characteristics. It searches until it finds the food;
then it stops searching and uses that food source to satisfy its need for
food until the food is stops appearing. Then it starts searching again. The
food pellets it gets have no effect on changing this organization.

Sounds good to me, although I could argue about whether a system that
"searches until it finds food" is learning something -- like where the food
is, or what behavior is required in order to obtain it. After all, in the
broadest sense, the brain is a "fixed system" with parameters, and thus,
under your definition, incapable of learning. But that is just a
definitional matter; calling it learning or not doesn't change the mechanism
you have in mind, or how it might work.

I'll be away for most of this weekend.

Regards,

Bruce

[Bill_Powers1]

[From Bill Powers (970927.0858 MDT)]

Bruce Abbott (970926.2030 EST)--

This "everything following" is what I call "indirect effects." "How the
organism deals with them" refers to the state of the organism at the time
the direct effects occur; this state determines, via the relevant
mechanisms, what the "indirect effects" will be.

But the state of the organism is not something passive that is simply acted
upon by the food pellets. The organism is actively controlling something,
and "the system" consists of numerous interacting variables.

Oh, to hell with it. My teeth are set on edge by the way you're talking
about all this, but I guess the best course is just to wait to see what you
come up with.

Best,

Bill P.

[R_Marken]

[From Rick Marken (970927.1230)]

Bruce Abbott (970926.2030 EST)--

But Bill, the rate of delivery of a food pellet is not a
reinforcer

Right. It's a _controlled variable_, in theory _and_ in fact (as
determined by testing for stability in the face of disturbance).

(although an increase in the rate of delivery might be).

Ok. It _might_ be a reinforer. How do we determine whether or
not it _is_ a reinforcer? If you can explain how I can test
to determine whether or not "increase in rate of food delivery"
(or any other variable) is a reinforcer then I'll stop believing
that reinforcers don't exist and start believing that reinforcers
_might_ exist -- and I'll start testing to see if they do.

The amount of pain in my back muscles is not a reinforcer
(although a decrease in that amount certainly would be!).

How do you know for certain that a decrease in the amount of
pain is a reinforcer?

Your CV simply does not correspond to the everyday notion of a
reward, from which the concept of a reinforcer was originally
derived.

That's for sure. PCT shows that "rewards", like reinforcers, are
not what they seem. In most cases, a "reward" is simply a
disturbance that moves a controlled variable closer to it's
reference state; the resulting decrease in error is probably
what makes a reward feel rewarding.

Best

Rick

···

--
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken/

[Bruce_Gregory9]

[From Bruce Gregory (970928.0700 EDT]

Bill Powers (970927.0858 MDT)]

to Bruce Abbott (970926.2030 EST)--

Oh, to hell with it. My teeth are set on edge by the way you're talking
about all this, but I guess the best course is just to wait to see what

you

come up with.

Let me remind everyone of the words of Quine that I find
so insightful:

"I am not suggesting the dependence of being upon language.
What is under considerationis not the ontological state of
affairs, but the ontological commitment of a discourse. What
there is does not in general depend on one's use of
language, but what one says there is does."

"Logic and the Reification of Universals" in
_From a Logical Point of View_

Sancho

[Bruce_Abbott]

[From Bruce Abbott (970927.0855)]

Bill Powers (970927.0858 MDT)

Bruce Abbott (970926.2030 EST)

This "everything following" is what I call "indirect effects." "How the
organism deals with them" refers to the state of the organism at the time
the direct effects occur; this state determines, via the relevant
mechanisms, what the "indirect effects" will be.

But the state of the organism is not something passive that is simply acted
upon by the food pellets. The organism is actively controlling something,
and "the system" consists of numerous interacting variables.

Don't put words in my mouth. I neither said nor implied that the organism
was "something passive that is simply acted upon by the food pellets." What
I said was that the effects of a variable on other variables in this complex
system depend on the state of the system. As I recall, any physical
determinate system can be defined completely by its variables, their
functions, and an initial state (e.g., at time t0), from whence one computes
all subsequent states at time t1, t2, etc. Or are you now claiming that a
control system is an indeterminate system?

Oh, to hell with it. My teeth are set on edge by the way you're talking
about all this, but I guess the best course is just to wait to see what you
come up with.

Yes, I know what you mean. Why, _my_ teeth were set on edge just last night
by this little example [Bill Powers (970928.1100 MDT)]:

2. On a radio there is a knob fastened to the shaft of a potentiometer. A
clockwise rotation of the knob causes the sound volume from the radio to
increase.

Geez, Bill, everyone knows that the only effect of clockwise rotation of the
knob is clockwise rotation of the shaft. It doesn't do anything at all to
the sound volume from the radio.

Bill on my analysis making the reinforcer something that has an effect on
the CV and not the CV itself:

I know you've been trying to convince me of that , but in case you haven't
noticed, I remain unconvinced. I think the rate of delivery of food pellets
is a controlled variable, and that the nutrient control system acts by
setting a reference level for delivery (or intake) rate of food pellets.

A reasonable model. But Bill, the rate of delivery of a food pellet is not
a reinforcer (although an increase in the rate of delivery might be). The
amount of money I have in my pocket is not a reinforcer (although an
increase in that amount might be). The amount of pain in my back muscles is
not a reinforcer (although a decrease in that amount certainly would be!).
Your CV simply does not correspond to the everyday notion of a reward, from
which the concept of a reinforcer was originally derived. Achieving a given
rate of pellet delivery might be rewarding, but not rate-of-delivery itself,
which could be anything from zero up.

Bill's argument for why the reinforcer should be identified as the CV:

. . . .

Not especially convincing, Bill.

And you have left me completely in the dark as to how you feel about these
replies:

What do you mean by a "purely mechanistic way?" We can speak of the effect
of an input quantity on a perceptual signal, because the perceptual signals
depends on the input quantity alone. But we can't speak of "the effect of
an input quantity on behavior" (in a control system), because the input
quantity _depends on_ the behavior at the same time the behavior depends on
it. You have to solve the system equations to see how the input quantity
and behavior are related, and when you do you find that neither one is an
independent variable; they both depend on two other variables, the
reference signal and the disturbance. The appearance that either one
"causes" the other is spurious.

Yes, I know. I am not speaking of linear cause-and-effect. What I mean by
a "purely mechanistic way" is that one can specify a physical mechanism,
involving nothing but ordinary physical interactions, that will behave in
the required way.

I thought satiation occurred exactly when the rate of intake equals the
rate of loss (the rate of loss is assumed proportional to nutrient level).

No, it occurs when the system reaches its reference level and thus no
further action occurs. When you've had enough, and don't want no more, you
are satiated. Beyond that point you will take action to _prevent_ more food
from entering your system.

Regards,

Bruce

[HANS_BLOM2]

[Hans Blom, 970929c]

(Rick Marken (970927.1230))

PCT shows that "rewards", like reinforcers, are not what they seem.
In most cases, a "reward" is simply a disturbance that moves a
controlled variable closer to it's reference state; the resulting
decrease in error is probably what makes a reward feel rewarding.

You're so right. Now, if we could only learn to somehow manipulate
those "rewarding disturbances" so that they consistently result in a
decrease in error... Too bad PCT doesn't allow that ;-).

Greetings,

Hans