Control of reinforcement

CSGnet Archive CSG1995
1

[From Bruce Abbott (950730.1300 EST)]

Rick Marken (950729.2020) --

I think I now understand what Bruce was up to in the post I took
to be a satire. Bruce was showing that the reinforcement rates
observed in ratio experiments (ratio requirement < 64 presses)
could be produced by a rat pressing at a constant rate after a
fixed delay (the consumption period, c). This is, indeed an
interesting possibility and it is why Bruce claimed that reinforcement
rate is not a controlled variable; the reinforcemnt rate could
be a side effect of the actions of an automaton that emits a
fixed delay (c) after a reinforcer and responds at a regular rate (p)
thereafter, until the next reinforcemnet. It is also why Bruce
suggested that response rate (p) and/or consumption period (c)
might be controlled variables.

By Jove, I think he's got it! (:->

Of course, response rate and
consumption period are consequences of action; but these would
be rather perculiar variables to control since doing so would mean
relinquishing control of nourishment. Moreover, there is no evidence
that they are controlled anyway, other than their computed stability . . .

But CYCLIC2 seems to show that response rate may not be as stable as it
seems, once you remove a constant collection time from the analysis;
moreover, Bill's control model seems to do an excellent job of fitting these
data under the assumption that reinforcement rate IS under control. So
perhaps things are not so strange as they seem.

I think the control model of operant conditioning (control of nourishment
in the context of changing feedback functions) keeps getting better as
we improve our understanding of the environmental constraints that exist
in operant conditioniong research. I think the fit of the control model with
reinforcement rate as the controlled perceptual variable makes a pretty
convincing case that reinforcement rate is, indeed, the variable under
control.

An' you ain't seen nothin', yet!

But I also think this episode shows that conventional operant
conditioning experiments are not the way to go about studying control.
Studies like Staddon's baroque "cyclic ratio" cantata show how difficult
it is to make sense of data that was collected without any notion that
variables are being controlled by the subjects in these experiments.

But Rick, as I never tire of repeating to you, this is just preparation, to
gain some insight into the variables at work and their relationships.
Moreover, Staddon's cyclic ratio study was conducted to do exactly what you
ask for: to evaluate a control model. Staddon assumed that rate of food
intake was being controlled. Both you and Bill approached the data just as
Staddon did, assuming that the imposition of higher ratios acted as a
disturbance to food rate which was countered by means of increasing the rate
of lever pressing. So I really don't understand the criticism.

It's fun to try to make sense of data like that from Staddon's study
to the extent that we can -- but I still think it would be best to
just start doing some simple control experiments the right way. The
ambiguity about what is being controlled in operant experiments could
be solved with a few weeks of tests aimed at determining which
variables are, indeed, systematically protected from disturbance --
ie, controlled.

Rick, analyzing the Ettinger and Staddon (1982) data has been more than fun;
it has been something of an eye-opener. Having conducted this analysis, we
are now in possession of information which will help to improve the design
of our own control studies. For one thing, we now have a better idea as to
what data we need to collect. For another thing, we know that a naive model
which neglects collection time is inadequate.

Furthermore, we aren't done yet, as there are other aspects of the situation
we need to be thinking about and debating (when I have a bit more time).
We're having to think about some important issues that haven't really been
dealt with as yet within PCT, such as the allocation of behavior. In my
judgement, learning how to take such factors into account within a
control-system model will represent an important advance for PCT.

This isn't an either-or thing. We can do basic control studies and we can
anlayze extant data to learn what we can from them which will help up to
develop more adequate control models that apply more generally to real-world
situations.

Regards,

Bruce

2

From Tom Bourbon [950730.2245]

[From Bruce Abbott (950730.1300 EST)]

Who replied to

Rick Marken (950729.2020) --

. . .

Bruce:

Furthermore, we aren't done yet, as there are other aspects of the situation
we need to be thinking about and debating (when I have a bit more time).
We're having to think about some important issues that haven't really been
dealt with as yet within PCT, such as the allocation of behavior. In my
judgement, learning how to take such factors into account within a
control-system model will represent an important advance for PCT.

I'm not quite with you on this idea, Bruce. My impression is that Staddon
and others mistakenly tried to model animals as though they (the animals)
control their own _behaviors_. Staddon, and those who follow a similar
lead, speak of _behaviors_ being "in competition" with one another, or of
_behaviors_ being "allocated," or "selected." They have completely missed
the point that animals are controlling their perceptions of someting other
than their "behaviors." As I suggested in my earlier post (Tom Bourbon
[950730.1313]), I think a series of simple, direct, tests for the animals'
controlled variables would provide real data for PCT modeling. Furthermore,
the results of the modeling effort would include a demonstration of the
serious flaws in all of the research in which people have tried to study
animals as though they are controllers of "behavors." I don't believe they
are, and if they are not, there's no point in trying to use PCT to model
them as though they are. (I might be wrong. Maybe the animals in all of
those studies existed in a different kind of world, in which they could
survive by controlling their percpetions of their behaviors, rather than
perceptions of other variables that were affected by their actions. I've
read science fiction more strange that that.)

This isn't an either-or thing. We can do basic control studies and we can
anlayze extant data to learn what we can from them which will help up to
develop more adequate control models that apply more generally to real-world
situations.

Fine. Absolutely no problem with the idea that we can do basic control
studies _and_ analyze extant data. But I have some reservations about the
idea that extant data will help us develop "more adequate control models."
They might, but I don't think so. For one thing, the data were collected
with procedures designed to support the notions that animals control their
behaviors, and that the behaviors they select or allocate can be in
competition, or in conflict.

On the other hand, I believe good research that tests for variables
controlled by animals will provide material for PCT modeling _and_ that the
results of the modeling will reveal the illusory nature of earlier ideas
that animals allocate, select, and control "behaviors."

Just an opinion.

Tom

3

[From Bill Powers (981014.0522 MDT)]

Bruce Abbott (981014.1650 EDT)]

I'd better reply to this lest I forget altogether.

I _have_ stated that _some_ psychologists do understand control theory quite
well. (This is quite another matter from saying that "conventional
psychologists" -- which you take to be all psychologists save yourself and
perhaps two others -- do.) I don't have the references here with me at the
moment, but I'll supply a few examples when I get home. Meanwhile, do you
remember Ettinger and Staddon (1983)? This is the study in which the
evidence was said to support the view that rats control the rate of
reinforcement on FR schedules. It's a perfectly good control-system
analysis. The only problem was that, as my reanalysis and replication
showed, rats don't do anything of the sort.

I think you're distorting the findings. First of all, it was not the
_authors_ of the study who claimed it showed control of reinforcement; that
was me. Second, what you found was that variations in the rate of
bar-pressing did not control the rate of reinforcement. However, the reason
was not that there were variations in pressing rate which failed to control
the rate of food ingestion. The rats controlled the rate of food intake
very well. Without their behavior, the rate would have been zero. They just
didn't do it by varying the rate of bar-pressing. Your major finding was
that the rate of pressing was actually a constant on FR schedules; it
appeared to vary only because of an artifact involving the finite
collection time. While actually pressing the bar, the rats pressed it at a
constant rate independent of the schedule. The way they controlled their
food intake was to divide their time (variably) between pressing the bar
and doing other things. You have now demonstrated this effect for several
other studies and for the one you did yourself.

It was, I'll admit, educational to me to find that I could fit a
control-system model to the data (particularly the data of Motheral) when
the variations in output pressing rate were illusory. I can claim
innocence, however, in that the data were presented as variations in
pressing rate and I accepted them at face value. It was only when you
discovered how collection time created the illusion of a variable rate of
pressing that we both realized that the fit was spurious (the basic
discovery, however, was entirely yours).

By the way, I recall that you were quite sure the story would be different
for Variable-Interval schedules. As I recall, your more recent experiments
with VI schedules seemed to be showing the same effect: no significant
variations in pressing rate on different schedules. Is that still holding up?

I believe your discovery that pressing rate does not change on FR schedules
is of enormous significance in EAB. I'm sure many experimenters have been
fooled, as you and I were, by the _apparent_ variations in pressing rate on
FR schedules. Given the apparent variations, many conclusions must have
been drawn from studies using FR schedules, starting at least with Ferster
and Skinner's, which have no basis at all. This means not only that the
conclusions from the initial studies are baseless, but that all subsequent
works that relied on those conclusions have lost some of their justification.

If the same conclusion should hold for VI schedules, most of the
experimental work in EAB would be called into question and would have to be
re-examined, or even re-done if the necessary raw data were not preserved.
Are you approaching the stage of publication of these findings? It will be
interesting to see how the EAB community reacts to them.

Best,

Bill P.

4

[From Bruce Gregory 9981015.1132 EDT)]

Rick Marken (981015.0820)

Tell me, Bruce A., are you ever going to answer my questions:

1. Do you agree that controlled variables (PCT sense) are
real (observable) phenomena?

2. If so, then what are these variables called in conventional
psychology? In particular, what are these variables called in
textbooks on behavioral research (such as yours)?

If I see the hand of God everywhere I look in Nature, is God an observable
phenomenon? This question is not a flip as it might seem. We can look at the
"same phenomenon" and see very different things. In order to see controlled
variables, you need to looking from a conceptual framework in which control
is significant. You want a world in which controlled variables either exist
or they do not. We live in a world where even electrons fail this test.

Bruce Gregory

5

[From Rick Marken (981015.0820)]

Bruce Abbott (981014.1650 EDT) --

I _have_ stated that _some_ psychologists do understand control
theory quite well.

Bill Powers (981014.0522 MDT) --

..it was not the _authors_ of the study who claimed it showed
control of reinforcement; that was me.

Oops. One more conventional psychological study of controlled
variables down the tubes.

Tell me, Bruce A., are you ever going to answer my questions:

1. Do you agree that controlled variables (PCT sense) are
real (observable) phenomena?

2. If so, then what are these variables called in conventional
psychology? In particular, what are these variables called in
textbooks on behavioral research (such as yours)?

As I said, I'm working on a paper about controlled variables.
The target audience for this paper is conventional psychologists
like yourself. So your answers will help me tailor the paper to
this audience. If, as you say, you are a proponent of PCT then
surely you will help me on this project.

Thanks

Rick

···

--

Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken/

6

[From Rick Marken (981015.0830)]

Bill Powers (981014.0522 MDT)

... The rats controlled the rate of food intake very well. Without
their behavior, the rate would have been zero. They just
didn't do it by varying the rate of bar-pressing...

Great post, by the way, Bill. Very clear. Excellent points.

I am particularly interested in hearing Bruce A.'s reply to the
following.

If the same conclusion [constant response rate regardless of
reinforcement schedule] should hold for VI schedules, most of the
experimental work in EAB would be called into question and would
have to be re-examined, or even re-done if the necessary raw data
were not preserved. Are you approaching the stage of publication
of these findings? It will be interesting to see how the EAB
community reacts to them.

Interesting, indeed.

Best

Rick

···

--

Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken/

7

[From Bruce Abbott (981015.1140 EST)]

Bill Powers (981014.0522 MDT) --

Bruce Abbott (981014.1650 EDT)

I _have_ stated that _some_ psychologists do understand control theory quite
well. (This is quite another matter from saying that "conventional
psychologists" -- which you take to be all psychologists save yourself and
perhaps two others -- do.) I don't have the references here with me at the
moment, but I'll supply a few examples when I get home. Meanwhile, do you
remember Ettinger and Staddon (1983)? This is the study in which the
evidence was said to support the view that rats control the rate of
reinforcement on FR schedules. It's a perfectly good control-system
analysis. The only problem was that, as my reanalysis and replication
showed, rats don't do anything of the sort.

I think you're distorting the findings.

So what else is new? You always think that.

First of all, it was not the
_authors_ of the study who claimed it showed control of reinforcement; that
was me.

Really? I wonder, then, how you managed to insert your control-of
reinforcement-rate model into their article, over 10 years after they wrote it.

Second, what you found was that variations in the rate of
bar-pressing did not control the rate of reinforcement.

What I found was the rate of reinforcement was not controlled. By any means
whatsoever.

However, the reason
was not that there were variations in pressing rate which failed to control
the rate of food ingestion. The rats controlled the rate of food intake
very well. Without their behavior, the rate would have been zero. They just
didn't do it by varying the rate of bar-pressing.

You must have a strange new definition for control of rate of which I am
unaware. Disturbances to rate of reinforcement (produced by changing the FR
ratio) went unopposed. Please explain how a variable, disturbances to which
are not opposed by control action, is controlled. Rate of reinforcement
failed the Test, for godsakes.

It was a two-phase situation. When there was no food in the cup, the rats
acted to restore the state of the cup to "food-present." When there was
food in the cup, a second control system acted to pick up and consume the
food, thus disturbing the variable being controlled by the first system.
The first system then acted to restore the cup to food-present. And so on.

Your major finding was
that the rate of pressing was actually a constant on FR schedules; it
appeared to vary only because of an artifact involving the finite
collection time. While actually pressing the bar, the rats pressed it at a
constant rate independent of the schedule. The way they controlled their
food intake was to divide their time (variably) between pressing the bar
and doing other things. You have now demonstrated this effect for several
other studies and for the one you did yourself.

Hypothesis: Sometimes other control systems switched in and used the
skeletal muscles to controlled other variables rather than lever-pressing.
As the animal filled up on pellets, less time was spent controlling for
pellet presence and more time was spent using the same output devices (the
skeletal muscles) to oppose disturbances to other variables. As a result,
rate of food intake diminished.

By the way, I recall that you were quite sure the story would be different
for Variable-Interval schedules. As I recall, your more recent experiments
with VI schedules seemed to be showing the same effect: no significant
variations in pressing rate on different schedules. Is that still holding up?

Your recall of the data is incorrect. What the data show is that response
rate declines as the average interval length increases. That is, lower
rates of reinforcement are accompanied by lower rates of responding,
contrary to your earlier prediction, based on PCT, that the reverse would be
true.

Are you approaching the stage of publication of these findings?

Yes. I plan to present these studies at a convention next spring and should
have something submitted for publication by then.

Regards,

Bruce

8

From [ Marc Abrams (981015.1206) ]

Just a passing thought.

Rick, I realize it might not be as much fun but I think if
you spent as much time trying to find out _how_ you might
interest conventional Psychologists in PCT rather then
_telling_ them what they should be interested in you might
make better headway.

There is no question that CV's are central to the operation
of the control process, but other things are not any less
important ( at least to some people ). For instance;
We control any number of things at the same time. _How_ do
we prioritize which ones to attend to first. Can we model
that in some way? Reference levels can change by a number of
different means, _How_ does that happen? What effect does
that have on the rest of the hierarchy? Can we model that in
some way? qi is a nice neat mathematical function in the
model. It can get pretty messy in the real world. can we
begin to explore _how_ to model that? What about memory?
The list goes on. There is a lot of great stuff to look at.

I just don't see the point in spending a whole lot of time
moaning and groaning about what isn't, when we have so much
on the plate. If we do interesting and meaningful things
_to others_ ) then we will attract people, if we don't then
we won't. It's really that simple. Bruce Abbott's business
is his business. Why Rick, as a Human factors engineer have
_you_ not published about PCT use in the design of control
systems? Don't answer, it was a rhetorical question. It
doesn't matter. What matters is your contribution to this
list. I view Bruce Abbott, Bruce Gregory and everyone else
the Same way. I don't believe they are being given a chance
to contribute. By this I mean there are questions, such as
the ones I stated above that currently have no clear cut
answers. Is this list open to talking about the
possibilities or are we restricted to talk about what Rick
and Bill deem to be essential and only that.

Marc

9

[From Rick Marken (981015.0855)]

Bruce Gregory 9981015.1132 EDT) --

If I see the hand of God everywhere I look in Nature, is God an
observable phenomenon?

God is not but the "hand of God" apparently is.

Best

Rick

···

--

Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken/

10

[From Bill Powers (981015.1605 MDT)]

Bruce Abbott (981015.1140 EST)--

Meanwhile, do you
remember Ettinger and Staddon (1983)? This is the study in which the
evidence was said to support the view that rats control the rate of
reinforcement on FR schedules. It's a perfectly good control-system
analysis. The only problem was that, as my reanalysis and replication
showed, rats don't do anything of the sort.

I'm sorry for confusing one study with another. I was thinking of Staddon's
cyclic ratios, and Motheral's stuff cited by Staddon.

What I found was the rate of reinforcement was not controlled. By any means
whatsoever.

This is odd, I must admit.

You must have a strange new definition for control of rate of which I am
unaware. Disturbances to rate of reinforcement (produced by changing the FR
ratio) went unopposed.

That is not a disturbance of the rate of reinforcement, but of the
environmental feedback function. But I was thinking of the wrong
experiment, so it's all moot.

Please explain how a variable, disturbances to which
are not opposed by control action, is controlled. Rate of reinforcement
failed the Test, for godsakes.

Maybe that was not what was controlled. Not everything in the universe is a
rate.

It was a two-phase situation. When there was no food in the cup, the rats
acted to restore the state of the cup to "food-present." When there was
food in the cup, a second control system acted to pick up and consume the
food, thus disturbing the variable being controlled by the first system.
The first system then acted to restore the cup to food-present. And so on.

Why wasn't this evidence of control? A logic-level model could be defined
this way:

reference = food in cup
perception = food/no food in cup
action = if error, press bar at some rate, possibly constant
feedback function: if number presses > n, food to cup, reset counter

This is a pretty crude control system, but judging from what we saw, it may
be appropriately crude. Clearly, varying n will not have any effect: the
system will just press the bar until the error goes to zero.

Your major finding was
that the rate of pressing was actually a constant on FR schedules; it
appeared to vary only because of an artifact involving the finite
collection time. While actually pressing the bar, the rats pressed it at a
constant rate independent of the schedule. The way they controlled their
food intake was to divide their time (variably) between pressing the bar
and doing other things. You have now demonstrated this effect for several
other studies and for the one you did yourself.

Hypothesis: Sometimes other control systems switched in and used the
skeletal muscles to controlled other variables rather than lever-pressing.
As the animal filled up on pellets, less time was spent controlling for
pellet presence and more time was spent using the same output devices (the
skeletal muscles) to oppose disturbances to other variables. As a result,
rate of food intake diminished.

This is what we observed during _each_ experimental session. The rat would
swarm all over the lever at first, grabbing the food and eating while
working on the lever. Then it took longer and longer to eat each pellet,
eventually sitting back on its haunches and holding the pellet while it
nibbled. We were guessing that its stomach was getting full, although its
level of nutrition was still not up to specs.

However, I believe a different effect occurs when the ratio is large (or
the interval long). The rat then simply stops pressing altogether and
starts looking elsewhere for food. When it does press the bar, it presses
as fast as it can, not at a variable rate. The appearance of a relationship
between ratio and rate at large ratios comes partly from collection time
effects and partly from averaging periods of non-pressing with periods of
fast pressing. At low ratios it comes almost exclusively from
collection-time effects.

By the way, I recall that you were quite sure the story would be different
for Variable-Interval schedules. As I recall, your more recent experiments
with VI schedules seemed to be showing the same effect: no significant
variations in pressing rate on different schedules. Is that still holding

up?

Your recall of the data is incorrect. What the data show is that response
rate declines as the average interval length increases. That is, lower
rates of reinforcement are accompanied by lower rates of responding,
contrary to your earlier prediction, based on PCT, that the reverse would be
true.

Do you mean that the actual rate of pressing during periods of continuous
pressing varies with the interval? My impression from your earlier reports
was that the apparent change in rate of pressing was caused by collection
time effects and by spending periods of time away from the lever. Those
effects, of course, would not contradict any PCT prediction. Particularly
if there is a duty-cycle effect, practically any model would predict a
simultaneous decline (or rise) in input and output rates.

If there was a clear relationship between scheduled interval and rate of
pressing during continguous periods of pressing, that would be something
new to me. I wasn't aware that you had found any situation in which that
kind of relationship was found.

Best,

Bill P.

11

[From Rick Marken (81015.1540)]

Marc Abrams (981015.1206)--

Rick, I realize it might not be as much fun but I think if
you spent as much time trying to find out _how_ you might
interest conventional Psychologists in PCT rather then
_telling_ them what they should be interested in you might
make better headway.

I have no idea how to interest anyone in PCT if they have no
interest in the phenomenon PCT explains: controlled variables.
I'm not telling conventional psychologists what they should
be interested in. I'm trying to show that conventional
psychologists are unaware of the existence of controlled
variables (a fact that goes a long way towards explaining
why conventional psychologists have taken no particular interest
in PCT).

There is no question that CV's are central to the operation
of the control process, but other things are not any less
important ( at least to some people ). For instance;
We control any number of things at the same time. _How_ do
we prioritize which ones to attend to first.

Why would someone who knows nothing of controlled variables
be interested in how they are prioritized? How does someone
study this question if they have don't know anything about how
to determine whether or not a variable is under control?

Can we model that in some way?

Sure. But how do we know our model has anything to do with human
behavior? We build models to account for variations in the
variables we observe. But the variations in the most important
variables we can observe in the behavior of living systems
(controlled variables) are never reported in conventional
psychological research.

Reference levels can change by a number of different means,
_How_ does that happen?

We have to do experiments to find out. People who don't know
about controlled variables can't do these experiments.

qi is a nice neat mathematical function in the model. It can
get pretty messy in the real world. can we begin to explore
_how_ to model that?

We have to have data that tells us what qi is.

What about memory? The list goes on. There is a lot of great
stuff to look at.

You betcha! But you can't look at this stuff (that is, you can't
do the apporpriate experiments) in a way that is relevant to
PCT if you have no idea that there are controlled variables to
be observed.

Is this list open to talking about the possibilities or are we
restricted to talk about what Rick and Bill deem to be essential
and only that.

This list is not restricted in any way. But Bill and I tend to
talk about things that have to do with understanding the purposive
behavior of living systems. You can't learn about the behavior
of living systems just by tinkering with models. You do have to
know how the model works -- that's why Bill is giving the tutorial
on modeling control (purposive) systems using Vensim. But in order
to learn about the things you mentioned above (prioritizing goals,
setting references, memory, etc) you have to do _research_. You
have to see what happens to variable Y when you vary variable X.
If you do these studies without awareness that variable Y might
be under control or that variable Y might be a variable that affects
the state of a controlled variable, then your conclusions about
what you have found in such an experiment may be completely
misleading.

I think you believe that it's possible to learn about the things
you mention above by just playing with the model. You can't.
As I said in a post about 8 years ago: phenomena first! Models
are important because they help us understand (and predict)
phenomena. But models alone (like phenomena alone) are useless.
And we can't even think about how to model some phenomenon until
we have observed the phenomenon to be modeled: phenomena first.

Best

Rick

···

--
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken/

12

[From Chris Cherpas (981015.1745 PT)]

Bill Powers (981015.1605 MDT)--

However, I believe a different effect occurs when the ratio is large (or
the interval long). The rat then simply stops pressing altogether and
starts looking elsewhere for food. When it does press the bar, it presses
as fast as it can, not at a variable rate. The appearance of a relationship
between ratio and rate at large ratios comes partly from collection time
effects and partly from averaging periods of non-pressing with periods of
fast pressing. At low ratios it comes almost exclusively from
collection-time effects.

Bill, has Bruce A. told you just how many of these
kinds of discussions have filled the pages of JEAB
since the 1960s? If not, I think it's a little
unfair, since I know he knows, and it's not obvious
that you would be aware of it otherwise.

Bruce, I'm not against any quest for knowledge,
but why subject innocent bystanders to this
kind of torture?

Best regards, and possible regrets,
cc

13

From [ Marc Abrams (9810115.0027) ]

[From Rick Marken (81015.1540)]

I have no idea how to interest anyone in PCT if they have

no

interest in the phenomenon PCT explains: controlled
variables.

*You* can't interest anyone. Your work can.

I'm not telling conventional psychologists what they should
be interested in.

Sure you are and your being *very* specific about it.

I'm trying to show that conventional psychologists are
unaware of the existence of controlled
variables (a fact that goes a long way towards explaining
why conventional psychologists have taken no particular
interest in PCT).

Maybe they do know but don't understand the significance it
plays in the *entire* model. Bill published in _Science_,
his book was well received. Are you sure the hangup is the
lack of knowing what a CV is? If so how are you sure. For
all the arguing and BS bruce Abbott has never once ( not
that i can recall ) that a CV is a non entity. that it
doesn't exist. Hhe has argued about how the CV is "utilized"
in the model but never denied the existance. In fact can you
show me _one_ instance where the concept of the controlled
variable was dismissed on the net.

There is no question that CV's are central to the

operation

of the control process, but other things are not any less
important ( at least to some people ). For instance;
We control any number of things at the same time. _How_

do

we prioritize which ones to attend to first.

Why would someone who knows nothing of controlled
variables be interested in how they are prioritized? How
does someone study this question if they have don't know
anything about how to determine whether or not a variable

is >under control?

Because a common belief is that perceptions control
behavior, and behavior is treated as a the controlled
variable.

Can we model that in some way?

Sure. But how do we know our model has anything to do >with

human behavior?

The same way we somehow believe that things called atoms
have something to do with matter and black holes have
something to do with outer space, and gravity and
electromagnetic fields actually exist.

We build models to account for variations in the
variables we observe. But the variations in the most
important variables we can observe in the behavior of

living >systems (controlled variables) are never reported in

conventional psychological research.

So, If *you* haven't figured out what kind of data to gather
how do you expect others to get excited about it? Knowing
*what* someone is controlling for is in itself meaningless.
We control any number of things at any one time and that is
in a continual state of change. I think *this* process of
change and prioritization needs to be discussed.

Reference levels can change by a number of different
means,
_How_ does that happen?

We have to do experiments to find out. People who don't
know about controlled variables can't do these experiments.

You know about controlled variables Where is your research
in this area? Where is anyone's research in this area?

qi is a nice neat mathematical function in the model. It

can

get pretty messy in the real world. can we begin to

explore

_how_ to model that?

We have to have data that tells us what qi is.

What *kind* of data? How do we get it? How do we know we
have it? Seems like something worth trying to model and then
see what kind of data might support the model.

What about memory? The list goes on. There is a lot of
great stuff to look at.

You betcha! But you can't look at this stuff (that is, you

can't

do the apporpriate experiments) in a way that is relevant

to

PCT if you have no idea that there are controlled variables

to

be observed.

Or if you spend the next 30 years defending waht you did 40
years ago. When do you move on? I have been on this list for
close to 5 years. The one *new* change to B:CP ( and this
happened way before I came on the list ) was a change in the
hierarchy by Gary Cziko. Has *anything* else changed since
B:CP was published 25+ years ago. Rick, Mmaybe, just maybe
if PCT were able to explain what qi was maybe others would
be more interested. Maybe if we had a better understanding
of how our reference levels changes and the kinds of effects
it has , others would become more interested.
I see you moan and groan. How about some recommendations for
the types of research that would help in these areas.

Is this list open to talking about the possibilities or

are we

restricted to talk about what Rick and Bill deem to be
essential and only that.

This list is not restricted in any way. But Bill and I tend

to

talk about things that have to do with understanding the
purposive behavior of living systems. You can't learn about
the behavior of living systems just by tinkering with

models.

Really?, Is that how you account for your fundamental
understanding of qi? What about your cherished CV's? What
are they comprised of? How did you get your understanding
of coercion? What data do you have to support your claims of
your coercion model? What experiments have you devised to
get them? Talk the talk, walk the walk.

You do have to know how the model works -- that's why Bill
is giving the tutorial on modeling control (purposive)
systems using Vensim. But in order to learn about the

things >you mentioned above (prioritizing goals,

setting references, memory, etc) you have to do _research_.

You have to see what happens to variable Y when you vary
variable X. If you do these studies without awareness that
variable Y might be under control or that variable Y might

be >a variable that affects the state of a controlled
variable, then >your conclusions about what you have found
in such an >experiment may be completely misleading.

Yes, which is why building ( or trying to ) a model and then
trying to corroborate it against some data might be a better
way to go.

I think you believe that it's possible to learn about the

things

you mention above by just playing with the model. You

can't.

As I said in a post about 8 years ago: phenomena first!

I mentioned some Phenomena, *if* they can be modeled you
*might* learn something about PCT you have never known. In
25 years you haven't been able to devise the experiments to
get the data to test all the aspects of the model. Maybe you
need to rethink how to approach it.

Models are important because they help us understand (and
predict) phenomena. But models alone (like phenomena
alone) are useless. And we can't even think about how to
model some phenomenon until we have observed the
phenomenon to be modeled: phenomena first.

OK, Here is a list of Phenomena:

I have *seen* people sitting, talking on the phone, watching
TV, and eating. The doorbell rings How does the person
react and why and what does the person do *after* the
disturbance has gone?

In the MOL when Tim is talking with a client can he identify
the *level* he is on and what level he goes to, or is this a
*different* hierarchy from HPCT?

I am going out to eat lunch with my friends. We decide the
evening before to have pizza. We go out in the street and I
see a new billboard for a new Chinese restaurant. I change
my mind ( reference level ) and decide I would prefer
Chinese to Pizza. Why did my reference level change?

Marc

14

[From Bill Powers (981016. 0341 MDT)]

Chris Cherpas (981015.1745 PT)--

Bill, has Bruce A. told you just how many of these
kinds of discussions have filled the pages of JEAB
since the 1960s? If not, I think it's a little
unfair, since I know he knows, and it's not obvious
that you would be aware of it otherwise.

No, Bruce tells me very little that could be interpreted as critical of
anything about either EAB or conventional psychology.

Best,

Bill P.

15

[From Bruce Abbott (981016.0845 EST)]

Chris Cherpas (981015.1745 PT) --

Bill, has Bruce A. told you just how many of these
kinds of discussions have filled the pages of JEAB
since the 1960s? If not, I think it's a little
unfair, since I know he knows, and it's not obvious
that you would be aware of it otherwise.

Bruce, I'm not against any quest for knowledge,
but why subject innocent bystanders to this
kind of torture?

Chris, would you be so kind as to tell me what in the world you are talking
about? The molar versus molecular debate?

Totally lost,

Bruce

16

[From Bruce Abbott (981016.1055 EST)]

Bill Powers (981015.1605 MDT) --

Bruce Abbott (981015.1140 EST)

You must have a strange new definition for control of rate of which I am
unaware. Disturbances to rate of reinforcement (produced by changing the FR
ratio) went unopposed.

That is not a disturbance of the rate of reinforcement, but of the
environmental feedback function. But I was thinking of the wrong
experiment, so it's all moot.

The experiment in question was the cyclic ratio study. Increasing or
decreasing the ratio is a kind of disturbance in that, to preserve the same
rate of reinforcement, the rat must adjust its rate of responding.

Please explain how a variable, disturbances to which
are not opposed by control action, is controlled. Rate of reinforcement
failed the Test, for godsakes.

Maybe that was not what was controlled. Not everything in the universe is a
rate.

True, but I was rebutting YOUR assertion that rate of reinforcement was
controlled. It isn't on cyclic ratio (Ettinger & Staddon), and it isn't on
ratio (Motheral). You make it sound as though it was my claim and not yours.

It was a two-phase situation. When there was no food in the cup, the rats
acted to restore the state of the cup to "food-present." When there was
food in the cup, a second control system acted to pick up and consume the
food, thus disturbing the variable being controlled by the first system.
The first system then acted to restore the cup to food-present. And so on.

Why wasn't this evidence of control? A logic-level model could be defined
this way:

reference = food in cup
perception = food/no food in cup
action = if error, press bar at some rate, possibly constant
feedback function: if number presses > n, food to cup, reset counter

This is a pretty crude control system, but judging from what we saw, it may
be appropriately crude. Clearly, varying n will not have any effect: the
system will just press the bar until the error goes to zero.

Come one, Bill, wake up. That's what I was _proposing_. _You_ said that
rate was controlled, and I offered this as an alternative.

This is what we observed during _each_ experimental session. The rat would
swarm all over the lever at first, grabbing the food and eating while
working on the lever. Then it took longer and longer to eat each pellet,
eventually sitting back on its haunches and holding the pellet while it
nibbled. We were guessing that its stomach was getting full, although its
level of nutrition was still not up to specs.

This observation was from the weight-control study, where the rats were
always on an FR-1 (CRF) schedule and could earn a _lot_ of pellets in a
short time. At higher ratios it takes longer to "fill up" and,
consequently, the deceleration of responding is lower.

However, I believe a different effect occurs when the ratio is large (or
the interval long). The rat then simply stops pressing altogether and
starts looking elsewhere for food. When it does press the bar, it presses
as fast as it can, not at a variable rate. The appearance of a relationship
between ratio and rate at large ratios comes partly from collection time
effects and partly from averaging periods of non-pressing with periods of
fast pressing. At low ratios it comes almost exclusively from
collection-time effects.

In fact we didn't observe any slow-down in rate of responding (excluding
collection time) up to the largest ratio we tested, FR-64. The changes in
average rate in which the rat spent increasing time away from the lever
occurred in weight-control study as the time-in-session and number of
pellets consumed increased.

At very high ratios, instability sets in, marked by periods of responding
alternating with periods of nonresponding, and at higher ratios yet,
responding falls to essentially baseline rates as the rat hardly ever
returns to the lever and presses. As there is no food to be found elsewhere
in the chamber, one would think that any "search for food" would eventually
be given up for the same reason that lever-pressing is. I think it more
likely that there is simply not sufficient reward for lever-pressing at the
higher ratios, given the animal's state of hunger, and so responding collapses.

If food can only be obtained by pressing the lever (no post-session
supplemental feeding), then rats continue to press the lever almost
continuously, no matter how high the ratio (at least up to the FR 1500
tested by Collier).

What the data show is that response

rate declines as the average interval length increases. That is, lower
rates of reinforcement are accompanied by lower rates of responding,
contrary to your earlier prediction, based on PCT, that the reverse would be
true.

Do you mean that the actual rate of pressing during periods of continuous
pressing varies with the interval? My impression from your earlier reports
was that the apparent change in rate of pressing was caused by collection
time effects and by spending periods of time away from the lever. Those
effects, of course, would not contradict any PCT prediction. Particularly
if there is a duty-cycle effect, practically any model would predict a
simultaneous decline (or rise) in input and output rates.

Collection-time effects become important at short ratios or very short
intervals. In the variable-interval schedules I investigated, any effect of
collection time on average response rate would have been opposite to the
changes actually observed as the average VI was lengthened.

I'd have to review the data to be sure, but my recollection is that both
changes occurred -- sustained rate of pressing declined and time spent at
the lever declined as the size of the average interreinforcement interval
increased. Reinforcement theory predicts both effects based on the reduced
rate of reinforcement, both absolutely and relative to other potential
sources of reinforcement available in the chamber. PCT predicts changes
opposite to those observed if it is assumed that the rats would increase
their response rates in an attempt to oppose the reduction in reinforcement
rate produced by the increase of average interreinforcement interval.

Regards,

Bruce

17

[From Rick Marken (981016.0900)]

Marc Abrams (9810115.0027)

Are you sure the hangup is the lack of knowing what a CV is?

Yes, I'm pretty sure. It's not knowing what a CV is _and_, thus,
not knowing of their existence.

If so how are you sure.

Because CVs are virtually _never_ the target of coventional
psychological research. They are also never mentioned in textbooks
on how to study behavior. CVs are an observable aspect of
behavior but textbooks on how to study behavior say _nothing_
about them or how to observe them.

For all the arguing and BS bruce Abbott has never once ( not
that i can recall ) that a CV is a non entity.

True. He talks about CVs on CSGNet. But he talks about them
as though they were _theoretical constructs_. He doesn't talk
about them as observable variables, in the same class as
independent and dependent variables. Most important, he doesn't
talk about them in his textbook on how to study behavior. Like
all psychologists, he acts as though controlled variables don't
exist or as though they exist only in the PCT model. In fact,
CVs are right out there in your own perceptual experience _right
now_.

Me:

We have to do experiments to find out. People who don't
know about controlled variables can't do these experiments.

Marc:

You know about controlled variables Where is your research
in this area?

It's reported in _Mind Readings_ and there are several demos
at my website. All research on control is a study of controlled
variables.

Me:

We have to have data that tells us what qi is.

Marc:

What *kind* of data?

Data like Jeff's. Data that shows evidence that a variable
(like the position of a cursor or the cost of a schedule) is
being protected from disturbance by the actions of a controller.

Rick, Mmaybe, just maybe if PCT were able to explain what
qi was maybe others would be more interested.

PCT does explain what qi is; its a controlled representation
of some aspect of the sensory input to the behaving system. The
goal of research on behavior is to discover what the qi's are
in various behavioral situations. For example, what are the
qi's involved in the behavior we see as "catching a fly ball"?
The evidence suggests that two of those qi's are lateral and
vertical optical velocity of the ball. What are the qi's involved
in the behavior we see as "tracking"? The evidence suggests that
one qi is distance from cursor to target.

I see you moan and groan. How about some recommendations for
the types of research that would help in these areas.

I'll try to think of some. That's a good suggestion.

Me:

You can't learn about the behavior of living systems just
by tinkering with models.

Marc:

Really?

Really.

Me:

Models are important because they help us understand (and
predict) phenomena. But models alone (like phenomena
alone) are useless. And we can't even think about how to
model some phenomenon until we have observed the
phenomenon to be modeled: phenomena first.

Marc:

OK, Here is a list of Phenomena:

I have *seen* people sitting, talking on the phone, watching
TV, and eating. The doorbell rings How does the person
react and why and what does the person do *after* the
disturbance has gone?

I could model this pretty easily, as control of the relationship
between doorbell ring and answering the door. But that's not the
point. What I want is observations that reveal the existence of
controlled variables. Observations like the one's Jeff is making
of control of schedule cost.

Best

Rick

···

--
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken

18

[From Tim Carey (981017.0535)]

[From Rick Marken (81015.1540)]

A great post Rick, thanks.

Tim

19

[From Rick Marken (981016.1250)]

Bruce Abbott (981016.1055 EST)--

I think it more likely that there is simply not sufficient reward
for lever-pressing at the higher ratios, given the animal's state
of hunger, and so responding collapses.

Is responding always being maintained by reward or is it only
when the animal is in a partcular state of hunger?

Any progress on getting those answers to my questions, by
the way:

1. Do you agree that controlled variables (PCT sense) are
real (observable) phenomena?

2.If so, then what are these variables called in conventional
psychology? In particular, what are these variables called
in textbooks on behavioral research (such as yours)?

Best

Rick

···

--
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken

20

From [ Marc Abrams (981916.1422) ]

Rickala, I really got to get your e-mail package. The AI in
it is tremendous. How it selectively edits posts is
extroadinary :slight_smile:

[From Rick Marken (981016.0900)]

Marc Abrams (9810115.0027)

Are you sure the hangup is the lack of knowing what a CV
is?

Yes, I'm pretty sure. It's not knowing what a CV is _and_,
thus not knowing of their existence.

_Perspectives on Personality_ 3rd Edition. Carver and
Scheier Allyn and Bacon This is a _text book_.

Pg. 503
..."What makes a feedback loop special is that the
proccesses in it permit self regulation. As indicated in the
main text, the loops _input function_ ( they italized it )
iis it's _perception_ of the way things are. The
_comparater_ checks this perception against a _reference
value_ , assessing whether the perception and reference
level are the same. If they're not the same , the result is
an _output_ ( or a change in output ). The output is aimed
at reducing the discrepency between perception and reference
value.
    The system is called a "loop" because, when the change
in output occurs, it leads to another perception, which is
then compared against the reference value...."

Yep Rick, these conventional Psychologists don't have a
_CLUE_. Do you think "output" means behavior? But since CV
is not _specifically_ mentioned they are full of cow plop,
right? Pleeez.

If so how are you sure.

Because CVs are virtually _never_ the target of coventional
psychological research. They are also never mentioned in
textbooks on how to study behavior.
CVs are an observable aspect of behavior but textbooks on
how to study behavior say _nothing_ about them or how to
observe them.

Well I guess the quote from Carver and Scheier turns this
into a bit of mush. You _really_ should look at this text
and look at the _context_ in which this is all handled. You
are _way_ off the mark about conventional Psychology. ( at
least the points presented by Carver And Scheier ) You might
( but I doubt it ) get a better idea of _what_ PCTer's
could do to interest more main stream Psychologists. ( that
is, having a better understanding of what mainstreamers are
interested in )

For all the arguing and BS bruce Abbott has never once

>>not that i can recall ) that a CV is a non entity.

True. He talks about CVs on CSGNet. But he talks about
them as though they were _theoretical constructs_.

They are. When I posed the question to you yesterday about
defining qi in terms _other_ then a mathematical function,
you explained that we needed more data. Fine, So until we
_get_ that data it's theoretical.

He doesn't talk about them as observable variables, in the
same class as independent and dependent variables. Most
important,

He can't. They are not the same thing. CV's are
_combinations_ annd could contain both independent and
dependent variables, they also could have imaginary
components. I don't see a common bridge between the data
needs. PCT needs _new_ research methods.

he doesn't talk about them in his textbook on how to study
behavior.

So what? Does that make him any less valuable to us on the
net?, or lessen his potential contribution. When I see a
paper ( let alone a book ) about what Human factor
practitioners _should_ be designing and that current methods
do not factor in PCT and are BS, I wouldn't spend my time
worrying about Bruce Abbott so much.

Like all psychologists, he acts as though controlled

variables >don't exist or as though they exist only in the
PCT model. In >fact, CVs are right out there in your own
perceptual >experience _right now_.

Yes, Just like Carver and Scheier. How silly of me to think
otherwise. Give me a break.

Me:

We have to do experiments to find out. People who don't
know about controlled variables can't do these
experiments.

Marc:

You know about controlled variables Where is your
research in this area?

It's reported in _Mind Readings_ and there are several
demos at my website. All research on control is a study of
controlled variables.

Mind Readings is terrific. It's also 10 years old. B:CP is
terrific, thats 26 years old. Again, Besides Cziko's
addition to the hierarchy what _new_ insights have been
added, modified, In the last several years? I think it's
time to try and move _ahead_. What difference does it make
what conventional psycologists think, if _we_ don't have a
clue as to _what_ kind of experiments should be done. or
what kind of data, in what kind of context is actually
needed.

Me:

We have to have data that tells us what qi is.

Marc:

What *kind* of data?

Data like Jeff's. Data that shows evidence that a variable
(like the position of a cursor or the cost of a schedule)

is

being protected from disturbance by the actions of a
controller.

Ok, we _know_ that we "protect" the position of the cursor
from disturbance. Now what?. We know control systems work.
_WHAT THE HELL IS THE NEXT DAMN STEP_. Besides beating the
crap out of every conventional pyschologist :slight_smile:

Rick, Mmaybe, just maybe if PCT were able to explain
what qi was maybe others would be more interested.

PCT does explain what qi is; its a controlled

representation

of some aspect of the sensory input to the behaving system.

DORMITIVE. What is it _composed_ of? _How_ is it composed?

I know the answer. It's composed of qo, and d. Hey,
everything is right there. why do we need any more research?

The goal of research on behavior is to discover what the

qi's >are in various behavioral situations.

Simple, they would be various qo's and d's. Am I getting the
hang of it?

I see you moan and groan. How about some
recommendations for the types of research that would help
in these areas.

I'll try to think of some. That's a good suggestion.

OK, But this is not the first time I've asked this of you.
If you can't come up with something how do you expect others
with less knowledge to do it?

Me:

You can't learn about the behavior of living systems just
by tinkering with models.

Marc:

Really?

Really.

How about answering the _rest_ of the paragraph?

Really, Is that how you account for your fundamental
understanding of qi? What about your cherished CV's? What
are they comprised of? How did you get your understanding
of coercion? What data do you have to support your claims of
your coercion model? What experiments have you devised to
get them? Talk the talk, walk the walk.

Me:

Models are important because they help us understand
(and predict) phenomena. But models alone (like
phenomena alone) are useless. And we can't even think
about how to model some phenomenon until we have
observed the phenomenon to be modeled: phenomena >>first.

You mean like Physicists?

Marc:

OK, Here is a list of Phenomena:

I have *seen* people sitting, talking on the phone,
watching TV, and eating. The doorbell rings How does the
person react and why and what does the person do *after*
the disturbance has gone?

I could model this pretty easily, as control of the

relationship

between doorbell ring and answering the door.

What about all the _other_ stuff thats going on? How do you
know _if_ the person would even be controlling for the
doorbell?

What about my other questions

In the MOL when Tim is talking with a client can he identify
the *level* he is on and what level he goes to, or is this a
*different* hierarchy from HPCT?

I am going out to eat lunch with my friends. We decide the
evening before to have pizza. We go out in the street and I
see a new billboard for a new Chinese restaurant. I change
my mind ( reference level ) and decide I would prefer
Chinese to Pizza. Why did my reference level change?

But that's not the point. What I want is observations that
reveal the existence of controlled variables. Observations
like the one's Jeff is making of control of schedule cost.

To quote an eminent Control Theorist (Marken 981016.0900)

"CVs are right out there in your own perceptual experience
_right now_".

Marc