What is control; rat expts; PCT & OpCond; Bowlby

[From Bill Powers (951026.0530 MDT)]

Hans Blom (951026) --

     Let's take the classical example: a living room thermostat
     "controls" the room temperature despite fluctuations in, amongst
     other things, the outside temperature. But the outside temperature
     "controls" the burner's on/off cycle ratio, other things being
     equal.

The answer to the second (erroneous) assertion is obvious from the Test.
How would you apply the Test to support your assertion that the outside
temperature is capable of control, and that the variable it controls is
the furnace's on-off cycle? Aren't you applying the S-R criterion for
determining cause and effect instead of the PCT criterion for
determining control?

     We, educated in control theory, have a deeper insight into
     differences between these two modes of "control". Many discussions
     have been fought about terminology, e.g. "control" vs. "influence".
     But my point was that in a "causal loop" it is not fruitful to view
     one variable as independent and another one as dependent. But if
     you do so, it is not important _which_ one you take as the
     independent variable.

In a thermostat controlling room temperature, there is no causal loop
that includes the outside temperature. I agree that in the causal loop
that does exist, which includes the _room_ temperature, there is no
dependent or independent variable. The independent variables are the
reference setting of the thermostat and, among other disturbances, the
outside temperature, the solar input, and the wind velocity.

     And by only focussing on PCT's "causality" one runs the risk of
     missing the great extent to which people's actions are determined
     by changes in the environment. Seen this way, reinforcement theory
     is PCT's other side of the coin. Both formally and factually :-).

Where have you been, Hans? The relationship between actions and
disturbances is one of the main points we make in PCT. We even say that
this relationship is probably the origin of S-R theory -- mistaking an
independent disturbance for a stimulus. Do you understand how a
disturbing variable can seem to be a stimulus even though it is not
actually sensed?

Reinforcements are part of a control loop; they are not independent
changes in the environment.

···

--------------------------------
[in Blom (951026 to Nevin]

     Take the distinction between "control" and "influence", for
     instance. Do I "control" my perceptions or merely "influence" them?
     Maybe the latter, since I observe that I am frequently content with
     realizing my goals only approximately, thus not removing _all_ of
     the disturbances. So these terms are fuzzy as well.

The fuzziness of which you speak here is simply a matter of loop gain.
In a control system with a gain of 1, disturbances will have an
influence on the controlled variable equal to the influence of the
system's reference signal. If the loop gain is 100, a disturbance will
have 1 unit of effect and the reference signal 99 units, given equal
changes in disturbance and reference signal. It is not the terminology
of PCT that is fuzzy, but certain non-PCT ways of thinking about
control.

     Yet that doesn't matter much. We're still able to communicate. We
     know sufficiently well what we mean when we use the terms "control"
     and "influence". They may apply equally in a certain case, yet one
     draws our attention to the fact that we realize a desire, the other
     to the fact that there are other factors as well that govern how we
     act. But because of the fuzziness of concepts we will probably
     never be able to truly "understand" each other in all details.

That's a nice fuzzy generalization, but you miss the point that the
difference between influence and control is very precisely and
quantitatively defined in PCT (Bill Leach reminded me that this needs
saying). Influence is open-loop; control is closed-loop. By your
insistence that both open-loop and closed-loop systems control, you
create fuzziness where it is not necessary, as in your example of the
thermostat above.

We are not communicating if each time I use the word "control" you
translate it into "influence."
-----------------------------------------------------------------------
Bruce Abbott (951025.1710) to Phil Runkel --

     Phil, Rick was just having a bad day, having been roused from net
     hibernation before he had gotten enough sleep. What my analysis
     showed was that the rats in the experiment to which Rick refers
     were not varying their response rates across different ratio
     schedule requirements, although this meant that at higher ratios
     the rats would earn less food owing to the extra time required to
     complete the ratio. Thus the rats failed to control food rate by
     compensatory changes in response rate. But Rick believes that the
     rats SHOULD have controlled food rate; they didn't, so they must
     have been up against some limit that prevented them from adjusting
     their response rate. Their control system must have been bang up
     against the stops, i.e., the rats must have been responding as fast
     as they could and thus could not increase their response rates any
     further to counter the disturbance. And if the system had been
     driven into saturation, it must have been overdriven by a huge
     error signal; thus the rats must have been starving.

I agree with Rick. The rats should have controlled food rate. We know
that at some long-term level of intake, bar-pressing behavior would
essentially disappear, so there is clearly a reference level for food
intake (defined as that level of food intake at which the behavioral
output would just drop to zero). The question is where the transition
from high to low pressing rate takes place.

What your analysis seemed to show was that we had misidentified the
controlled variable: it was not the short-term rate at which food
pellets were received (my initial assumption). The error driving the
rate of pressing appears to derive from a variable that is more
indirectly, and more slowly, affected by food intake averaged over a
longer period -- perhaps body weight, perhaps some other long-term
effect of the food intake.

However, if you recall, your analysis was not completely certain. The
constancy of pressing-rate was only approximate, and by assuming
different collection times we could create a curve that showed pressing
rate increasing with (presumed) error for small drops in reinforcement
rate, reaching a maximum for larger ratios. Your method of fitting a
straight line to the data gave a very large weight to behavior at the
highest ratios, whereas the curvature of which I speak was seen only for
two or three of the lowest ratios, squeezed in near the origin of the
plot. A log fit might have been more appropriate. It is still possible
that pressing rate does increase (very rapidly) with ratio, topping out
at ratios of 10 or so (although that could well vary across rats).

The only way to be sure of these results is to record the bar-presses
individually so we can actually measure the pressing rate and
colledction time, rather than making assumptions. When we re-do these
experiments, we can also vary the pellet size or number delivered on
each completion of a ratio; if there is a proportional effect, it may
not become visible until the amount of food delivered per press has
become large enough. It is possible that even at a 1:1 ratio, the rate
of food intake is far below the reference level being set at that level
of deprivation.

I hope that we can get started some time soonish on gathering the data.
What I expect is that we will see, as you have already suggested, a two-
level system with the lower level behaving relative to a reference level
set by the higher system. But if anything in the lower-level system
saturates, we have to do some fiddling to find the range of control. If
something is being driven out of limits, we will get strange results.

Your generalization, "Thus the rats failed to control food rate by
compensatory changes in response rate", is premature. We don't know that
yet.
-----------------------------------------------------------------------
Chris Cherpas (951017.1113 Pacific) --

      Behavior as a means to an end sounds like operant behavior to me.

Right. Operant behavior is control behavior.

     Even perceptual behavior (e.g., see Skinner's _About Behaviorism_)
     can occur in the absence of the thing seen because of high
     deprivation as when we "see" a friend in a crowd whom we miss --
     according to (radical) behaviorism.

Different subject; see "imagination" in B:CP.

     Unfortunately, when we talk about a "stimulus," and this is
     particularly true in a discussion of "reinforcement", it is not
     always clear whether this is meant to be from the observer's
     ("procedural") or the subject's ("effective") perspective. From my
     understanding, the radical behaviorist assumes that "effective"
     stimuli always involve perceptual behavior, and this includes the
     perception of the reinforcing "stimulus."

"Perceptual behavior", to my mind, stretches the meaning of "behavior"
to the point where the term loses its usefulness. If you call
_everything that happens_ "behavior", you lose the ability to
distinguish between sensing and acting. The main trick in system
analysis is to break the system down into components whose properties
can be distinguished from the properties of other components, but not to
the level where the analysis becomes impossibly complex. It is possible
to go too far in the other direction, where you lose the distinctions
between the components; that makes a system model impossible, too.

     In the behaviorist literature, numerous analyses are made with
     respect to feedback functions, such that, for example, a certain
     rate of key-pecking (the "independent variable") controls a given
     rate of reinforcement (from a procedural perspective, but, in this
     context, the "dependent variable") on a given schedule of
     reinforcement. People who like optimization models have made
     elaborate analyses of schedule performance using such functions.
     An instantiation of "melioration" theory assumes that the variable
     being controlled is local reinforcement rates; whenever a
     difference is detected across alternatives, relatively more
     behavior shifts to the locally more reinforcing alternative. When
     all alternatives are experienced as equally valuable, then
     behavioral allocation is stable.

Are you doing some translating into PCT terms here? In all the analyses
I have seen (not many), the allocations are represented in terms of a
single system equation, as in Herrnstein's matching law. I have never
seen a second equation used, representing the connection from
reinforcment to behavior (via the organism) and solved simultaneously
with the equations representing the schedule effects. All the "elaborate
analyses" I have seen are simply manipulations of the equations
describing the external world. I have never seen anyone acknowlege that
you can't uniquely solve a system of equations in which the number of
equations is less than the number of variables.

     There are places, however, as in verbal behavior, where the
     behavior of a listener is necessarily what the speaker's behavior
     controls. In this case the difference between what the
     "experimenter" sees as relevant can be quite close to what the
     verbal behaver is controlling.

How do you know? The point of a formal method like the Test is to allow
finding out what a person is controlling without having to ask the
person (where possible).
-----------------------------------------------------------------------
Bruce Abbott (951025.2005 EST) --

     Although at one level I can appreciate your feeling, on another I
     can't help but think that you've skimmed over much of what Bowlby
     had to say in the middle chapters and that this is why you are
     offering vague generalities in the above paragraph--you could have
     said these things without reading anything at all.

True, I could have.

     In particular you seemed to have missed some significant passages
     that might change your view of what Bowlby did and did not
     understand about control systems.

I read the passages you cite, and they are one reason for my saying that
Bowlby understood more that is consistent with PCT than anyone else I
have read. However, when the subject turns to his clinical field of
interest, these insights into control do not seem to be applied. This is
why I turned to his 1988 book, _A Secure Base_, to see what came of his
earlier thoughts about control theory. Not knowing what others in his
field think is the right way to approach the same phenomena, I can't
judge how much his analysis is being influenced behind the scenes by his
awareness of control theory, but what he _says_ about it in this later
book is primarily the same TOTE-like idea: some conditions initiate
behaviors or behavior sequences, and others terminate them. I recognize
that as a PCT-like idea, but did Bowlby see the same control process as
we see it?

One important thesis is in the earlier books and the later one as well:

     The distinction I have drawn between the function served by a
     certain form of behavior and our knowledge of, and our striving to
     reach, the conditions that will terminate that behavior is one of
     the criteria that distinguish the biological realm from the
     psychological (A Secure Base, p. 64).

Note that "the conditions that will terminate the behavior" is the same
as the PCT definition of a reference level, although there is no sign
that Bowlby took this idea any further.

Here he distingushes between what I would call learned control systems
and intrinsic ones. However, he doesn't exactly make the distinction
that way. He is saying that when the behavioral systems seek a goal, the
effects of achieving it ("terminating the behavior") are to be judged in
terms of the biological function served by attaining that goal, not by
the goal itself. This is similar to my concept of a reorganizing system,
in that the reason for acquiring any behavioral control system is
ultimately to maintain certain internal critical variables near their
inherited reference levels. But it differs from my concept in a
fundamental way.

Bowlby does not cast the deeper reason in terms of more control systems;
he is saying that the reason for existence of a behavioral goal is its
biological function AS THE BIOLOGIST SEES IT. For example, we have the
goal of eating food containing vitamin D because without vitamin D we
will develop a vitamin deficiency disease and become less able to
reproduce ourselves. This puts the "because" into the realm of objective
facts about the organism, as observed from the viewpoint of an
independent observer, and throws the burden onto evolution, the catch-
all explanation.

In PCT we don't use such "becauses;" what the external observer knows
about the organism has no influence on how it works. It may be true that
if the organism doesn't eat, it will die, but the organism doesn't know
that, and this fact about not eating is not what makes the organism eat.
What makes the organism eat is a reference signal for a state of
nutrition, being compared against a perception of the actual state of
nutrition, or something like that -- not an abstract fact. Organisms
lacking this intrinsic control system will not eat, and sure enough,
they will die. But if they eat, and live, it is not because they will
die if they don't eat, but because they contain this control system.

The way biologists speak of functions bypasses the means by which these
functions are accomplished. This creates a very peculiar mode of
explanation, in which the consequences of a certain kind of behavior are
offered as a direct explanation for why that behavior occurs. The
biologist says that the goose pulls the egg back into its nest because
this improves the chances of survival of the species. That's the
functional explanation of this behavior. However, although it is true
that if geese do not pull stray eggs back into the nest the survival of
the species will be adversely affected to some degree, it is not true
that they retrieve the eggs _because_ doing so will promote survival.
They retrieve the eggs because of the operation of a specific control
process inside of them. As a consequence of the existence of this
control system, the species which commonly contains it will have a
better chance of survival; the effect on survival is a consequence of
the operation of this control system, not a cause.

When future consequences of behavior are cited as causes of the behavior
that leads to them, one is really talking about a control process. But
one has to be careful that in doing so, one is not assigning the
consequence the role of a reference signal. The consequence follows from
behavior, but the reference signal does not. The reference signal is a
prerequisite for reaching the goal-state.

Furthermore, as we have been saying recently on the net, not every
consequence of behavior that we can notice is necessarily a controlled
variable. Geese that control the positions of their eggs certainly
control the positions of their eggs, and that is a controlled
consequence of their behavior. But the same behavior has an uncontrolled
side-effect, which is to improve the chances that there will be a
hatchling from the egg, and as another consequence that the population
of geese will be higher than it would be if the position of the egg was
not controlled. Those side-effects are not under control by the goose at
any level of its organization. They are not _reasons for_ its control
behavior, but _consequences of_ its control behavior.

So biological functional explanations are really not explanations at
all; they are simply observations of consequences. The wolf marks its
territory, and other wolves stay away (sometimes). The stickleback darts
at an object with a red mark on it, and the object moves away. The chick
begins to follow a large warm object, and follows that object to food or
water or safety. But as far as research can prove to date, the wolf does
not mark its territory IN ORDER TO drive other wolves away; the
stickleback does not dart at the red mark IN ORDER TO drive a rival
away; the chick does not follow the mother IN ORDER TO get food, water,
and safety. What follows "IN ORDER TO" is simply a consequence of the
observed behavior -- until we can establish that these consequences are
_intended by the organism_. With the Test, we can do that, or else prove
that the consequence is merely a side-effect of controlling something
else.

The mistake that the functional explanation makes is to assume that ALL
important consequences of behavior are intended; that they all work like
reference signals. This is clearly not true, and can't be assumed _a
priori_. It is certainly true that maintaining a body temperature of 37
C makes our biochemical processes run at a favorable rate, but it is not
true that we maintain this temperature _because_ it does this. As far as
I know, we have no sensors that can detect the rate of a biochemical
reaction, and no reference levels for such rates, either.

So I disagree, on PCT grounds, with Bowlby's distinction. I say it's
control all the way down, and that functional analysis is irrelevant in
explanations of behavior.

However, I agree that Bowlby had many of the critical insights about
control, and given a longer research career, might well have ended up
with PCT. Or is he still alive and active?
-----------------------------------------------------------------------
David Wolsk --

Thanks for the vote of confidence. What experiences did you have with
your colleagues, in telling them about your opinions of control theory?
-----------------------------------------------------------------------
Best to all,

Bill P.

[francisco arocha, 95/10/26 10.32 EST]

[From Bill Powers (951026.0530 MDT)]

Hans Blom (951026) --

      And by only focussing on PCT's "causality" one runs the risk of
      missing the great extent to which people's actions are determined
      by changes in the environment. Seen this way, reinforcement theory
      is PCT's other side of the coin. Both formally and factually :-).

Where have you been, Hans? The relationship between actions and
disturbances is one of the main points we make in PCT. We even say that
this relationship is probably the origin of S-R theory -- mistaking an
independent disturbance for a stimulus. Do you understand how a
disturbing variable can seem to be a stimulus even though it is not
actually sensed?

How coud have missed this one? So according to Hans, people's actions are
determined by the environment to a great extent. This is S-R all the way.
According to PCT, people's actions are not under control. Perceptual
consequences of actions are. Actions have to vary to counteract
disturbances. This is not simply a control theory assertion; it is based on
the study of actual behaving systems (people, rats). It is a factual
statement.

Control theory is a formal system general enough that you can apply it to
systems made of different stuff. Having an understanding of control theory
is not enough to have produced PCT -- necessay but sufficient. For this, it
is required that the formalism of control theory be given a factual
interpretation. This factual interpretation is committed to the type of
system (e.g., the system must have a sensor of some kind) and the
interpretation must correspond to the way the system (a living system) is
organized. Other interpretations of PCT exist, but they have misidentified
the correspondence between the elements of the theory and the actual, real
parts of the living system, for instance interpreting the reference signal
as an input.

All this to simply say what I said before: We should not forget the
distinction between the formal aspects of the theory -- formulas,
equations; the factual interpretation of the theory (as a psychological
theory, which it is), and the fact of control (as Rick marken said in hes
Behavioral Science paper). The first one is used to express the second one
which is based on the third one.

Cheers,

francisco

j. francisco arocha Tel: (514) 398-4985
1110 Pine Avenue W. Fax (514) 398-7246
Cognitive Studies in Medicine
Centre for Medical Education E-mail: francisco@medcor.mcgill.ca
McGill University (alt) email: cybn@musica.mcgill.ca
Montreal, QC H3A 1A3
Canada

<[Bill Leach 951026.19:41 U.S. Eastern Time Zone]

[Bill Powers (951026.0530 MDT)]

If Dag is listening, the referenced posting is a candidate for the "gems"
list.

Bill that was a fine summary of the issues, what is important and what is
not, for a number of discussions!

-bill

i.kurtzer (951025.1830)

again i agree with bill almost point for point, but there was/is one
minor point that i feel should be made. those who remember my post on
the definition of control as an theory-free empirical fact should
see the connection.

Powers:

--------------------------------
[in Blom (951026 to Nevin]

     Take the distinction between "control" and "influence", for
     instance. Do I "control" my perceptions or merely "influence" them?
     Maybe the latter, since I observe that I am frequently content with
     realizing my goals only approximately, thus not removing _all_ of
     the disturbances. So these terms are fuzzy as well.

The fuzziness of which you speak here is simply a matter of loop gain.
In a control system with a gain of 1, disturbances will have an
influence on the controlled variable equal to the influence of the
system's reference signal. If the loop gain is 100, a disturbance will
have 1 unit of effect and the reference signal 99 units, given equal
changes in disturbance and reference signal. It is not the terminology
of PCT that is fuzzy, but certain non-PCT ways of thinking about
control.

i can't be more emphatic on this point. i should not allude to loopgain
or any other system property in dermining the fact of control unless you
yourself had built that control system. the fact of control is an empirical
fact that has loop gain as its most obvious system analogue. the
difference between influence and control should not refer to differing
charcteristics of systems; whether they are open or closed loop, or if
they are closed loop what is the loop gain of the system BUT based on the
fact that some portion of the world remaining immobile in some state(s)
while bombarded by forces that (except for one--the "system"--that for the
observation can be hidden) should remove it from that state. that the
criteria should 1/10 of the expected deviation or 1/100th is i simply
don't know, but i do know it should not be couched in terms of a
theoritic postulation.

control as a process should obviously include the "system" but to point
to some fact and say this piece of the world is controlled should not
refer to anything excepting that it does just that. this does not create
tautolgy but makes explicit what is and what is not what is to be explained.
loop gain is not what is to be explained; it is an explanation. CONTROL
IS NOT AN EXPLANTION; IT IS TO BE EXPLAINED.

fervently
i.

<[Bill Leach 951027.06:02 U.S. Eastern Time Zone]

i.kurtzer (951025.1830)

Hummm, I don't know if you are going to view this as good or bad but once
again Isaac, I have to congratulate you upon stating the most basic issue
clearly and concisely.

Your last paragraph contains the gem to which I refer and truely is the
essence of what PCT is about. PCT does not "assert" that living systems
are control systems. PCT "asserts" that if living systems are control
systems then control theory applies as that is the only known theory that
is capable of explaining control behaviour. Experience with the theory
has taught us a means of determining if control exists or not (what we
call THE TEST).

If an observed phenomenon fails THE TEST, then (assuming that THE TEST
was applied properly) the phenomenon observed is not a control phenomenon
and control theory does not apply. Further effort may be "interesting"
and can well be science but they are not science in the domain of control
theory.

-bill