# Determined by error, Learning
**URL:** http://discourse.iapct.org/t/determined-by-error-learning/3453
**Category:** CSG1994
**Created:** [December 22, 1994, 10:19pm UTC](http://discourse.iapct.org/t/determined-by-error-learning/3453 "1994-12-22T22:19:28Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@Marken\_Richard\_S](http://discourse.iapct.org/u/Marken_Richard_S)
#### Post date: [December 22, 1994, 10:19pm UTC](http://discourse.iapct.org/t/determined-by-error-learning/3453/1 "1994-12-22T22:19:28Z")
[From Rick Marken (941222.1415)]
Bruce Abbott (941222.1230 EST) --
I'll let Tom Bourbon (941222.0919) (and Bill Powers, when he returns) explain
why natural selection is not selection BY consequences. I want to stick with
just the simplest examples of control we can find.
You say:
> I view the "consequence" AS the effect of the behavior on the error signal,
> which IS a change in the difference between consequences [as you define
> them] and reference signal.
If "select" means "determine", "consequence" means "error signal" and
"behavior" means "output" then I agree that "behavior is selected by its
consequences". All you are saying is that:
o = f(r-p)
Output is determined by the error signal, which is a pretty true statement
about a control loop. Since the error signal (r- p) is a consequence of
(influenced by) the output, then behavior (o) is selected by a consequence
(error signal) of itself.
My beef with "selection by consequences" only exists if the consequences
doing the "selecting" (influencing) are the objective results of output or
the perceptual representation of those results. If you really think of the
"selecting" consequence as "error signal" and behavior as "output" then I can
live with the idea that behavior is selected by its consequences. I thought
the behaviorists were talking about consequences like "food pellets" or
"brain stimulation" and such. If you're not talking about that, then your
notion of "selection by consequences" is quite different (and a LOT better)
than that of Skinner and the rest of the Behaviorists.
If the "error signal" is the consequence that selects behavior, then in order
to know what's doing the selecting, you have to determine 1) what perceptual
variable the organism is controlling and 2) the (possibly varying) reference
setting for that variable. You can't tell that a consequence like "error
signal" is selecting outputs just by looking at the objective consequences of
the organism's behavior; you have to do The Test for the controlled variable
to determine what variable is being controlled and at what level.
I have been very high gain about the "selection by consequences" issue
because it has methodological implications. If you believe in the
Behaviorist's version of "selection by consequences" then one of the aims of
your research is to find the consequences that select behavior. This is a
hugh waste of time because consequences (in the Behaviorist sense) don't
select behavior; it is the difference between perceived aspects of those
consequences and the desired state of those perceptions that continuously
determines the outputs that drive those perceptions to their desired levels.
Since you understand that the consequences that "select" outputs are the
values of the error signal, why don't you get out there and start doing the
kind of research we need done in order to understand what constitutes the
error signals in living control systems. You do this by testing for
controlled perceptual variables and the reference states of those variables.
This is PCT research and it will provide the kind of information we need
to do PCT modelling -- information about what states of the world organisms
consider "right" and what states they consider "wrong". Once you have
this data, you can you test (via modelling) to see whether outputs are
selected (determined) by error.
Me:
> This will seem puzzling until you learn how a control system works. The
> best way to do that is by running simulations of control systems
Bruce:
> Doesn't it strike you as odd that someone can WRITE simulations of
> perceptual control systems without having a clue as to how they work?
No. We have had one or two control engineers on this net -- people who BUILD
control systems -- who were basically clueless about the aspects of the
operation of a control system that are relevant to understanding living
systems. For example, one of these engineers completely rejected the notion
that a control system controls a perceptual signal. Failure to understand
this did not mean that this engineer could not build control systems -- and
good ones too. The engineer sees the control problem from a differnet
perspective than does the PCTer; the engineer builds a control system to
control some "objective" (from the engineer's perspective) variable in the
environment. The PCTer, on the other hand, has to figure out what variable(s)
an existing system controls; so the PCTer has to understand that the variable
is determined by the perceptual functions in the control system itself;
control systems don't control what the observer perceives; they control their
own perceptions.
> [Probably not. I'm sure you will find an explanation that allows you to keep
> up your perception that I do not understand them. It seems to be a highly-
> valued reference level for you and thus strongly defended.]
Control systems don't defend reference levels; they defend perceptions,
maintaining them at the reference levels specified by reference signals. I do
have a reference to perceive "understanding of PCT" at a particular level,
and it is important for me to control that percpetion (it's a high gain
control system). C'est la vie.
Me:
> I can't think of any sense in which control could be described as "feedback
> regulated"; if anything, control REGULATES FEEDBACK, where feedback is the
> perceptual consequence of the \>controller's own output.
Bruce:
> Ah, more sand in the gears.
Felt like "clarity" to me;-)
> This is a common description of the process, in which "feedback-regulated"
> indicates the TYPE of control.
It's common with people who don't understand how control systems work. Why
continue to use their muddled terminology?
> Although in PCT, "control" apparently refers to one, and only one, process
> (thus, no need for the qualifier), in other contexts it also refers to open-
> loop situations,
Yes. And the only reason that ambiguity exists is because in these "other
contexts" people don't understand the nature of the phenomenon of control. If
they did, they would realize that the term "open-loop control" is, at best,
a lousy way to decribe on ordinary causal relationship between variables and,
at worst, an oxymoron. Once you understand PCT, you will find these seemingly
innocent terminological conventions to be quite grating.
Martin Taylor (941222 11:10) --
> it would be nice if the two of you could at least agree to talk about the
> same thing, whether it be the behaviour of the mature organism, the
> reorganization within the organism that allows it to mature, or the
> evolution that allows purposeful organisms to come into existence.
A large part of the problem results from the fact that conventional
psychology has defined behavior in a way that makes it difficult to
distinguish ordinary control from learning. I seem to recall "behavior"
defined as "what organsms do" and "learning" as "a change in behavior that
results from practive or experience". Well, how do you know whether a change
in what an animals does is the result of "practice and experience"? In fact,
just about anything that looks like a "change" in behavior is considered
"learning" in psychology texts. This includes, for example, nearly all the
operant "conditioning" data. Changes in behavior on different schedules,
changes in behavior in different "two key" conditions, etc. etc are all
counted as "learning" phenomena; a good, hefty portion of them are just
steady steady control phenomena; the control we see must be the RESULT of
some learning process; but we are not seeing the learning process itself;
just good old control.
In PCT, we can explain skilled controlling extemely accurately; we know
a lot about the control loops that accomplish this kind of behavior. The
"learning" question is "how do those control loops change in order to
maintain control in a changed environment?" To study this, I presume you
would change the environment is a way that requires a change in the control
system if control is to be maintained -- and you monitor characteristics of
the control system to see how they change during this time. This would be
the way I would go about the study of "learning" from a PCT perspective. It
would not be easy to do this research; but it seems to me that we have to
have learning data of this sort before we can start developing plusible ideas
about how learning works.
Knowing what we do know about how control works, the models Bruce presented
as "learning" models are NOT learning models at all. Tom Bourbon and Bill
Powers have developed simulations of REAL PCT learning models -- models that
illustrate the PRINCIPLE of learning (called reorganization) in PCT.
Best
Rick
---
### Author:  [@\_Martin\_Taylor](http://discourse.iapct.org/u/_Martin_Taylor)
#### Post date: [December 22, 1994, 11:00pm UTC](http://discourse.iapct.org/t/determined-by-error-learning/3453/2 "1994-12-22T23:00:26Z")
[Martin Taylor 941222 17:30]
> Rick Marken (941222.1415)
> A large part of the problem results from the fact that conventional
> psychology has defined behavior in a way that makes it difficult to
> distinguish ordinary control from learning. I seem to recall "behavior"
> defined as "what organsms do" and "learning" as "a change in behavior that
> results from practive or experience". Well, how do you know whether a change
> in what an animals does is the result of "practice and experience"? In fact,
> just about anything that looks like a "change" in behavior is considered
> "learning" in psychology texts. This includes, for example, nearly all the
> operant "conditioning" data. Changes in behavior on different schedules,
> changes in behavior in different "two key" conditions, etc. etc are all
> counted as "learning" phenomena; a good, hefty portion of them are just
> steady steady control phenomena; the control we see must be the RESULT of
> some learning process; but we are not seeing the learning process itself;
> just good old control.
That's a very fair comment. But I don't think it is connected with my beef
to which it was a response:
> > it would be nice if the two of you could at least agree to talk about the
> > same thing, whether it be the behaviour of the mature organism, the
> > reorganization within the organism that allows it to mature, or the
> > evolution that allows purposeful organisms to come into existence.
All of these, and all Bruce's simulations, have to do with systems whose
internals the analyst/theorist can see (or imagine). The issues you
have not been agreeing to discuss have been unrelated to what most
psychologists think about learning in real animals. They have been
about what happens in particular simple models, where it is (or should
be) much clearer what constitutes control by a single level controller,
what constitutes changes in the structure or parameters of a control
system (of one or more control units), and what constitutes the birth
and death of individual control systems.
You KNOW, you don't guess, whether your simulation program changes the
structure or the parameters of the control systems it incorporates.
There may be room for argument as to whether the changes implemented
in the program constitute control by another (simulated) reorganizing
control system. But there's no room for argument whether or not the
executive control systems change--whether there IS reorganization in
the simulation.
> Knowing what we do know about how control works, the models Bruce presented
> as "learning" models are NOT learning models at all.
Don't parameters in the control systems change during the simulation runs?
If they do, then the models ARE learning models. That in no way means
that they exemplify selection by consequences (though I think the evolutionary
one does). If the parameters of the models don't change, then there is
no learning.
There is a grey area, exemplified by Tom's version of adaptive control,
in which the parameters (gain) of an executive control system are modified
by another control system. Should we say that the joint system does not
learn, or should we say that the executive system learns and that the
controller that acts upon it is the mechanism by which it learns? Would
changes in the parameters of the modifying control system constitute
"learning to learn?" The answer, I think, is a question of choosing a
dictionary definition so that the choice leads to maximum convenience in
further discussion. My preference, I think, would be to say that
learning occurs whenever changes happen in the parameters or structure
of a control system or a set of connected control systems. Then the
executive control system in Tom's setup "learns" but the total system
does not "learn to learn." This usage seems to be workable, but maybe
something else is better.
Whether a change in actions is occasioned by control in a fixed control
system, by reorganization (control by a system in which the state of
another control system constitutes the environment), or by evolution
(the total substitution of one control system for another), should be
clear. Which effect (of the first two) is occurring in a living system
may well be less clear, and is a reasonable topic for discussion. But it
seems irrelevant to the cross-purposes interchanges that you and Bruce have
been conducting.
[I had a mail hiatus because of a disk problem that started around noon
yesterday and was found this afternoon. There may be mail items sent here
in that period that go lost, but some things did get through so maybe
everything did. If I don't answer something that was sent then, you
might send it again, just in case. And Rick, I did get your offer to
write by Feb 1 a paper on using PCT in human factors, for which I thank you].
Happy Xmas.
Martin