[From Rick Marken (941221.2300)]
Bruce Abbott (941221.1030 EST) --
I am puzzled by your assertion that
"Selection BY consequences" exists BUT IT DOES NOT PRODUCE CONTROL.
In reorganization, do not the consequences of a set of system parameters
(control or failure to control) determine whether those parameters will be
preserved in the system or abandoned in favor of yet other variations?
No. The consequences do NOT determine it. The DIFFERENCE between consequences
and REFERENCE signal determine it. Because reorganization is a closed loop
process, what is actually happening is that the reorganizing system is
determining the consequences (by varying parameters); the consequences do not
determine the parameters. Strange but true in the wonderful world of circular
causality.
See my diagram of the reorganizing system in my previous post. The
reorganizing system is a control system, controlling intrinsic perceptual
variables (which are among the consequences of the activities of the
perceptual control hierarchy). Consequences don't select anything in a
control loop -- because they are a variable in a loop. Rather, the
_perceptual_ consequence of action in the loop -- the consequence that
is compared to the reference signal -- IS controlled in a control loop.
This will seem puzzling until you learn how a control system works. The
best way to do that is by running simulations of control systems (Simcon
is a particularly good "control loop" development environment) and watching
the dynamic behavior of all the variables in the loop.
Me:
Are you saying that this new demo shows that control (purposeful behavior)
can result from selection by consequences?
Bruce:
Yes, of course!
Ouch. Wince. It looks like you're still standing at the point where
the two roads diverge in a wood-- and you keep wanting to take the road
more travelled by (and stumbled down) ;-).
Me:
If so, what is the controlled variable?
Bruce:
Stored nutrient level ("fuel").
For individual organisms? How can the survival or non-survival of
individual organisms be a consequence that selects the controlling
done by these organisms? Where is the control system that is
controlling this variable? Please provide a diagram.
Me:
How do you know that it is under control?
Bruce:
It stays near its reference level despite disturbances (fuel consumption and
variable nutrient density).
In an individual organism? If so, doesn't it seem like the control system
is inside the organism itself? Diagram please, showing controlled variable,
disturbance, outputs affecting controlled variable, perceptual representation
of controlled variable, comparator and error. Thank you.
I think that throughout this debate, you have been repeatedly describing to
me the operation of intact control systems and I have been repeatedly trying
to get through to you that I am NOT discussing how intact control systems
behave but rather, how such systems are created as a result of experience
with the consequences of a given system's attempts to control some
perception. Until you are able to grasp the difference, we will continue to
talk past one another.
Again, this learning thing is a red-herring. Before we start talking about
how control systems "are created as a result of experience" we have to under-
stand how control systems operate. Can we agree that the controlling done
by a control system that has already been "created" (like a grown-up E. coli)
does not involve "selection BY consequences"? In other words, do you
agree that the behavior of a rat pressing a bar for food in a Skinner
box -- a rat that has already learned to do this -- involves the
selection (by the rat) OF the consequences it wants; do you agree that
the rat's bar-pressing is NOT selected BY its consequences (the
delivery of food pellets)? Do you agree that the controlling done by the
rat does not involve selection by consequences? If so, we can start
to talk about learning -- which also doesn't involve selection by
consequences; but first things first.
I agree with you that purposeful behavior is produced by a set of perceptual
control systems rather than by a simple "selection by consequences"
reinforcement mechanism.
Hooray! So you agree that purposeful behavior (control) does NOT involve
selection BY consequences. What was all that stuff up there about purpose-
ful behavior resulting from selection by consequences?
The e. coli demos show that purposive behavior is best understood in terms of
perceptual control systems.
Close. Not "best understood"; ONLY understood. Only the control system model
produced the controlling seen in the E. coli demo; all other models failed.
My ECOLI4a demo showed that the parameters of a successful control system
can be established through a process in which the organism varies its
control parameters based on the immediate consequences of
its own behavior.
The ECOLI4a model was a control system: the parameters of this control
system, p(tumble|up the gradient) and p(tumble|down the gradient) were
continuously changed by the consequences of behavior. These parameters
remained near values that produced controt (p(t|d)>>p(t|u)) as long
as the result of a tumble when going up the gradient was most likely to
be a punishment and the result of a tumble when going down the gradient
was most likely to be a reward. When you change the likelihood of these
results, p(t|d and p(t|u) end up at values that produce poor or NO control
(as when p(t|d) becomes equal to p(t|u)). Real control systems do NOT lose
control when the environment changes in this way, as was demonstrated by the
control model whose parameters are not at the mercy of consequences and by
human subjects. In fact, that's why control systems are called CONTROL
SYSTEMS; they control the environment; a control system tells the environment
(actually a perceptual representation thereof) what to do; in ECOLI4, the
environment told E. coli what to do.
You can call the changes in p(t|d) and p(t|u) made in Ecoli4 "learning" all
you like. But as I said before, this is the most dysfunctional type of learn-
ing I can imagine. I thought leaning was supposed to adapt an organism to
changes in the environment. Your "learning" system results in an adaptive
organism (one that gets up the gradient to the target) ONLY in a single
environmental situation. If the environment changes so that rewards and
punishments are nearly equally likely after a tumble when going up or down
the gradient, the organism quite reliably "learns" to be dysfunctional.
Your demonstration that such learning could easily be disrupted by
changing the behavioral consequences has the same force as "proving"
that PCT doesn't work by demonstrating that human participants fail
to control their e. coli spots when wearing blindfolds.
This is not a correct analogy at all; the changes I made in the environment
showed that your "learning" model learns to lose control in circumstances
where real control systems (and people) don't. The "learning" is not
being disrupted; it's working just as you said it does -- and the result is
that E. coli learns to lose control. Demonstrating that people lose control
of a perception that they can no longer perceive is hardly a demonstration
that PCT doesn't work. Want to re-think that one? What does such a
demonstration actually show?
Me:
PCT shows that purposeful behavior MUST be viewed as selection OF
consequences. Control theory shows how organisms must be organized in
order to be able to select and produce the selected consequences of
their actions.
Bruce:
Yes, and I agree. But you are talking about execution, not acquisition.
OK. So you seem to admit that ordinary control is not based on selection
BY consequences. You seem now to be trying to preserve the notion of environ-
mental selection by saying that it is involved in acquisition (learning). OK.
How about presenting some actual learning data so we can see what we have to
account for; and then we can see whether we need to resort to selection
BY consequences. The ECOLI4 program might contain what you consider to be
a learning algorithm but it is a model that does not behave like a real
organism. Let's see what you think of as learning data; then we'll see what
kind of model we need to account of "learning to control".
Me:
Why is this so hard?
Bruce:
I dunno--you tell me. Sand in your gears? (:->
My guess is that it's because most people don't realize what they are
getting into when they stumble across PCT. PCT is not a new, fancy way
to incorporate "feedback control" into a behavioristic perspective on
behavior. PCT is what Bill Powers implied that it was in his reply to
Baum in teh 1973 Science interchange -- PCT is a conventional psycho-
logist's worst nightmare; if control theory is right then conventional
psychologists of all stripes have been making a career out of studying
an illusion. Th nightmare comes in the form of two cute little equations:
1) p = r and
2) o = -1/g(d)
A nightmare for conventional psychology; a sublime reality for me.
Best
Rick