[From Bill Powers (2004.09.03.1418 MDT)]
Bruce Abbott (2004.09.02.1300 EST)–
Reorganization under the current
HPCT proposal is a continuous process whose rate increases with the
magnitude of persistent error in the so-called “intrinsic”
variables. The reorganization process introduces small, random changes to
the perceptual/control hierarchy. Changes that lead to better control
over the intrinsic variables reduce the level of persistent error,
slowing the rate of reorganization and thus tending to preserve the
changes.
The problem with this proposal, as you know, is that it does not
necessarily make those changes in just the right places where they would
do the most good.
Yes, that is a problem, although there are possible solutions for it. The
same problem holds for reinforcement theory, doesn’t it? A stimulus is
considered to be reinforcing if the behavior it follows increases in
frequency. But how does the reinforcer know which behavioral systems to
reinforce? The assumption that the system producing the reinforcer is
made to act more frequently asserts this specificity from the start; it
doesn’t provide the mechanism. What steers the effect of reinforcers to
just the parts of the brain that produce that behavior instead of some
other behavior? Same problem.
Reinforcement increases in frequency if the behavior that produces it
increases in frequency; I don’t see any argument with that. But what
makes that particular behavior increase in frequency? Skinner proposed
that the behavior is controlled by its consequences, picking the
consequences as a place to start keeping track of cause and effect. But
that is arbitrary; you can also start with the behavior if you wish. I
think that neither is correct, because we’re talking about a closed
loop.
(You have suggested that
consciousness may have something to do with focusing those changes where
they are needed, but a rigorous proposal as to how this mechanism works
is currently lacking.)
What is needed in HPCT is a
mechanism for reorganization that focuses its changes where they do some
good.
There is actually such a mechanism already present; awareness isn’t the
only possibility. Study how the reorganization works in the
27-degree-of-freedom arm model. In this model, every control system
continuously reorganizes its own input function independently of all the
others even as the systems interact. The result is that the set of input
functions becomes the transpose of the set of output functions and
control becomes very effective. Because of that effectiveness, errors are
kept small and reorganization gets very slow.
In effect, reorganization occurs everywhere at once in this system. But
recall that the rate of change of parameters is very slow to start with,
and is proportional to the rms error, so as the error gets smaller, the
changes due to reorganization become much smaller and the process goes
much more slowly. It’s the very control system that becomes organized
that “selects” the right reorganizing action, turning the
reoreganization down as the control system becomes more able to keep the
error small. No third party is needed.
This would seem to fit with Bruce Nevin’s/Isaac Kurtzer’s observation
about neurons growing and destroying connections like mad in vitro. That
would look like the maximum rate of reorganization, and it would be going
on whenever control wasn’t working (as it doesn’t in vitro). In vivo, as
the neuron’s connections become more suitable for controlling some
variable, the error that is driving the reorganization declines, going to
some very low level when control becomes efficient. What we would like to
know is the kind of error that results in this mad random branching and
that can be reduced in vivo when this branching produces the right
control systems. That remains to be determined.
An example where such a
change does occur is provided by the hungry rat that learns to press a
lever in order to obtain food. Here, the animal already has a (rather
complex) control system organized to locate food and, when food is found,
consume it. In part, error in this system may set a reference in a
sub-system controlling for the perception of food, whose actions may
include what we might describe as “searching” behavior.
Eventually, the rat does something in the course of this activity that
depresses the lever, and food immediately appears. The rat may learn from
this experience to press the lever, or it may require a few more
repetitions of the lever press before the rat seems to “get the
idea.” But what emerges is a new means of controlling for the
perception of a food pellet.
How is this reorganization to be explained?
I’m not sure it should be called reorganization. It could be a systematic
learned control process (sequence). A search strategy is a control
process that should terminate when its object has been achieved. It’s
pretty systematic in rats that I have seen, courtesy of your videos. But
systematic or not it would fit the same general paradigm as E. coli
reorganization, in that error would produce the search, the cessation of
the error should terminate the searching action. Here the “searching
action” consists of activating lower-order systems by adjusting
their reference signals, first this set, then another set, and so on
until something happens to reduce the error.
It seems to me that you’re saying the same thing:
We have an established system
that at times sets a reference for perceiving that food is available to
eat. As the way to make the pellet appear is unknown to the rat, the
system defaults to search mode. There are already control systems present
that allow the animal to engage in various activities such as approaching
a specific location, sniffing in particular places, rearing, and so on.
When the animal happens to bring its paw down on the lever during the
course of such activities, it already has numerous finely-tuned control
systems that were actively positioning the animal in the chamber at the
time the lever moved downward. As this action happened, suddenly, a
pellet appeared. This immediately eliminated the on-going error in the
system controlling for perceiving available food. A few repetitions of
this coincidence and the rat no longer searches the chamber, apparently
at random, for food. Now, each time it wants another pellet, it
approaches and presses the lever. It would seem that the control system
engaged in obtaining food now includes a new conditional branch in the
program: if lever present, then approach and press.
I don’t know if you ever knew of Wayne Hershberger’s doctoral thesis. He
set up chicks in an apparatus such that if the chicks approached the food
bowl, it receded from them. To eat, the chicks had to back away from the
food bowl. Most of them were able to master this unnatural control
task.
I agree with the description of events as you present it, but not with
Fred Nickols’ characterization of it:
“Special
reinforcing perceptions”
are perceptions informing me my actions had the desired
effect(s).
II would amend this to state that what “reinforces” the change
in the organization of the system might better be described as an
immediate reduction in the level of error of the system initiating the
actions (in my example, the system having a reference for finding
food).
These ways of describing it omit the most obvious fact: the “special
reinforcing perception” is precisely the one that the animal is
trying to control. It’s not some separate aspect of the food or water;
it’s whatever aspect of the food or water itself for which the animal has
a nonzero reference level. That IS the desired effect. Putting it Fred’s
way makes it sound as if the reinforcer is something separate from the
desired effect, like a post-it stick to the food pellet saying “This
is what you wanted.” But that is an unnecessary embellishment. The
animal has a reference level for ingesting certain objects and already
knows what it wants. It has the ability to search for such objects and to
learn how to make them available. Sometimes a learned systematic search
process suffices; sometimes reorganization is required, as when an
experimenter thinks up some odd requirement that must be met to get
food.
There is also one other point I
wish to discuss. Some perceptions are accompanied by an experience of
pleasure or displeasure. If the experience is pleasurable, we will
often control for that perception; if aversive, we will often control for
the absence of that perception.
I disagree with this way of describing it, if I understand you, which
implies that a pleasant perception is so because it triggers a separate
signal saying that this perception is pleasant or unpleasant. I
understand that this is the received wisdom – endorphins and all that.
But I think that is an unparsimonious proposition, and it ignores the
fact that the same perception can be either pleasant or unpleasant,
depending on your current reference level for it. Isn’t it
sufficient to say that the experience of bringing a perception, any
perception, to match a reference signal that we have set to a high level
(including all the auxiliary experiences that go with this) is itself
what we mean by “pleasure”? And the experience of a high amount
of a perception, any perception, for which we have set a low reference
level is itself what we call unpleasant or painful?
If I recall correctly, you have
argued that these feelings (perceptions, really) are merely
side-effects.
Not side-effects: the effects themselves ARE the pleasure or displeasure.
We have been looking elsewhere for something that is right under our
noses. I don’t think there is any other signal to find. That there was
some other was somebody’s idea long ago that apparently never got
questioned. It was proposed long before anyone heard of endorphins or
pleasure centers, and I think it heavily influenced the interpretation of
the data that led to the idea of pleasure and pain centers, endorphins,
and so on.
We have references for the initial
perceptions and the feelings are simply interpretations of our own
actions with respect to those stimuli (as James proposed with respect to
emotions).
That’s not a bad idea, but what I’m proposing is even
simpler.
An alternative, which I favor, is
that what we are controlling for is the experience (pleasure, absence of
displeasure) that accompanies certain of our
perceptions.
So the only reason you control for keeping your car on the right side of
the road is that you experience pleasure or absence of displeasure from
doing so? That just does not ring a bell for me. I think it is the
perceptions that we control for, period. We don’t need all these extra
signals floating around to explain that. I’ve proposed a connection
between learning and our physical well-being in the form of a
reorganizing system, which takes care of basic learning. We learn to
control our experiences; when we succeed we are content; when we fail, we
feel driven to do something about it, or withdraw.
The link between perception
and feeling-state may be innate. For example, under proper conditions we
humans usually find a sweet taste to be pleasurable. Because sweetness is
a taste quality evoked by sugars, and sugars both supply energy and are
usually associated with needed vitamins and minerals (in fruit), natural
selection presumably would have favored an association of sweetness with
pleasure and, seeking the things that give us pleasure, we would thereby
be led to consume what the body needs. It could also be acquired through
experience.
I don’t buy those “rational natural selection” arguments. Why
not just say that we have a high reference level for sugars and the
feelings we get from ingesting them? You can attribute that reference
level to evolutionary processes, but that is only to say that organisms
who set those reference levels too low didn’t survive. We, perhaps, can
see why that might be so, but that’s a just-so story, unprovable,
unfalsifiable.
Since any perception can be either pleasant or unpleasant, I can’t accept
the idea that there is an innate connection between perceptions and
feeling states.
If these feelings are merely a
side-effect of control action,
Not “merely a side-effect” – that is not what I say. They are
among the essential effects of control actions, and they are controlled
perceptions themselves. There is no other signal saying that a
perception is pleasurable of painful; the states of perceptions relative
to our reference levels (of the moment) define pleasure and
displeasure.
I think there has been a long search, for a century or more, for what
determines pleasure and pain. Lacking a workable model and being stuck
more or less with the simple S-R view, theorists postulated an elaborate
set of ad-hoc explanations, none of which, in the PCT model, are
necessary.
Best,
Bill P.
···
I fail to see why they would
exist as phenomena of consciousness – why the ability to experience them
would have been preserved during the evolutionary process.
Best wishes,
Bruce A.