[From Bill Powers (960404.0930 MST)]
Bruce Gregory 960403.1635 --
PCT uses reorganization to refer to a random process "provoked" by
failures to control. That I mean is a processs of replacing one
"model" by another, also provoked, but the change is not random.
... we try to get the students attention in a variety of less
dramatic ways. One is to confront a student with an anomaly by
asking her to predict the outcome of a demonstration and then
allowing her to observe something very different from what she
anticipated. (If she does not first make a prediction, she may
well convince herself that the demonstration is no surprise; she
will treat the demonstration as though it were _not_ a
disturbance.)
When you make a prediction, you're stating what will happen. Then you
observe what does happen. If there's a difference you have to change
something to make the difference go away. The reference condition is
"prediction matches observation." So basically you're talking about a
control process, which you're disturbing.
"Reorganization" is, as you say, the word I have used for the basic
random shuffling that's the only way to get organized when there are no
systematic ways available. Maybe we should just say "learning" or
"problem-solving" when the method isn't random. Or we could go the other
way: use "reorganization" as the generic term, and "trial-and-error" for
the non-systematic method. The language in this area needs some work.
In learning how to control something, there are many systematic methods
that can work, if you have learned a system. Simple example: learning
which way to exert an effort to make a door open. You could simple try
pushing and pulling at random, or you could always push first, and if
that doesn't work, pull. In looking for a lost item in a field, you
could simply wander at random through the field, or mark the field into
a grid pattern and systematically search each square in a raster scan.
Systematic methods are probably always more efficient than random ones.
But they have to be learned. That's basically why the underlying "E.
coli" kind of reorganization has to get into the act somewhere. Once
you've acquired a systematic control system for keeping some variable
under control, you no longer have to use trial and error: the more
effective method of control keeps errors from getting large enough to
start the more basic process working again. But it's got to be the
random process that starts this bootstrap process, and that provides a
fallback method when the environment changes enough to render the
learned system inadequate.
Random reorganization is probably always involved, to some degree, when
a learned system doesn't succeed. We have to think of success and
failure on a continuum, however, not as an either-or judgement. There is
better and worse control. The student who makes a wrong prediction does
so from some systematic base for making predictions. When the wrongness
is discovered, obviously the existing method of making the prediction
has to be altered. But altered how? If the student knew how, the mistake
would not have been made. So at some level of detail, all the student
can do is change something at random and see if the result is a better
prediction. If the choice is binary, however, the concept of "do the
opposite" is a useful _systematic_ method, so maybe random
reorganization is not always necessary.
The basic principle is always error-correction, but the method depends
on the degree to which the system is already organized. Random changes
at a high level in the hierarchy make use of existing organizations at
lower levels, if they exist. Needed organizations that don't exist can
be brought into existence only by trial and error at the appropriate
level. The end-point is the acquisition of a systematic control process
that keeps errors small enough that the trial-and-error method doesn't
start up.
As you can see, I haven't done a lot of productive thinking on this
subject -- just a few sketchy principles. It's a wide-open field.
···
-----------------------------------------------------------------------
Chris Cherpas (960403.1545 PT) --
You took that very well -- only a little bristling.
A reminder: I've had much less invested in my technical categories
than you've had in yours; I'm not proud, just less skilled in
developing, and less invested in, a particular theory -- and a
little younger.
It's funny, but I don't really think of control theory as a "theory" --
at least not in the sense of being a conjecture. It's more like the
theory of electronics, or of planetary movements. Behind the concept of
a control system is a mathematical analysis, but also a structure of
_physical_ theory that rests on the same observational foundations as
those of physics. This isn't just a matter of inventing categories and
giving names to them.
The problems between PCT and other theories arise when the theories are
compared on the basis of the overall interpretation of behavior that is
involved. Bruce Abbott has pointed out that the "reinforcement"
interpretation makes internal sense; if you assume that a reinforcer has
a specific kind of effect on the probability of a response, you can
speak consistently about the observations. At this level of comparison,
you can say that PCT is one interpretation and reinforcement theory is
another, each equally valid in its own terms.
But PCT does not exist just at this level. Behind it are all the
theories of physics and neurology. Internal consistency is not the only
consideration: there is also consistency with all the basic knowledge of
other fields of science. PCT is simply an extension of engineering
control theory, which in turn is an extension of basic physics.
Reinforcement theory, on the other hand, exists in a universe of its
own, with essentially no connection to physics or neurology. The kinds
of causes and effects that are appealed to in reinforcement theory are
not connected to the physical world; they exist only in a self-contained
conceptual world. There are no mechanisms, as a physicist would
understand a mechanism, behind the interpretations of reinforcement
theory.
You will notice that my approach to a PCT analysis of the "adjusting"
achedule (as I understood it) did not begin with asserting that any
control process was involved. It began by trying to characterize the
observable dependence of reinforcement rate on behavior rate. The next
step would be to try to deduce, from observations of actual behavior
rates and reinforcement rates, the transfer function of the behaving
system. This step would not depend on assuming a control system, either.
Only at the end, after the closed causal loop had been properly
analyzed, could we reach a conclusion as to whether control is or is not
involved. In fact, the term "control" is simply a name for a particular
arrangement of causation in a closed causal loop -- not for _all_ such
closed loops, but for closed loops having certain specific
characteristics. The analysis would tell us whether to use the term
"control" or not.
I don't think there's any corresponding method of analysis behind
reinforcement theory, or melioration theory, or any theory in the
behavioral sciences. People in the behavioral sciences are not taught
this method of systems analysis. They don't (in my experience) even know
that such a method exists. There is no method, as far as I know, for
verifying that a reinforcer has any specific effect on the organism, or
for showing that it has no such effect. All you can say is that if the
probability of a response increases when a certain consequence occurs,
that consequence must have been reinforcing. There is no more detailed
level of analysis to which you can appeal.
-----------------------------
But, by the way, don't control systems minimize error? Can't that
be called optimization?
No. Control systems may behave in such a way that error is
(approximately) minimized, but few of them work by minimizing error.
They work by producing output that is proportional to the magnitude of
error, and aimed to oppose the direction of error. Both magnitude and
sign of error are involved in most control systems. The usual concept of
a minimizing or maximizing system assumes that directional information
is not available: if the variable is not at its minimum or maximum, the
error does not indicate which way to move to correct the error. In the
standard PCT control system, the error signal carries both a magnitude
and a sign, and the sign indicates the direction of action needed to
make the error smaller.
Your question brings me back again to a problem that keeps coming up,
and which I seem to be very poor at talking about. As observers, we can
say that a control system minimizes error, because that is a valid
description of the outcome of its behavior. But this does not mean that
inside the control system there is something concerned with finding a
minimum. What we see as minimizing is a _side-effect_ of the actual mode
of operation.
Consider "utility," a concept often said to be maximized. We can speak
of body temperature as having a utility curve with a maximum at 98.6
degrees F. That temperature is the most useful, the optimum, for
carrying out the body's biochemical functions, or so we assume after the
fact. But the body's temperature control systems never compute utility,
not do they maximize or optimize anything. They compare sensed body
temperature with a reference temperature, and activate warming or
cooling processes as appropriate and to the appropriate degree. The
result is that body temperature stays very near the set-point. That is
_all_ that the control system does.
But an observer can see that this body temperature is probably the best
temperature for meeting all the requirements of life. So the observer
describes the system as if it knew that this is true and was seeking to
optimize the system, or maximize the utility of the temperature. This
leads to elaborate models that compute utility functions or optimality
criteria and use them as the basis for producing warming or cooling
outputs. And all unnecessarily, because the actual system does not do
any of those calculations. It just reacts to a temperature error by
sweating or shivering. All the rest is in the observer's imagination.
The way the system actually works is much simpler than the way it is
imagined to work, although it produces exactly the same _outcomes_ as
the more elaborate system.
Perhaps one must become a "reborn" control theorist: we are born
control theorists, we get indoctrinated in various misconceptions,
and then we see the light again through discipline and
experimentation.
Saved but not healed,
I hope that's not how it really works. If control theory really applies,
it applies because the real organism is organized as control systems are
organized. Within obvious limits, control theory is as solid as the
theory of gravitation, or Ohm's law, or an engineer's analysis of the
strength of a bridge. It's not a "psychological" theory or a
"behavioral" theory. It's a way of understanding what happens when
components of a system, any system built of any materials, are put
together in the right way. That's the beauty of it: you don't have to
believe in it. You can doubt all you wish, but every time you start with
the basic concepts and work out the consequences, you end up in the same
place. It's not just a "perspective."
Now, after me: "How firm a foundation ..."
----------------------------------------------------------------------
Rick Marken (960403.1810) --
Your diagram showing a continuous transition is at least closer to the
truth than mine. I should have said explcitly that I was diagramming the
_steady-state_ transfer function. But in a plot of one variable against
another, it's hard to show the dynamics at the same time, so your
diagram isn't really the correct one, either. The best you can do is
plot each variable against time, and doing that introduces initial
conditions and assumptions about the waveform of the independent
variable.
A simulation like the one in your Hypercard stack will give the truest
picture.
-----------------------------------------------------------------------
Best to all,
Bill P.