[From Rick Marken (941208.1010)]
Bruce Abbott (941207.1540 EST) --
Did Bill Powers actually state in the above that an innately-organized, pre-
programmed, negative feedback loop illustrates REORGANIZATION? I am totally
mystified.
The re-organizing system was always conceived of as an innately-organized
negative feedback system -- controlling intrinsic error by operating on the
perceptual control hierarchy rather than by having direct effects on the
environment. Schematically, the reorganizing system looks like this
r-->C--intrinsic error-->rate of RANDOM change --->control hierarchy
^ |
> -----------intrinsic variable, p -------------------|
r is the reference for some intrinsic perceptual variable, p, like CO2
concentration. The difference (computed by C) between the value of the
intrinsic variable and its reference level is intrinsic error which affects
the rate at which random structural changes are made to control systems in
the perceptual control hierarchy (systems in this hierarchy control
perceptual representations of aspects of the external environment -- not
the internal [intrinsic] environment). The rate at which these random
changes are made drops to zero (or so) when the intrinsic variable matches
its reference (the drop to zero seems to be rather abrupt, based on
reorganization data obtained by Dick Robertson). This reorganizing
system is a control loop that is NOT guaranteed to work because there may be
no change in the control hierarchy that that has the effect of moving the
intrinsic variable to its reference level.
Is reorganization now to include the ordinary operation of a control system
in response to perceptual error? Please explain...
Yes; but reorganization is control of perceptions of intrinsic variables --
variables that represent important physical conditions of the organism
itself. And it is control of a peculiar kind because there is no regular
relationship between error and output (as there is in a "regular" control
system); the output of the reorganizing system is random because the
reorganizing system cannot possibly know (Hugh Petrie's "Meno paradox")
WHAT to control in order to nudge intrinsic variables toward their reference
states. It cannot know, for example, that controlling the position of a
bar has an effect on an intrinsic variable like nutrient level. The
reorganizing system randomly changes the structure of the percepual control
hierarchy and, if it's lucky, a change will result in the construction of a
(or the tailoring of an existing) control system so that the reference
signal sent to that control system specifies a particular level of bar
position.
My "learning" model e. coli shows reorganization by your (old) definition;
your original (static) e. coli most definitely does not.
Your "learning" model shows that external circumstances can "set" the
parameters of a control system model so that it controls. The control system
organization of your model never changed -- just the quality of control
exhibited by the model (good control when consequences selected good control
parameters and poor or no control when consequences made the probability
of tumbling in positive and negative gradients equal). In fact, your model
did not "learn" to deal with a changed environment; when the probabilities of
the consequences of tumbling were altered, your model parameters were changed
by this change in circumstances so that control was lost).
Whether your "learning" model of E. coli became a good or poor controller was
completely a function of circumstance; the model itself could do nothing
to improve its own ability to control in environments where the probability
of an improved gradient after a tumble was the same when going up or down the
gradient. Your model was NOT an implementation of reorganization. It was just
an exrtemely poor implementation of a control system, demonstrating the
problem of allowing the parameters of control to be determined by
environmental cirsumstances (selected by consequences).
A reorganizing model of E. coli would have randomly changed things like the
NATURE of the function that computed dNut or the nature of the
function that computed the difference between dNut before and after a tumble
or the function that transformed error (dNut-0) into the probability
of an angles of movement after a tumble; it would make these changes until
some intrinsic variable (such as the average size of the error signal over
time) stayed at some reference level (like 0).
Bill said:
The concept of reorganization gives us a new understanding of "reward."
It is not that getting a reward causes the behavior that leads to the
reward; it is that lack of the rewarding thing creates a state of
reorganization that will cease only when the lack is corrected.
Bruce says:
Now this seems more reasonable, although it has nothing to do with the
earlier discussion of e. coli behavior. However, it needs to be restated
slightly.
Uh Oh. I feel a teaching session coming on.
It is the fact that a behavior PRODUCES the reward that leads to the
repetition of the behavior.
Now, I agree that the behavior PRODUCES the reward, but are you (as an expert
in PCT) sure that the reward "leads to the repetition of the behavior"? How
come there are times when the reward produced by behavior suddenly seems to
lead to the cessation of behavior after its apparently been leading to the
repetition of behavior for some time? Does a reward sometimes decide to stop
selecting behavior? Does its reinforcing power get temporarily used up until
it gets re-charced? Why do rewards change like that? How do they know when to
do it?
Did Bill Powers really get things so profoundly confused when he said:
It is not that getting a reward causes the behavior that leads to the
reward;
Now you are teaching us that "reward leads to repetition of the behavior"
that produces the reward.
Have you found this statement to be an accurate charactrization of the
behavior of the PCT operant conditioning model that Bill recently posted to
the net?
Best
Rick