[From Bill Powers (950806.2300 MDT)]
Avery Andrews (950807)--
So replicators that can maintain local conditions suitable for
their replication will have an edge over those that can't, or are
less good at it. This, we assume, means the replicators will tend
to develop control systems.
A few years ago I wrote a paper on exactly this point -- it should be
appearing in World Futures some time soon (via Cliff Joslyn). The basic
idea is that in a soup of replicating molecules, negative feedback
effects on the substrate that tend to stabilize conditions on which the
accuracy of replication depends would be favored. Eventually they would
dominate. One point I made was that it is not mutation that needs an
explanation, but the incredible stability of living molecules. Repair
enzymes are an example of active stabilization of the genetic code
against disturbances. There's also a nice PCT explanation of punctuated
equilibrium. Reorganization comes out as an evolved simulation of
evolution that doesn't require the individual organism to die in order
to deal with "selection pressure."
Genetic Evolution (Darwinian selection in populations of
replicators) is a very powerful method for developing effective
control systems (that (a) manage to control their perceptual
variables, and (b) thereby stabilize the environmental variables
favorable for replication), basically because large numbers of
variants can be tried out simultaneously, and only a fraction need
to survive.
The big challenge is to devise a genetic strategy that doesn't give the
organism credit (by letting it reproduce) for going partway to a
solution of a problem. The evolution of internal reference signals gives
the organism an internal criterion against which to judge reduction of
error. In standard natural selection, there is only one error: death. I
think there are limits to what this crude method of selection can do in
a finite time.
But learning/reorganization, at least within single organisms, is a
much more problematic affair then Genetic Evolution: the organism
can only try a few possibilities, one after the other, must survive
all the experiments, and find an adequate solution in a timely
manner. So what you'd expect to find is not some general,
nonspecific reorganizing capacity, analogous to random mutation,
crossing-over, etc. om GE, but a variety of special-purpose
mechanisms, optimized to work well under specific conditions (the
native environment of the organism). (This is Chomsky's basic
point about learning.)
I agree that there must be some preorganization to allow reorganization
to work within an individual. But reorganization is centered on control
of specific variables toward specific reference levels, and that is what
makes it so much more powerful than an undirected evolutionary process.
The reorganizing system can recognize a move in the right direction even
though there is still an error, and a move the wrong way through an
increase -- but not a fatal increase -- in error. If the only way to
recognize an error is to die, no other organism can take advantage of
this information: it is lost with that final "oops." That makes natural
selection very much slower than reorganization.
···
-----------------------------------------------------------------------
Rick Marken (950806.1812) --
Very neat, Rick, I missed the application of the Test and didn't see the
subtle point you were making.
To expand a bit:
The Test involves making a prediction of the effects on a proposed
controlled variable of a change in the environment, under the hypothesis
that the organism will NOT change its behavior so as to counteract those
effects. In Bruce's analysis, he predicts that when the ratio changes,
the organism's actual bar-pressing behavior will NOT change in such a
way as to reduce the changes in the reinforcement rate, which under
special circumstances can turn out to be true. In that case, even though
the mean behavior rate is seen to change, the actual bar-pressing
behavior remains constant, disproving the hypothesis that the
reinforcement rate is under control.
This is the part of the Test that requires us to demonstrate exactly
what change in action is resisting the disturbance. Even if the
disturbance seems to be resisted, the Test is failed when the action of
the proposed system does not explain the resistance.
When the only disturbance is a change in the proportionality constant of
the environmental feedback function, it gets harder to separate control
from noncontrol. In the ratio experiments, there is necessarily a loss
of loop gain as the ratio goes upward from 1. There is a rapid loss of
control as the ratio increases, so it's a matter of arbitrary definition
to say just when there is "no control." I've proposed a criterion of a
loop gain of 5-10 or greater. If the loop gain at FR-1 is 30, then at
FR-2 it is 15 and at FR-4 it is 3.75 -- and by my arbitrary criterion,
control is lost. Since control theory is supposed to explain control, we
should avoid using it to explain behavior in which control is
negligible.
There's one case where control can work over a wide range of ratio
schedules: when there's a time-integrator somewhere in the control loop.
A time integrator has infinite loop gain in the steady state. As the
static loop gain drops, the system simply takes longer and longer to
counteract a disturbance, and is less and less capable of counteracting
varying disturbances. But it will eventually correct any steady-state
error.
Anyhoo, very nice demo.
-----------------------------------------------------------------------
Best to all,
Bill P.