[From Bruce Abbott (941115.1400 EST)]
Bill Powers (941114.2200 MST)
The logic here is extremely complex, but there is a simple way to see
whether the previous value of dNut is really acting as a reinforcer.
. . . . .
So who ever said that the previous value of dNut is acting as the reinforcer
here? I recall explicitly stating otherwise! (See below for a recap.)
The quickest results of all come from simply randomizing NutSave:
NutSave := random - 0.5;
So what is making the probabilities change is not any systematic effect
of NutSave, but something else about the nonlinear and circular geometry
of this situation, combined with the complex logic. I really don't have
the faintest idea why the net effect is swimming up the gradient.
[Enterprise makes a run for the Motaran Nebula on impulse power, closely
pursued by Reliant, which has been pirated by Kahn and his crew . . .]
Reliant helmsman, speaking to Kahn: If they go in there we'll loose them.
Kahn: Explain it to them.
(a) NutSave is the dNut value resulting from movement in the new direction
selected by the last tumble. Its value is a function of the angle
selected at random by Tumble(Angle), and thus is itself random. Your
substitutions simply replace one random number with another, which is
why they have little effect on the model.
(b) NutSave is NOT the reinforcer here, as I took pains to explain. To
repeat: Certain consequences of tumbling are NOT random. Tumbling
while moving up the nutrient gradient usually makes things worse;
tumbling while moving down the nutrient gradient usually makes things
better. Better = reinforcement, worse = punishment.
(c) A naive e. coli does not know what the consequences of tumbling are.
All it knows is that it LIKES to go up-gradient, and DISLIKES going
down-gradient.
(d) To gain control over nutrient change (dNut), e. coli has only one
response it can try: tumbling. But it can try this response under
different conditions and see what results. So it tries tumbling when
the nutrients are increasing. Because (unknown to e. coli) tumbling
produces a random change in dNut, the usual result is that nutrient rate
gets worse. So e. coli learns not to tumble when nutrients are
increasing. It tries tumbling when nutrients are decreasing. Because
this usually improves the nutrient rate, e. coli learns to tumble
immediately when nutrients start decreasing.
The result is that e. coli learns a very efficient control system:
if dNut < 0 then tumble else don't tumble
So what is making the probabilities change is not any systematic effect
of NutSave, but something else about the nonlinear and circular geometry
of this situation, combined with the complex logic.
It is not any systematic effect of NutSave, nor is it a consequence of the
nonlinear and circular geometry of the situation. It is the systematic effect
of the change between pre- and post-tumble nutrient rates, as observed
separately under positive and negative nutrient gradients. The logic is
really not all that complex, once you get to know it--certainly no more
complex than yer average perceptual control system. I hope I've been able to
make that logic clear.
The model is informative, for it tell us what conditions are necessary for
learning in this situation:
(a) e. coli must be able to sense the rate of change in nutrients that
result from its forward motion.
(b) e. coli must be able to compare the rate of change before and after a
tumble in order to determine the effect of tumbling.
(c) e. coli must be able to discriminate the results of tumbling separately
for tumbles that take place while dNut is increasing and while dNut is
decreasing.
(d) the selection mechanism must work so as to favor responses that tend to
increase dNut and to suppress those that decrease dNut.
I'm still thinking about how to implement this explicitly as a learning
control system. Somehow, experience with the change in nutrient rate
following a tumble in positive and in negative gradients should determine the
form of the function controlling the tumble interval in the lower-level
system. The model would start without a systematic relationship between dNut
and tumble interval; experience would then change the parameters until the
correct function emerged. How about you or Tom or Rick giving it a shot?
Regards,
Bruce