.. as I figure it. That's where I got the numbers 0.37 and 0.63.
···
----------------------------------------
You are still trying to convince me of something that I agreed to some
time ago: if E. coli remembers what happened the last time it executed a
behavior, and behaves logically with the aim of avoiding bad results and
achieving good results, assuming that the same relationships still hold,
it will learn to tumble when going down the gradient and not tumble when
going up the gradient. That method would be sufficient to get E. coli up
the gradient.
It is a sufficient method, but is it necessary? To say it is necessary
is to say that unless the system is capable of remembering past
perceptions and their relation to past behaviors, and of computing the
logical relationships described in your program, it cannot learn to go
up the gradient. That is not true. I can think of two other methods,
neither of which requires memory or logic.
One method is to specify that the nutrient itself can be "bad" or
"good", which implies a low reference level ("bad") or a high reference
level ("good") for NutCon. As it happens, deviation of NutCon from a
reference level creates an error signal that can be used to adjust the
gain of the first-level E. coli system controlling dNut, either
positively or negatively. Depending on whether NutCon is above or below
its reference level, the gain of the first-level system will go either
negative or positive, so the first-level system will tumble its way
either down or up the gradient.
By setting the reference level for NutCon to zero, we obtain a system
that adjusts itself to go down the gradient as long as any gradient is
detectable: avoiding a repellent. By setting the reference level for
NutCon to the maximum possible, we get an E. coli that tumbles its way
up a gradient: seeking an attractant. The value judgement that
distinguishes an attractant from a repellent is in the setting of the
reference signal. The mechanism for adjusting the sign and amount of
gain according to whether an attractant or a repellent is present is in
the effect of the second-level system on the gain of the first-level
system.
A second method, which I still haven't implemented, is reorganization.
Given that going up the gradient is "good," increment the ratio of delay
period to error (the gain) by some small amount, delta, on each
iteration as long as dNut is averaging positive. If the running average
of dNut goes negative, change delta randomly between limits of +d and -
d, where d is a small number, then go on adding it to the gain on each
iteration. The average of dNut must be sampled at some relatively long
interval to give the changes in delta a chance to be reflected in the
average dNut. This method will gradually adjust the mean ratio of delay
to error until the right relationship exists for E. coli control. It
uses the same principle for reorganization that E. coli uses for
steering. This method will not automatically switch between seeking
attractants and avoiding repellants; we must manually alter the
direction of comparison (as in your model) to make that switch.
This reorganizing method could also be used in the conversion of second-
level error to first-level gain in the first model. Then it would treat
NutCon as an attractant or a repellent depending on whether the
reference level is set high or low.
In your model, "gain" is represented by the comparison dNut > 0 and so
on; the magnitude of the gain is set by LearnRate, and the sign by the
direction of the comparison and/or the effect on either increasing or
decreasing the probabilities. But your model requires memory of past
values of quantities as well as temporal relationships, and the ability
to perform logical computations. E. coli is not likely to contain the
necessary computational machinery and memory for events. Furthermore, to
switch from seeking attractants to avoiding repellents, you must change
signs in at least four places in the model. That is still a manual
adjustment.
I realize that you were not trying to say that E. coli learns. But the
models I have offered would allow E. coli to behave in the same way you
define as learning without going beyond mechanisms we could reasonably
imagine existing in E. coli's biochemistry. If these mechanisms would
work in E. coli, then they would work in more complex organism, too.
--------------------------------------
So the model is entirely consistent with the law of effect as stated by
Thorndike over 80 years ago, as you now agree...
Yes, I agree. But Thorndyke's concept of the law of effect, while
sufficient to account for certain learning phenomena, is not necessary,
nor is it the simplest possible model that would accomplish the same
thing.
One thing we can agree on: whatever the mechanisms involved, they will
produce an appearance that is the same thing we would expect under the
Law of Effect. A desired consequence is, eventually, brought about by
one of the behaviors that is actually effective in producing that
consequence. If only one behavior is effective, then that behavior will
probably appear. That is the phenomenon for which we are trying to
account. We are not trying to account for how organisms remember
associations of outcomes with past actions; that is a proposed
mechanism, not a phenomenon (except in those cases where we can show
that this mechanism is specifically involved).
---------------------------------------
What Rick has been trying to do is to devise a situation in which the
association between present values of dNut and past values is broken;
where the past value no longer predicts the outcome of a tumble. His
examples still seem a bit too ad-hoc for my taste; I would like to see
something more natural-sounding. We know that it is not necessary for
the mechanism of gradient-climbing to use that information, because our
PCT model does not use it, yet it climbs the gradient quite efficiently
(more efficiently than your model does, because it never causes a
shortening of the delay to the next tumble when dNut is positive).
This is more of the necessary-sufficient distinction. While it happens
that in this situation the previous value of dNut does predict the next
value, and while it is true that this information can be used as the
basis of a model which will increase PTS- and decrease PTS+
appropriately to behavior in an attractant gradient, the same result can
be achieved without using that information.
You objected to my simplified model, which in a little more detail is
if (dNut > 0) then PTS+ := PTS+ - LearnRate else
if (dNut <= 0) then PTS- := PTS- + LearnRate
The only basis for the objection that I can see is that it does not make
use of past information. The behavior of this model is qualitatively
identical to the behavior of yours. The behavior would be quantitatively
identical if we made the learning rate in my model 0.26 times the
learning rate in yours (0.63 - 0.37), because in your model, a positive
dNut is is only 26% effective in producing a decrease in PTS+, on the
average. My simplified model works in all the situations where yours
works, and it does the same thing. What is wrong with it, except that it
doesn't use the Law of Effect?
------------------------------------------
One last time:
Suppose it is shown rigorously that
Mechanism A implies phenomenon B.
Does it then follow that
Phenomenon B implies Mechanism A?
------------------------------------------------------------------------
Best,
Bill P.