[Bruce Abbott (941104.1000 EST)]
[From Bruce Abbott (941104.2020 EST)]
Rick Marken (941104.1400)
Tell the aliens that ECOLI4 still contains no reinforcement. The stimulus
effects just go back one extra trial. That is, the change in response
probability depends on the current input (dNut) and the prior input
(NutSave) but there is no differential change in the probability of the
response depending on what response was actually made after the prior
stimulus - - ie. there is no reinforcement. Because there is no
reinforcement, E. coli still does its biased random walk to the target; that
is, it controls.
I have consulted with the aliens, and they refuse to accept your analysis.
There has been a new development. Here, let's tune in--the last episode of
Star Trek, The Final Solution is in progress.....
[We are once again on the bridge of the Enterprise. On the main viewscreen,
the alien creature from the previous episode appears.]
Spock: Captain, I have programmed the ship's computer to translate the
alien's language.
Kirk: Put him on.
Alien: Greetings, Enterprise. We had no sooner transmitted our last
message when we became aware of a fatal oversight in our ECOLI4
simulation; perhaps it is the flaw to which your first officer
alluded in his as-yet-unreceived message. In converting ECOLI3 to
ECOLI4, our simulation engineer had overlooked the code that
integrates dNut over timecycles. Thus we were under the false
impression that there was no carryover of dNut from pre- to
postreinforcement time. Given this false belief, it seemed that
the simulation was proof of principle. We were in error.
When we removed the dNut carryover, our simulation began to behave
exactly as you specified it should.
Spock: It is only logical.
Alien: Yes, we see that now. Apparently we are suffering from having
made too many trips through the transporter room. We now realize,
as your Mr. Spock pointed out, that there is no way for
reinforcement to operate on the tumbling response. This had been
our initial conjecture, but we had come to believe that the three-
term contingency would provide a solution. Having seen the result
of our own corrected simulation, we now know that it does not, and
why. You had tried to explain it to us, but there was
interference in the hyperspace channel and we were not able to
completely decode your message.
Spock: As you are doubtless now aware, the consequences of tumbling are
as likely to produce negative as positive dNut, regardless of the
conditions existing prior to the tumble. Thus there can be no
systematic change in p(Tumble|S+) or p(Tumble|S-).
Alien: Correct, as usual. And that is why, in our initial simulation, we
chose to work with the consequences of forward motion rather than
with the consequences of tumbling. At that time we recognized
that tumbling could not be the response on which reinforcement
would operate. Reinforcement can produce consistent changes in
response probabilities only when the consequences of the response
are reasonably consistent. But tumbling's consequences are
completely inconsistent. It was your insistence that we attempt
to model reinforcement of TUMBLING that led us to thinking that
the addition of discriminative stimuli might provide this
consistency. We now see that it does not.
Spock: Then you agree that the "three-term contingency" of which you
spoke does not work.
Alien: Yes, not when tumbling is modeled as the response on which
reinforcement acts. However, our mistaken simulation with dNut
carryover does nicely illustrate how a control system can learn to
develop and maintain a negative feedback relationship between
tumble rate and dNut. We take it that this is not a feature of
your own e. coli simulation, although we have no doubt that you
could have provided it had you wished to do so.
Kirk: So what do you propose as the next step?
Alien: As we indicated in our previous communication, we will modify our
simulation so that it explicitly models a feedback-regulated
control system that is able to learn appropriate behavior that
will keep its perceptions close to its internal reference level.
Kirk, turning to Spock:
Perhaps there is hope yet for this species, don't you think?
Spock: We shall see, captain, we shall see.
Cheers,
Bruce