[From Bill Powers (940720.0745 MDT)]
Jeff Vancouver (940718) --
Unfortunately, the E.coli, where you "showed" reinforcement does not
work confirms my straw model argument. Reinforcement is
operationalized as "use the last turn that resulted in denser
foodstuffs" according to your documentation. Why not remember the last
vector. Why not remember the last set of vectors?
Because none of these interpretations results in a working model,
either. We also tried "previous time rate of change of concentration"
and "average rate of change of concentration over past n episodes" and
"difference in average concentration between episodes before and after a
tumble." The basic problem is this: the next direction that will result
from a tumble is always selected at random. So no matter what criterion
you use as the basis for tumbling, it can't be used to select a more
favorable direction of the next tumble. There is no control over
direction, no way to predict what the next direction will be.
The point of this experiment was not to show that "reinforcement doesn't
work." It was to show that the organism can move in the right direction
even when there can be no basis for differentially reinforcing a
"correct response" on the basis of past consequences of responses. Thus
the explanation for the behavior we see has to be something other than
reinforcement theory. The paper offered a PCT model that _does_ work,
the model demonstrating its own correctness by producing behavior very
much like that of E. coli. In this model, E. coli senses the time rate
of change of concentration of the attractant (due to its swimming speed
and direction in the chemical gradient), and varies the delay before the
next tumble according to whether the _current_ sensed rate of change is
above or below a reference setting. The result is that the organism
tumbles sooner when swimming the wrong way and delays the next tumble
while swimming the right way. Previous consequences of tumbles don't
enter into the model at all.
There is no consequence of the _previous_ direction of swimming that has
any systematic relationship to the _next_ direction of swimming, because
tumbles are truly random. Since reinforcement theory depends on _past_
consequences of behavior to shape _future_ behavior, all attempts to
producing a working reinforcement model run into the snag that the
tumbles result in new directions unrelated to the old direction. When
all behaviors are randomly rewarded and punished, reinforcement theory
must predict either random behavior or superstitious behavior, neither
of which will result in the observed behavior of E. coli.
I should mention that the model is based on direct observation of the
behavior of E. coli tethered in a flow of liquid in which the
concentration of perfused attractants could be varied experimentally.
The spacing of tumble episodes was found in this way to be proportional
to the time rate of change of concentration relative to some particular
rate of change (which we took as evidence of a reference setting). The
relevant work is reported in Koshland, D. (1980); _Bacterial Chemotaxis
as a Model Behavioral System_, New York: Raven Press.
The referees who rejected the paper could not reconcile the fact that
the tumbles resulted in random new directions of swimming with the fact
that E. coli found its way very efficiently up the gradient. Most of
them simply refused to believe that the new directions were random
(Koshland cited actual measurements showing a uniform and random
distribution in space). The alternative explanations offered by the
referees all depended on NON-randomness of tumbling, on the existence of
discriminative stimuli which they made up out of thin air (and wouldn't
have helped anyway), or on geometric arguments that simply ignored the
fact of randomness. Their faith was being sorely tested, but they kept
to it in the usual way: by changing the facts or making up facts. They
rejected our paper because according to their beliefs neither E. coli
nor our working model could possibly have behaved they way they did. In
fact, their reactions were a clear-cut proof that reinforcement theory
could not handle this phenomenon (although that didn't help with getting
the paper published).
The other factor that all the refereees shared was that their arguments
were purely verbal; not one of them offered a refutation in the form of
a working model that actually behaved as they claimed such a model could
behave. Not one of them found any flaw in the construction of our
working model. Not one of them commented on the striking similarity
between the plots of the behavior of our model and the behavior of E.
coli. I don't think that any of them actually understood the concept of
a simulation as a way of demonstrating the predictions of a theory. Like
most psychologists, they simply assumed that a working model built to
fit their explanations would actually work as they claimed it would. In
this case, it would not.
···
-----------------------------------------------------------------------
Best,
Bill P.