# Reinforcement theory vs E. coli

**URL:** <http://discourse.iapct.org/t/reinforcement-theory-vs-e-coli/9713>\
**Category:** CSG1994\
**Created:** [July 20, 1994, 2:53pm UTC](http://discourse.iapct.org/t/reinforcement-theory-vs-e-coli/9713 "1994-07-20T14:53:54Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![William\_T\_Powers2](http://discourse.iapct.org/letter_avatar_proxy/v4/letter/w/c67d28/32.png) [@William\_T\_Powers2](http://discourse.iapct.org/u/William_T_Powers2)\
**Post date:** [July 20, 1994, 2:53pm UTC](http://discourse.iapct.org/t/reinforcement-theory-vs-e-coli/9713/1 "1994-07-20T14:53:54Z")

</div>

[From Bill Powers (940720.0745 MDT)]

Jeff Vancouver (940718) --

> Unfortunately, the E.coli, where you "showed" reinforcement does not  
> work confirms my straw model argument. Reinforcement is  
> operationalized as "use the last turn that resulted in denser  
> foodstuffs" according to your documentation. Why not remember the last  
> vector. Why not remember the last set of vectors?

Because none of these interpretations results in a working model,  
either. We also tried "previous time rate of change of concentration"  
and "average rate of change of concentration over past n episodes" and  
"difference in average concentration between episodes before and after a  
tumble." The basic problem is this: the next direction that will result  
from a tumble is always selected at random. So no matter what criterion  
you use as the basis for tumbling, it can't be used to select a more  
favorable direction of the next tumble. There is no control over  
direction, no way to predict what the next direction will be.

The point of this experiment was not to show that "reinforcement doesn't  
work." It was to show that the organism can move in the right direction  
even when there can be no basis for differentially reinforcing a  
"correct response" on the basis of past consequences of responses. Thus  
the explanation for the behavior we see has to be something other than  
reinforcement theory. The paper offered a PCT model that \_does\_ work,  
the model demonstrating its own correctness by producing behavior very  
much like that of E. coli. In this model, E. coli senses the time rate  
of change of concentration of the attractant (due to its swimming speed  
and direction in the chemical gradient), and varies the delay before the  
next tumble according to whether the \_current\_ sensed rate of change is  
above or below a reference setting. The result is that the organism  
tumbles sooner when swimming the wrong way and delays the next tumble  
while swimming the right way. Previous consequences of tumbles don't  
enter into the model at all.

There is no consequence of the \_previous\_ direction of swimming that has  
any systematic relationship to the \_next\_ direction of swimming, because  
tumbles are truly random. Since reinforcement theory depends on \_past\_  
consequences of behavior to shape \_future\_ behavior, all attempts to  
producing a working reinforcement model run into the snag that the  
tumbles result in new directions unrelated to the old direction. When  
all behaviors are randomly rewarded and punished, reinforcement theory  
must predict either random behavior or superstitious behavior, neither  
of which will result in the observed behavior of E. coli.

I should mention that the model is based on direct observation of the  
behavior of E. coli tethered in a flow of liquid in which the  
concentration of perfused attractants could be varied experimentally.  
The spacing of tumble episodes was found in this way to be proportional  
to the time rate of change of concentration relative to some particular  
rate of change (which we took as evidence of a reference setting). The  
relevant work is reported in Koshland, D. (1980); \_Bacterial Chemotaxis  
as a Model Behavioral System\_, New York: Raven Press.

The referees who rejected the paper could not reconcile the fact that  
the tumbles resulted in random new directions of swimming with the fact  
that E. coli found its way very efficiently up the gradient. Most of  
them simply refused to believe that the new directions were random  
(Koshland cited actual measurements showing a uniform and random  
distribution in space). The alternative explanations offered by the  
referees all depended on NON-randomness of tumbling, on the existence of  
discriminative stimuli which they made up out of thin air (and wouldn't

have helped anyway), or on geometric arguments that simply ignored the  
fact of randomness. Their faith was being sorely tested, but they kept  
to it in the usual way: by changing the facts or making up facts. They  
rejected our paper because according to their beliefs neither E. coli  
nor our working model could possibly have behaved they way they did. In  
fact, their reactions were a clear-cut proof that reinforcement theory  
could not handle this phenomenon (although that didn't help with getting  
the paper published).

The other factor that all the refereees shared was that their arguments  
were purely verbal; not one of them offered a refutation in the form of  
a working model that actually behaved as they claimed such a model could  
behave. Not one of them found any flaw in the construction of our  
working model. Not one of them commented on the striking similarity  
between the plots of the behavior of our model and the behavior of E.  
coli. I don't think that any of them actually understood the concept of  
a simulation as a way of demonstrating the predictions of a theory. Like  
most psychologists, they simply assumed that a working model built to  
fit their explanations would actually work as they claimed it would. In  
this case, it would not.

> **···**
>
> -----------------------------------------------------------------------  
> Best,
> 
> Bill P.