Ecoli6: a nifty simulation

[From Bill Powers (941107.1205 MDT)]

Bruce Abbott (941104.1000 EST) --

As you will see when you get my post, there is yet another problem with
Ecoli4. But I am glad to see that the aliens are proving to be an
intelligent race after all.

As to Ecoli6, I love it. The little guy zooms in toward one food source,
get full, zooms around some more, goes back, and eventually discovers
the other food source and does the same with it, back and forth.
Everything works for a reason, and the reasons produce the behavior
you'd expect, and a couple of surprises for a bonus. Every variable,
signal, and constant has physical significance in relation to the
physics of the environment. This is modeling the way it should be done.

Now what you need to do is subtract the nutrient absorbed from the food
sources, so eventually they get used up; when one is gone, E. coli will
concentrate on the other until it is gone too, then it will randomly
depart from the screen for greener pastures.

It doesn't bother me that E. coli may not be organized this way. The
point is that you have a model that behaves sensibly, and its behavior
can be compared with the real behavior under various circumstances. If
the model does everything the real E. coli does, we're as close as we
can get. If it doesn't, we have information about how the model needs to
be changed.

There's one more variation, which I doubt that E. coli can do but which
illustrates learning of a different sort, reorganization. I'll describe
it and you can try it on Ecoli 5 or 7 (was 5 skipped on purpose?).

Suppose you take the error from the second-level system, square it, and
subtract it from the old squared error. This gives you roughly the time
derivative of e^2, or 2*e*de/dt. If this number is positive (meaning
squared error is increasing), generate a random value between 0 and 1,
multiply it by e^2, and store the result in DG.

On every iteration add DG to G, the gain of the first-level system. You
will want some very small multiplier k, so that G := G + k*DG*e^2.

The value of G keeps being changed by DG on every iteration. If G is
changing the wrong way, the rate of change of e^2 will be positive, and
another change in DG will occur right away. Otherwise the same value of
DG will just keep adding to G, by an amount that depends on the square
of the second-level error.

You have to play with k to make changes slowly enough, but not so slowly
that nothing ever happens. Better use doubles instead of reals. You
might also need to average e^2 over n iterations and calculate de^2/dt
every nth time, to keep the reorganization from happening too fast.

The result should be that if you start out with any value of G (even the
wrong sign), eventually the E. coli will learn the best value of G for
going up the gradient. What it is learning is not a behavior or a
behavioral relationship, but a _parameter_ of its own organization. The
result of the learning will be to miminize the second-level error on the
average, the level of nutrition inside E. coli.

We've used this method in a number of applications, but so far never in
the E. coli model. If you try it out, it will be the first time. Think
it will work?

ยทยทยท

-----------------------------------------------------------------------
Best,

Bill P.