Alife models; Astro with sqrt

[From Bill Powers (920807.0700)]

Eric Harnden (920806) --

RE: Alife research:

... certainly, at this stage in the game there are a lot of >simplifying

assumptions being made, not the least of which is the lack >of certain
kinds of disturbance in the medium in which order is to >manifest itself.
and for certain directions of research, these will >continue to be valid

... if that which is perceived (the effect of action on the >environment)

is perceived without distortion (no noise, no disturbance, >no delay), and
if the environmental response to action is at the very >least deterministic
(possibly linear, certainly consistent, maybe even >just transparent),then
isn't the perception of the outcome equivalent >to the perception of the

Well, you've defined the problem quite clearly. Simplifying assumptions are
always necessary in modeling; if they couldn't be used, Schroedinger's
equation would be the basis of all models of everything (and even it would
be an approximation). Some simplifying assumptions, however, are more
critical than others -- especially if they happen to be false to fact in an
important way.

If perception could be equated to the objective physical state of the
environment, and if regular effects on the environment could be traced back
to regular outputs of the nervous system, cause-effect models such as the
stimulus-response model would work perfectly well. That is why they were
invented -- because everyone believed in exactly the simplifying
assumptions you propose. Everything I have seen in AI or AL tells me that
these assumptions are still being made. Without them, there's hardly any
proposition about living systems that would work, those propositions that
somehow entail closed-loop control processes aside.

In fact neither basic assumption is correct; neither the one concerning
input nor the one concerning output. Perceptions do not correspond
accurately (and often do not correspond at all) to objective events or
states, and outcomes of action are almost never regular functions of the
outputs of the nervous system. No theory is needed to prove either
statement. All that's necessary is to stop making these assumptions and
look at real behavior.

The easiest proposition to falsify is the one saying " ...the environmental
response to action is at the very least deterministic (possibly linear,
certainly consistent, maybe even just transparent) ..". The environmental
response to organismic actions can often be modeled as linear, but it can't
often be modeled correctly as deterministic or consistent, and never can be
modeled as transparent. Action does not determine outcomes, nor are the
outcomes of a given action consistent. A given action can easily have
opposite effects on an outcome from one trial to the next. And the
connection between all actions and their consequences is mediated by
properties -- usually variable properties -- of the external world.

One has to search for very special circumstances to find cases in which a
given action always has the same outcome, or in which a given outcome is
repeatable by repeating the same action. In fact, one has to set up a
laboratory experiment in which all normal variations in the environment are
suppressed, in which most normal causal links in the environment are
missing and the remaining ones remain precisely repeatable, and in which
the organism itself always begins the experiment in the same state. The
purpose of setting up such laboratory conditions is precisely TO MAKE THE
ASSUMPTIONS APPEAR TRUE. And even when such conditions are established,
variability still appears. All attempts to force organisms to behave
according to these "simplifying" assumptions have failed. Yet those who
purport to be studying the behavior of living systems have stubbornly
insisted that the assumptions must be correct.

If they aren't correct, all theories that assume their correctness are
simply wrong. They aren't just a little off because of making some
simplifying assumptions. They're qualitatively wrong and their predictions
are quantitatively as well as qualitatively wrong. Real organisms in real
environments would not behave as models that make these assumptions behave.

If you'll examine any behavior without making these simplifying
assumptions, you'll see that that there's a clear answer to your question,
"isn't the perception of the outcome equivalent to the perception of the
action?" The answer is "No." It's OBVIOUSLY "no," once you look carefully
at the way any behavior is produced.

The primary way in which people have made the answer seem to be "yes" is by
naming behaviors not in terms of actions but in terms of outcomes. When you
see someone hammering a nail into a board, you're seeing the outcome of a
series of efforts applied to a hammer, and you're naming the action by its
outcome: the hammer hits the nail and the nail sinks into the wood. Neither
of those processes is the action of the carpenter; they are environmental
processes. If the carpenter generated exactly the same motor command
signals over and over, the hammer would miss the nail most of the time.
When you say a person is steering a car down a road, you're referring to
the fact that the steering wheel somehow turns in just the way needed to
keep the car on the road; you're naming environmental processes, not the
actions by which the driver is making those processes occur. In the case of
steering a car, it's clear that the efforts applied to the steering wheel
are almost unrelated to the car's observed path or even the position of the
steering wheel. If the efforts were repeated exactly, the car would be in
the ditch in a jiffy.

When disturbances are omitted from a behavioral model, this is not just a
simplifying assumption. Omitting disturbances conceals the fundamental
feature of a control system. A model designed specifically to work in an
environment where outcomes and actions are the same thing will not just
work a little worse when disturbances are introduced: it will not work at

In normal environments, disturbances have at least as much effect on
outcomes as actions do; they often require reversing an action in order
that the same outcome repeat. In most behaviors, the PRIMARY reason that
continued action is needed is that disturbances need to be counteracted. If
you were driving a mechanically perfect car down a perfectly straight level
smooth road in absolutely still air, you could just aim the car in the
right direction with a telescopic sight, clamp the steering wheel, and then
read a book for the rest of the trip. In most real environments, outcomes
are not simply proportional to actions; they involve time integrals and
other processes extending through time. Even small errors in action are
magnified to create large differences in outcomes.

Even without disturbances, real outcomes of real actions are variable; they
are hypersensitive to initial conditions. Models that try to do without
negative feedback usually have to employ calculations of extreme precision
in order to continue working for even modest lengths of time; if required
to act continuously, each action starting where the previous one left off,
the calculations will depart further and further from reality, just as
Lorenz found in his weather models. In one mode of my arm model, the Little
Man points to a target that jumps randomly about in space every 3/4 second.
I can leave it running overnight (and have done so), and in the morning
it's behaving just as it was when I went to bed, despite the fact that the
integrations are done in 16-bit integer arithmetic. If you tried that with
any of the open-loop models being proposed, even using 80-bit floating
point arithmetic, you'd probably awaken to find an overflow error on the
screen and the Little Man with his arm wrapped around his neck.

I strongly reject the eclectic approach you imply when you say

different features of interest require, for the moment, different
experimental approaches. and if the bug walks, or the swarm gathers, or
the predator kills, or the eye tracks, or the perceiver controls... >then

the model is informative.

This approach treats all theories as if they were indistinguishable. In
psychology perhaps there's a good reason for this: all theories are equally
bad, so it doesn't matter much which one you use -- you'll still get
correlations of 0.5 or so. But in modeling behavior, one can easily choose
between models that predict behavior in the real world and those that
predict it only in a world based on assumptions known to be false. Control
theory clearly belongs with theories that allow for perception to be a
construction rather than a veridical report on reality, and for action to
be only one influence on outcomes among many. I don't know exactly how many
theories fall in this class, but my guess is 1. All the other theories are
based on a known falsehood; I find no trouble with rejecting them simply
for that reason.

If you can think of an example of a real behavior in which outcome is a
quantitatively regular function of action, I'd like to hear about it.


Avery Andrews (920806) --

I tried the square-root substitution. With dt = 0.1, k1 = 3, and k2 = 5,
the behavior goes like this:

Mother Astro Astro Astro-Mother
Pos'n Pos'n Accel vel

mx: 10.2 ax: 0.0 aa: 1.0 av: 0.1
mx: 10.4 ax: 0.5 aa: 48.4 av: 4.9
mx: 10.6 ax: 1.2 aa: 23.5 av: 7.3
mx: 10.8 ax: 2.1 aa: 10.5 av: 8.3
mx: 11.0 ax: 2.9 aa: 3.7 av: 8.7
mx: 11.2 ax: 3.8 aa: 0.1 av: 8.7
mx: 11.4 ax: 4.7 aa: -1.8 av: 8.5
mx: 11.6 ax: 5.5 aa: -2.7 av: 8.3
mx: 11.8 ax: 6.3 aa: -3.2 av: 7.9
mx: 12.0 ax: 7.0 aa: -3.5 av: 7.6
mx: 12.2 ax: 7.8 aa: -3.6 av: 7.2
mx: 12.4 ax: 8.5 aa: -3.6 av: 6.9
mx: 12.6 ax: 9.1 aa: -3.6 av: 6.5
mx: 12.8 ax: 9.7 aa: -3.5 av: 6.2
mx: 13.0 ax: 10.3 aa: -3.5 av: 5.8
mx: 13.2 ax: 10.8 aa: -3.4 av: 5.5
mx: 13.4 ax: 11.4 aa: -3.4 av: 5.1
mx: 13.6 ax: 11.8 aa: -3.3 av: 4.8
mx: 13.8 ax: 12.3 aa: -3.2 av: 4.5
mx: 14.0 ax: 12.7 aa: -3.0 av: 4.2
mx: 14.2 ax: 13.1 aa: -2.9 av: 3.9
mx: 14.4 ax: 13.5 aa: -2.8 av: 3.6
mx: 14.6 ax: 13.8 aa: -2.6 av: 3.4
mx: 14.8 ax: 14.1 aa: -2.4 av: 3.1
mx: 15.0 ax: 14.4 aa: -2.2 av: 2.9
mx: 15.2 ax: 14.7 aa: -1.9 av: 2.7
mx: 15.0 ax: 14.9 aa: -3.7 av: 2.3
mx: 14.8 ax: 15.1 aa: -8.2 av: 1.5
mx: 14.6 ax: 15.1 aa:-16.3 av: -0.1
mx: 14.4 ax: 14.9 aa:-10.6 av: -1.2
mx: 14.2 ax: 14.8 aa: -6.2 av: -1.8
mx: 14.0 ax: 14.5 aa: -3.3 av: -2.1
mx: 13.8 ax: 14.3 aa: -1.5 av: -2.3
mx: 13.6 ax: 14.1 aa: -0.5 av: -2.3
mx: 13.4 ax: 13.9 aa: 0.1 av: -2.3
mx: 13.2 ax: 13.6 aa: 0.4 av: -2.3
mx: 13.0 ax: 13.4 aa: 0.5 av: -2.2
mx: 12.8 ax: 13.2 aa: 0.5 av: -2.2
mx: 12.6 ax: 13.0 aa: 0.4 av: -2.1
mx: 12.4 ax: 12.8 aa: 0.4 av: -2.1
mx: 12.2 ax: 12.6 aa: 0.3 av: -2.1
mx: 12.0 ax: 12.4 aa: 0.2 av: -2.0
mx: 11.8 ax: 12.2 aa: 0.2 av: -2.0
mx: 11.6 ax: 12.0 aa: 0.1 av: -2.0
mx: 11.4 ax: 11.8 aa: 0.1 av: -2.0
mx: 11.2 ax: 11.6 aa: 0.0 av: -2.0
mx: 11.0 ax: 11.4 aa: 0.0 av: -2.0
mx: 10.8 ax: 11.2 aa: 0.0 av: -2.0
mx: 10.6 ax: 11.0 aa: 0.0 av: -2.0
mx: 10.4 ax: 10.8 aa: -0.0 av: -2.0
mx: 10.2 ax: 10.6 aa: -0.0 av: -2.0
mx: 10.0 ax: 10.4 aa: -0.0 av: -2.0
mx: 9.8 ax: 10.2 aa: -0.0 av: -2.0
mx: 9.6 ax: 10.0 aa: -0.0 av: -2.0
mx: 9.4 ax: 9.8 aa: -0.0 av: -2.0

So a nonlinear comparator doesn't make much difference. If you double the
loop gain the behavior near zero error will be about the same (sqrt(x)
expands to approximately x/2 for small x); the gain will be lower for
larger errors, but if the minimum loop gain is high errors will never
become large.

One thing to watch out for when you're reorganizing this or any other
system that's simulated on a computer. When the loop gain gets high enough
you can start getting computational artifacts due to the discrete nature of
the variables. The value of dt should always be made small enough so it
takes several iterations -- five is a nice number -- to make the fastest
change in a variable. Sometimes computational artifacts can be confused
with real instability of the system.

When you say that k2 fluctuates around .55, I assume that you're using a dt
of 1. If you use a smaller dt, you may find that k2 can become a lot
larger. You may be looking at a computational artifact (but the fact that
reorganization can compensate for it is interesting anyway).

Inertia really isn't much of a problem for control systems in a
frictionless environment. If Mother and Astro were in orbit, on the other
hand, the problem might become more complex, because you don't move in the
direction of thrust relative to another object in the same orbit. The most
difficult environments, in terms of physical properties, are those
involving slip-stick friction, thresholds, and limits.


As for Alife, etc: Many of these systems (and also Chapman & Agre's
video-game playing programs) do model significant aspects of keeping
oneself alive (that's why video games are fun). So either they are in
fact full of control systems, perhaps to a greater extent than their
creators realize, or they are leaving out aspects of reality for which
control systems are essential. Either way they provide lots of stuff >for

people to do, either in the way of improving our understanding of >how they
work, or in making them more lifelike, or both.

My problem with this generous view is that it's hard to know when such
models tell us something about life and when they just inform us about the
consequences of playing an arbitrary game. I'm all for video games, but
before I can accept any of them as models of living systems, I want to see
some explanation of why the rules are relevant to something about living

In a lot of models (like Pengi) the critical part that makes them work
isn't even in the model; it's in the modeler.
Best to all,

Bill P.