Inverse calculations; genetic algorithm

[From Bill Powers (930117.0915)]

Ft. Lewis was down yesterday, so there's an avalanche of
interesting posts today. Also an avalanche in the back yard;
we're back up to 32 inches of snow on the ground.

Avery Andrews (930116.1130) --

RE: Feedback too slow
(Ave to Chris Malcolm)

Presumably the implication of what your saying is that the
armdemo, even running on a very fast computer, couldn't keep up
with a fast & practiced assembly-line worker, at least
if its degrees of freedom were more like those of a real arm).

The arm demo can be tweaked so the arm can go from one
configuration to another in less than 0.1 sec, which is faster
than any human arm can do (the muscles would tear loose). It's
the visual level that slows it down, and I'm sure that could be
fixed, at least well enough to match human performance.

The Arm uses the main degrees of freedom that cause problems: the
big swings about the shoulder, and the motion about the elbow
joint. Rotations about the axis of the upper arm would introduce
some added inertial effects of a smaller magnitude. The other
main degrees of freedom (rotation about the forearm axis, motions
of the shoulder as in shrugging, wrist angle changes) involve
small moments of inertia and would cause no difficulties.

I see no reason to suppose that the human system does any
calculations of inverse kinematics or dynamics at all, under any
conditions.

RE: bumps in McMahon curves (private post): those curves are for
muscle preparations, and involve stretches far beyond what is
physiologically possible with the ends of the muscles attached.

For some experimental and modeling data about muscle properties
in arm movements, see the references that Greg and I used for the
Arm paper:

Winters, Jack. M. and Stark, Lawrence: Analysis of fundamental
human movement patterns through the use of in-depth antagonistic
muscle models. IEEE transactions on biomecial engineering, Vol;.
BME-32, No. 10, (826-839) Oct. 1985

------------; Estimated mechanical properties of synergistic
muscles involved in movement of a variety of human joints;
Journal of Biomechanics. Vol. 21, No. 12, 1027-1041 (1988).

RE: central pattern generation

Obviously, to produce a cyclical pattern the system responsible
must produce a cyclical output to make the reference signals of
lower-level systems change cyclically. But this says no more than
when one points out that in order to move an arm in a certain
trajectory, the muscles must produce a specific pattern of
torques at the joints. The question is not whether those outputs
occur; it's how they are produced. In the usual central-pattern
concept, the output variations are produced by a system designed
to emit the patterns blindly (computing the deviations from the
pattern that are needed to make the final result, rather than the
driving signals, follow the desired path). It's much easier to
create the required driving signals if the final result is
perceived, compared with the desired pattern, and continuously
maintained in the pattern by whatever output variations are
required.

An easily-overlooked property of cyclical behaviors like walking,
chewing, stirring the oatmeal, turning a crank, and so forth is
that the rate of execution can be varied smoothly over the whole
range down to zero speed, and reverse. This means that simple
oscillators or timers won't do the trick. I think that "path"
control is more likely; that a cyclical behavior is simply
behavior that follows a closed path, with continuous control of
where you are on the path. I'm still looking for a good way to
model this. The same problem showed up when we were exploring
cockroach neural nets. Real bugs can slow down all the way to
zero speed and even reverse, without any major change in the
phase relations between leg positions.

ยทยทยท

--------------------------------------------------------------

What about the evidence (discussed by Bizzi in the Posner
volume) that when people point from one place to another, they
calculate an ideal straight path (a virtual path) between the
two points.

They may calculate one, but they don't move in a straight path,
especially if they're moving fast. See

Atkeson,. C. J., and Hollerbach, J.M.; Kinematic features of
unrestrained vertical arm movements; Journal of Neuroscience, Vol
5., No. 9, 2318-2330 (1985).

We matched the arm trajectories in the model to data from this
paper. The trajectories are NOT straight lines.

If you change reference signals for several degrees of freedom at
the same time and at the same rate, the result may be a straight
line or some curve. There are some directions in space where the
arm naturally wants to move the fingertip in straight lines. In
other directions, curves result. Of course if you move slowly
enough, you can trace out any curve you like, including a
straight line but also including paths like your signature.
-------------------------------------------------------------
Gary Cziko (930116.0325 GMT) --

You could make the criterion whatever you want it to be--it
could be how close the programs comes to typing out "PCT
Forever" or coming up with a program to predict closing prices
on the stock market based on certain economic indicators. It
could also be how well a neural net (expressed as a LISP
program) is able to categorize and properly generalize
perceptual inputs. It is NOT the case that "something other
than the final criterion of fitness determines that survival."

I'm not getting my point across. When I refer to external
criteria, I mean external to the model.

Suppose that the program has succeeded in typing out PRT FOREN.
To an external observer, that is clearly better than SSQ !!23Y,
if the goal is to arrive at PCT FOREVER. It is not better if the
goal is to arrive at SSQ QQSS. The observer is calculating a
"distance" from the current output to the desired output, which
means that the observer must already know the desired output.

As a result, the observer becomes part of an environment in which
PRT FOREN allows the organism to survive, whereas SSQ !!23Y gives
it the axe. The environment is now purposive: it has the goal of
seeing PCT FOREVER, and it institutes random changes whenever it
sees that a mutation (or mating) results in an output that is
"farther" from that goal. The selection effects are occurring at
EVERY STEP OF THE WAY, and are goal-directed.

I'm not saying that this won't lead to a successful final result.
It probably will. It's just the E. coli reorganizing principle
with a slightly different mechanism for random variation of the
output. I'm trying to show, however, that this process will work
only if someone or something knows what final state is to be
reached and exerts selection pressures systematically related to
the error between the current output and the desired output. It's
a control process, not blind evolution.

In real natural selection, the ONLY criterion for survival is the
immediate effect of the next mutation or gene exchange. There can
be no "partial credit" for a move in the right direction, if that
move does not ITSELF improve the chances of survival for THAT
generation. My objection to calling procedures with external
criteria "genetic" is that such procedures give credit for
changes that in themselves have no survival value, but in the
eyes of the external observer constitute a step in a direction
that will eventually have survival value. This is quite
appropriate for a control system that has a goal for the organism
thus being selected for (like a cattle breeder). It's not
appropropriate if we're trying to model conventional natural
selection.

Sorry, mutation is not used in genetic programming (and not
much used in genetic algorithms either). "Sexual"
recombination seems to provide all the variation that is
needed.

I don't mean to harp on mutation. All that's required is that the
changes in action be random. If random sexual recombination is
enough to provide all the variation needed, fine. But don't
overlook the control system in the background, comparing the
result of a recombination with a desired result, and eliminating
the offspring whose behavior is judged farther from the desired
outcome.

And it is independent of how close the population (or any given
parent program) is to the goal.

The variations themselves are independent of the distance to the
goal (just as in the case of E. coli). But the offspring who are
allowed to mate are those whose behavior is judged to be closer
to the goal, not those whose behavior eliminates them because of
current consequences. This judgement is the selection process
that guarantees eventual arrival at the goal (even if the judge
switches to a different goal in the middle of the process).

you want the organisms to be able to "know" how well
they are doing so that they can "decide" whether to pass their
genes down intact (things are just fine) or scramble them up a
bit (things are not so good).

Well, yes, I think something like that is required to account for
the efficiency of evolution.

This is not how Koza does it, and it seems to work out quite
well anyway.

Yes it is how he does it, except that the organism that decides
whether to pass the genes along is not the test organism, but
Koza. Koza is simply supporting my view, that goal-directed
evolution is quite feasible and quite efficient.

This has got me wondering whether genetic programming could be
used as a model for reorganization within PCT.

It already is. The principle is the same. The main difference is
in using random gene-swapping to get the random variations in
output that are needed. If behavior is already organized into
chunks that work, each representable by a gene, then it seems
reasonable that organisms should swap working chunks at random
rather than mutating the genes themselves (which also goes on).
The chances of coming up with a new combination that does
something useful would be greater with the gene strategy. But you
still need a reference signal, a perception, and a mechanism for
turning the error into a rejection of outcomes that make the
error larger.
----------------------------------------------------------------
To whomever set the Agre post --

Agre is still trying to figure out what intentions are by looking
at the environment. A lot of his comments make good common sense,
but don't have much theoretical impact.
----------------------------------------------------------------
Avery Andrews (930117.1426) --

My wild guess about trills would go as follows. They are too
fast to be produced by a feedback system tracking kinesthetic
reference levels (e.g. for a feedback system to solve (or
?>better, evade) the dynamics problem). So something else has
to happen. A suggestion is this. alpha-gamma efferents can
(putatively) be driven not only by the outputs of ordinary
comparators, but by what I'll call `fixed pattern generators'
(FPG). Which is a box with an output wire over which comes a
waveform, and an input over which comes an error signal.

That's not so wild; it's the same model I would use, except that
I put one (midbrain or brainstem) level of control between the
pattern generators and the alpha-gamma efferents in the spinal
cord.
                             reference rate
                                   >
                            ----- Comp --->--- (ditto for
                           > > amplitude)
                      perceived rate pattern gen
                              \ |
                                \ --<---
                                 > >
                                  --- ref position-->-
                                 > >
                           perceived pos force output
                                > >
                                 <-mechanical effects-

I have, however, a counterproposal to your somewhat complex
model.

There is probably another system at the top level concerned with
perceived amplitude of the changes in position. So the error
signals from the two top level systems would alter the frequency
of variation in the perceived finger positions to match the
reference rate, and the amplitude of variation to match the
reference amplitude. When the reference signals for position vary
faster than the fingers can reproduce accurately, the resulting
amplitude errors cause the output of the higher amplitude-control
system to increase in amplitude, which makes the reference signal
changes larger. The larger reference signal changes bring the
amplitude of finger movements up enough to satisfy the higher
systems, even though the lower system has large errors. Those
large position errors produce large drive to the muscles, and
make the finger move enough to satisfy the higher systems.

This is an example of how control systems seem to work backward
(as per Primer 1). Imagine that the frequency control system
starts with a low reference frequency that is gradually raised.
The amplitude-control system malkes the pattern generator emit
reference signals that make the finger rise and fall; the
perceived positions track the reference positions closely. As the
frequency increases, however, the perceived positions begin to
vary less than the reference positions. This puts a larger error
into the amplitude control system, which raises the reference
signal fluctuations enough to make the finger excursions just as
large as before. As the reference frequency approaches the limit
of performance, the position reference signals go through larger
and larger excursions while the finger movements are maintained
at the same amplitude. Eventually, of course, the finger
amplitude will not be maintainable and the system as a whole will
have reached its limit of performance.

The implication is that at the level that controls the cyclical
behavior, there is an output function that can be independently
adjusted as to frequency and amplitude of output, with a separate
control system for each dimension.
---------------------------------------------------------------
Greg Williams (930116) --

I think Koza should have subtitled his book "On the Programming
of Computers by Means of Artificial Selection."

Hear, hear! Unless there is a selection criterion somewhere, the
evolving system will just evolve the way it evolves, in a random
walk. Natural selection provides a crude criterion; inner
reference signals would provide much sharper ones.
----------------------------------------------------------------
Gary Cziko (930117.0125 GMT) --

Why is it that if a human is involved in the selection process
its automatically artificial?

Because the human being has a goal for selection, whereas
nonliving nature doesn't.

If a bull finds a attractive (to him) cow (nice hindquarters)
on the pasture and inseminates her, natural selection is
involved.

Nice example: this is goal-directed selection. The bull is
selecting for females that turn him and others like him on. If it
weren't for the bull's preference for nice hindquarters, cows
would not evolve to have nice hindquarters. They would probably
evolve in some direction of more general benefit to cows. The
bull's (unsystematically evolved) preference leads to systematic
selection of the cows' secondary sexual characteristics.
----------------------------------------------------------------
Oded Maler (920117) -- [stuck in last year, I see]

In the higher-levels, maybe the principles of servoing are more
naturally described as a "program". Note that the more modern
notion of a program is not something that gets input and
computes output in front of a static environment, but as a
"reactive" system consisting of many concurrently running
components, having an ongoing interaction with the
envoironment. If you understand "motor programs" in this sense,
the dichotomy between this notion and feed-back seems (at least
to me, without going into the technical details) less

important.

True: once interaction is recognized, the road to control theory
is clear. The other thing about motor programs that needs to be
changed is the idea that they're just pattern generators. What we
need is a generator with inputs that vary parameters of the
pattern. Then the pattern can be controlled in the dimensions
where the perceived pattern can change. Note that the output of
the pattern generator will not in general be the same as the
perceived pattern, because the perceived pattern results from
outputs that have gone through a lot of intermediate processes
and that are subject to disturbance.
-------------------------------------------------------------
Best to all,

Bill P.

---------------------------------------------------------------

[Martin Taylor 930118 15:30]
(Bill Powers 30117.0915)

For once, here is a topic on which I understand and agree with Rick more
than Bill! No comment except approval on Rick's posting, but a minor cavil
on Bill's...

An easily-overlooked property of cyclical behaviors like walking,
chewing, stirring the oatmeal, turning a crank, and so forth is
that the rate of execution can be varied smoothly over the whole
range down to zero speed, and reverse. This means that simple
oscillators or timers won't do the trick.

I'm really not sure that this IS a property of these cyclic systems. As a
poorish pianist, I have certainly found that it is much easier to play
some things at the fastest I can do them than at a SLIGHTLY lower speed.
If I drop the speed appreciably, it seems that it is then reasonably
easy to choose a speed from a continuum, though it does seem that too
slow a speed makes for more difficulty, especially in maintaining a
constant speed. The range over which it is easy to change speed smoothly
seems to me to be quite small, perhaps 2:1 or 3:1. I have always thought
that movements at the really high-speed end reflect the physical mass dynamics,
which relieves internal control systems of the need to control fast, and
at slightly lower speeds the synchronizing effects of the mass dynamics
are all wrong.

Martin