[From Bill Powers (920602.0930)] (copy to Chapman and CSGnet)
David Chapman (920601a) --
You might actually find it useful to look at the work of Chris
Atkeson, a roboticist at (I'm afraid to say) MIT, and his students,
particularly Eric Aboaf. They make this same point, and have a model
of skill learning that sounds similar to what you are suggesting here.
(Unfortunately I don't have any cites other than MIT tech reports; you
could write to Atkeson at cga@ai.mit.edu.)
Is anyone "situated" to check this out? I'm feeling sort of unwilling to go
knocking on more doors. Some days the hill looks steeper than others.
You might want to look at the current issue of Machine Learning >Journal,
which I believe is a special issue devoted to TD techniques. >TD
["temporal difference" learning], btw, is pretty closely related to >both
dynamic programming and classical control theory.
Ditto.
Back to Pengi for a moment, David. When I was talking about putting the
intelligence of the modeler into the model, I was talking about things like
this:
"The-block-I'm-pushing
The-corridor-I'm-running-along
...
The-bee-that-is-heading-along-the-wall-that-I'm-on-the-other-side-of."
It seems to me that being able to recognize such things entails a very
complex perceptual system, capable of discriminating, recognizing, and
naming objects (block), processes (pushing), agency (I'm pushing),
relationships (I pushing block), and so on. I realize that we can't model
everything -- we have to use black boxes for what we aren't prepared to
model yet. But it seems to me that you're doing ALL the work for Pengi
instead of just some of it: the perceptions involved are yours, not
Pengi's. You're getting simple behaviors out of a system that's basically
extremely complex, chock-full of the modeler's knowledge about real-world
phenomena. It seems to me that these are descriptions of OUTCOMES of the
model's organization instead of descriptions of MECHANISMS FOR CREATING
THESE OUTCOMES. Perhaps I'm judging on too little evidence.
A couple of years ago I wrote a program for Clark McPhail (sociology) at U
of IL that simulates the movement of actors through a field of obstacles
and other actors. Each actor avoids collisions with other objects and
actors, seeks a goal position somewhere in the field, and at the same time
may seek a particular spatial relationship with another specific actor or a
group of actors. This is a simulation of crowd behavior (pardon me Clark,
behavior in gatherings). But this simulation doesn't contain the idea of
actors or obstacles or collisions or goal positions. Instead, the things a
human observer would classify in these ways appear in behavior that is
based on very simple control processes. The actors simply sense proximities
and adjust speed and direction of motion as a means of controlling several
different simple functions of proximities as perceived by each actor.
1. Each actor is equipped to perceive the sum of all proximities to objects
on the left, and the sum of proximities to the right (where proximity is an
inverse-square function corresponding roughly to retinal image area
subtended by an object).
2. Each actor can sense the direction to the goal (relative to the actor's
direction of travel) and the proximity of the goal (same function as
above).
3. Each actor can sense the left and right proximity to one specific other
actor or the centroid of a subgroup of actors.
4. For each proximity sensed, there is a fixed reference proximity.
Deviations from the reference proximities are reduced by changing turn rate
or changing velocity. There are four control systems of this kind involved
in each actor, of which three can be chosen to operate at the same time.
The result is what appears to be a quite intelligent and complex behavior.
The actors thread their way through a crowd of 50 or 100 (up to 255) other
actors and randomly-placed obstacles, backtracking out of traps, finding
open corridors, and eventually reaching their respective goal-positions
without collisions. If they're seeking proximity to another actor, they
will follow that actor around the field, sometimes following the same path
and sometimes taking short-cuts or running to catch up after a detour
around an obstacle or group of obstacles. You can have chains of actors
following the leader. Two groups of actors independently seeking different
goal positions can thread their way through each other when their general
paths cross, simultaneously avoiding stationary obstacles. You get things
like the after-you-Alphonse effect.
The behavior of any given actor could be described in words as in Pengi:
The-velocity-of-the-person-I'm-supposed-to-follow
The-rate-of-approach-to-obstacles-on-the-left
The-density-of-objects-in-front-of-me
The-nearest-open-space-in-front-of-me
The-shortest-path-to-my-goal
The-nearness-to-a-collision
And so on. Actually, none of these would be very relevant to how an actor
actually works, and most would be misleading (the actors do not detect open
spaces in front of them or "densities", although they seem to. Neither are
they concerned with "collisions," although a consequence of their
organization is that they adroitly avoid all collisions. They do not seek
the shortest path to anything, although sometimes they find it). If one
were trying to model the observed behavior of these actors, the Pengi-like
approach might eventually lead to a similar kind of result, but would do it
through an immensely complex system of rules and logic backed up by complex
information about the environment, where the "real" actors actually work in
terms of a few simple continuous control processes and know only what they
can sense of the environment: proximity to the left and proximity to the
right.
This is the basic difference I've always seen between the CT approach and
AI. In AI, it has seemed to me, the emphasis is on describing appearances
from some human observer's point of view, and then doing logical operations
on these descriptions. The modeling approach behind CT is to look for
simpler underlying mechanisms that would produce the observed appearances,
however they are described and with a minimum of influence from the
observer's verbal inferences. Complex behavior EMERGES from the CT model,
so it can be compared with real behavior for an evaluation of the model. It
has seemed to me that in AI, the starting point is the complex behavior,
expressed as generalizations intended to INCLUDE the actual behavior in a
given instance. That's a very different approach.
Well, it's a shame your facility at Stanford is limited to Unix systems.
Some of our computer demonstrations are real short-cuts to understanding
our approach to CT, PCT, and HPCT. Some day, perhaps, we will be able to
port our demos to Unix systems, but right now most of us work with IBM
clones or Macintosh systems (mostly IBM), because we don't have any
institutional backing for the most part (I don't), and who can afford a
Unix workstation anyway? Not that I'd really want one -- I've been spoiled
by Turbo Pascal and Turbo C. Don't you know any teenagers who would let you
run some programs on their teeny-weeny DOS machines?
Best,
Bill P.