[Hans Blom, 941130a]
(Bill Powers (941124.1340 MST))
I thought your general remarks about learning and adaptation were well
put and interesting.
Thank you.
My feeling is
that it will help to have a model of the system in its adult state,
where we can see _what_ is learned in its asymptotic state.
Yes, we have discovered this before. Whereas you are more interested in
control, I am more interested in learning/optimization/reorganization, its
laws and its possible implementations.
That should
help us to see the kind of learning that is needed, because we will then
know what it has to produce.
Is that still a question? Isn't it obvious that learning has to produce
(better and better) control? What we learn through the process of learning
is how to deal better with the world around us, i.e. how to better (with
more precision and/or more efficiency) achieve our goals and, possibly,
how to discover more worthwhile goals. Is any discussion about this topic
still necessary?
The most basic aspect of any learning model is a statement of what is to
be learned, and why. We can see that animals will learn all sorts of
behavior that are instrumental in obtaining food pellets, but that only
tells us what they do, not why they do it. Why is obtaining a food
pellet a "good" thing, where is this goodness defined, and how does the
goodness or badness relate to altering the organization of behavior?
Where do you see the "good" and where the "bad"? Certainly not in the
outside world. Maybe in your gut feelings, but those may be difficult to
communicate to someone who has very different ones.
I am not much concerned with what _is to be_ learned. It seems that we can
learn almost anything, however biologically implausible: making fire,
riding a bike, studying control theory, you name it. I am more concerned
with learning as a process: how come that you gradually (or suddenly) do
things better than before? What has changed? What has been discovered?
That does not mean that I find your questions uninteresting. It is just
that I do not believe in deep answers to "why" questions; such an answer
always points to and is an analogy with something that we know already,
i.e. it is a (newly discovered) correlation with something that exists
already.
I also do not believe in models as having explanatory power. A simulation
is, basically, a trial-and-error test which generates outcomes based upon
a selected number of hypotheses and on assumptions about how to connect
them. If those outcomes correlate well with something that we know al-
ready, we tend to say that we have an "explanation". In my opinion, how-
ever, we have only discovered a new analogy. Moreover, the number of
models that may "explain" certain outcomes is infinite. How can we ever
decide which is the "best" one, except in subjective terms of "elegance"
(as mathematicians do in their proofs), "conciseness" (number of symbols)
or some such.
That is maybe the lesson we can learn from Artificial Life simulations.
Here it is much more explicit what is done: select a number of plausible
hypotheses (which usually take the form of "if you perceive X then perform
action Y"), let them operate in parallel, and see what behavior results. I
don't know whether you are familiar with the ALife literature, but the
simulations show some surprisingly familiar results using only very few
and simple hypotheses. The behavior of a chimp male, for example, can be
"explained" (to an 80% fit or so, the authors say) by the following simple
"laws": "if you see another chimp, go find out if it is a willing female;
if so, mount her; if not, go look for food". Quite simplistic, but also
quite convincing that a few simple hypotheses can "explain" seemingly very
complex behavior.
Simulation is basically a form of mathematics. Take Euclidean geometry:
Choose a few axioms (which has, what is it, 5 or 6 axioms?), and devote
the remainder of your life to find out the implications of those axioms,
i.e. develop more and more theorems. Choose a different set of axioms, as
in non-Euclidian geometry (where only 1 axiom is different), and find out
what _those_ imply. But that might take another lifetime. And wherever it
leads, certainly not toward the "truth"...
That does not mean that I do not use simulations. They are valuable in the
discovery what the actual behavior of a designed system is, whether that
behavior fits the system's specifications, and whether the behavior is
"surprising" (unexpected) under certain conditions. The type of reverse-
engineering that is your objective is awfully difficult if one has to
treat the system as a black box and is not allowed to investigate the
insides. The engineering problem is this: given the specifications of a
system (however formulated), an infinite number of designs is possible. So
which design to choose? In practice, the design seems always based on the
methods that the engineer is most familiar with...
This may be what reorganization basically is: if you have a problem
that must be solved but you don't know its solution, perform random
actions and hope for the best...
Well -- hope for an outcome that will reduce some errors. It's only
because of error that you reorganize.
Yes, that is how we define reorganization, of which there seem to be two
types: hill-climbing, as in your ecoli simulation, which has a built-in
method of how to fine-tune parameters (as Bruce notes, this is better not
called learning but hierarchical control); and trial-and-error learning,
as in Bruce's ecoli simulation, where a built-in hill-climbing procedure
does not exist and which hence is less efficient but probably more gene-
rally applicable. This latter type of reorganization lacks built-in
knowledge of how to reach the optimum, but has a built-in more or less
random "action generator" or "sub-goal generator" that, through experi-
ments, is capable of discovery. Both could be called "active" kinds of
learning, because the organism employs actions to achieve some goal.
But there seems to be a third type. Let me give you an example. For many
years I rode my bicycle to the university along the same route, until once
I met and started to accompany a colleage, also on bike, who led the way
ALONG A DIFFERENT, SHORTER ROUTE. After that, I started to travel the
newly discovered way. Did I have an error before? Not that I know of. Did
I look for a better route? No, I accidentally stumbled upon one. This
process is different from control and the active type of reorganization in
the face of error, I think. It is not even trial-and-error learning,
because there are no trials. Yet this process of accidental discovery and
its subsequent use is a type of learning that occurs in me (this example),
as well as in some artificially intelligent systems.
But it is just as true to say that the existence of that contingency
has no ability to create the behavior that will satisfy it.
After a while it will. Once the cat has discovered the condition, it
will USE it.
Why is this point so damned hard to see? The contingency acquires no
properties it did not always have; it doesn't change. The _cat_ changes,
the _cat_ acquires new abilities. If it were not for the changes in the
cat, the contingency would not SEEM to have any new effects at all. And
the critical word is SEEM. In fact, the contingency is never anything
but a causal relationship in the physical world: if the cat's behavior
causes event A, event B will happen. And that is all that will EVER
happen (barring physical changes in the environment), whether the cat
decides to use that contingency or not.
Of course the contingency acquires no new properties. I thought I men-
tioned that. Of course it was the _cat_ that changed. We do not disagree
at all, although you seem to think so. But you do not go far enough. There
are no "causal relationships" in the physical world; they are internal in
us. Why is _this_ so hard to see? It is all perception, remember? The
world just IS. Causal relationships come into being when and because they
are discovered BY AN ORGANISM. There is no F = m * a in the outside world;
it is in our internal "world model". Similarly, there is no relation "if
the cat's behavior causes event A, event B will happen" in the physical
world; that as well exists only in our internal world model. It's all
perception, remember?
... it's really irrelevant whether you phrase it as selection
BY consequences or selection OF consequences. ... The important fact
is that the perceived consequences of behavior determine which
behaviors are retained and which are eliminated.
Hans:
Exactly.
What about reference signals? AAARRRRGH!
How about the above example? Did I have a reference signal for following a
shorter route to the university? Yet the perceived consequence of an
action led me to change my behavior. Some learning seems not to depend on
reference levels that are a priori given, but leads to the installation of
a new reference level (or a new control system?) _after_ a discovery.
Are things getting to complex? Am I missing something?
Greetings,
Hans