[Martin Taylor 960822 11:20]
Hans Blom, 960822b
... my speculation that there is always a mapping between a
neural-net-like distribution of control (HPCT) and an AI-like
discretization of control (MCT) ...
Can you explain what you mean by the above? Why do you compare HPCT
with neural nets? That is kind of premature, I think, as long as HPCT
misses a mechanism to adjust weights. And why do you call MCT a
discretization?
This is easier than your other question about missing data, which I will
try to discuss in a separate message.
In fact, the formulas of e.g. the Kalman filter are
very similar to those of backpropagation, the difference being that
Kalman filters (usually) do not contain nonlinearities (squashing
functions) and multiple layers.
We are looking at different aspects of the two systems. I am looking at
the structure of the system and you are looking at the content. I see
each individual model as a discrete representation of a separately
identifiable segment of the environment, regardless of whether its
parameter values are continuously variable. It is analogous to a bunch
of interconnected rules that perform some function in an "expert system."
In contrast, HPCT consists, in its pure form, of two interconnected
multilayer perceptrons, one feeding inward from the environment (the
interconnected perceptual functions are the nodes of the MLP), and one
feeding outward toward the environment (the interconnected output functions
are the nodes of the MLP). The two counter-directed MLPs are interconnected
at each node by the comparator functions that cause the inputs to the
outward-directed nodes to differ from the outputs at the higher level,
and they are also interconnected through the environment. It is the
relationship between the two sets of interconnection that defines the
control.
I'm not clear why you talk of HPCT as "missing a mechanism to adjust weights."
Bill Powers long ago experimentally demonstrated reorganization by weight
adjustment in either the inflowing or the outflowing (I think) hierarchy
in a quite complex environment. It's taken as given that an important
form of reorganization is the variation of weights, and indeed I hypothesize
that the ability of the control hierarchy to attribute a quality number
to _each_ node in the net rather than just to the peripheral (non-hidden)
nodes should permit a control hierarchy to learn _much_ faster than can
an equivalent teacher-trained feed-forward neural net with back-propagation
of error.
Bottom line: I compare an HPCT hierarchy with a neural net not by analogy,
but because it _is_ one, with all the benefits and problems that this entails.
It's more complex than a standard MLP, not least because it is assumed that
the "nodes" (perceptual functions) at different levels have characteristics
specific to the level (I'm not at all sure that this specificity is
required, and have so argued in the past, but that's part of the usual
description of HPCT). However that may be, an HPCT hierarchy is every bit
as much a neural net as is any MLP or Hopfield net or...
Martin