[From Erling Jorgensen (2005.01.21 1310 EST)]
Bill Powers (2005.01.21.0715 MNST)
Erling Jorgensen (2005.01.20 1430 EST)
Very helpful post, Bill.
Do you really mean “cumulative” error? For error to accumulate,
there must be an integrator to accumulate it.
Sloppy language on my part. I meant something more like “composite”
error or “global” error in the system at a given point in time.
At least that would be my first-pass approximation. Your post
goes on to describe monitoring error (and thus, directing
reorganization) at a much more local level. That’s a feature
I have been interested in, too. But I have wanted to move toward
it cautiously – i.e., because we “must” add it to the model, not
because we “can”.
The issue is this: How is an “extreme enough” error signal
different from any other high magnitude or high frequency
signal in the brain?
It isn’t different as far as I know. It’s just a large signal.
…its meaning is that control is not working very well.
Yes, this is my point. And “meaning” here is a function, not
of a particular segment or signal in the loop, but of how it
functions in the loop as a whole. As I said, “error” (per se)
is at a different logical level of analysis.
But your remarks remind me that error is a unique statement
about the state of control, at that composite level of analysis.
I had forgotten that large error is not needed to derive
large output (to counteract the effects of large disturbance).
We have a gain parameter, which if high enough, can derive large
outputs from even a small amount of error.
And that means, as you go on to point out, that we have a
signal uniquely situated to improve the efficiency of
reorganization – without any need for “intelligent” agency
as to the content of what particular errors represent.
But let’s consider the alternative. Suppose reorganization
were caused by any large signal, wherever in the loop it
appeared… Thus any control system that could move the
arm to one extreme position would have its organization
changed until it could no longer do that.
I agree, this is not a viable alternative. And if evolution
had attempted it, it likely would have been deselected (by
lack of survival) or reorganized away. Control systems need
the ability to counteract the perceptual effects of large
disturbances.
Therefore reorganization shouldn’t be started by ordinary
levels of error, or should proceed only slowly for small
errors.
Yes, and presumably slow integrators or threshold parameters
would be ways to simulate that requirement in a model. Let’s
also not forget Martin’s suggestion – maybe error shouldn’t
trigger any reorganization, until other intrinsic variables
are affected.
All that a comparator has to do is subtract the magnitude of
one input from the magnitude of another and generate an output
proportional in magnitude to the difference between the input
magnitudes.
Yes, that is the core feature of comparators, (together with
an additional feature that you mention later.) My problem
with that (until I read later in your post) was that all sorts
of neurons get both excitatory and inhibitory input, and
generate output proportional to the difference. I don’t think
every neuron is functioning as a comparator. That’s where
your additional feature from later in your post comes in –
The basic specific feature is that a comparator receives
one signal that originates lower in the afferent tree and
subtracts it from (or from it) a signal that originates
higher in the efferent tree (afferent = inbound, efferent =
outbound).
Outbound minus inbound (or vice versa), that would seem to
be the key. With that caveat, I think your (earlier)
statement is now correct –
In principle, all comparators are therefore alike. The
comparator is the only function in the loop of which this
is true.
And for us to capitalize on that feature (say, with some
kind of error-monitoring system), it has to be true of
the neuro-anatomy, too – at least in a broad functional
sense.
So the question is, is there something about neural
development that allows certain neurons or ganglia of
neurons to differentiate inbound from outbound? The
prediction of HPCT would be “yes”, if we indeed allow
this feature of comparators to do some of the heavy
lifting, with a meta-monitoring system.
In the brain stem, cells in the olivary nucleus receive
excitatory collaterals from the sensory pathways, and
inhibitory efferent snals from nuclei of the cerebellum
and other sources… the layer is a layer of comparators.
… And of course, the motor cells of the spinal cord are
all comparators.
These are useful anatomical examples from the developed
nervous system. They do not address the inheritabililty
requirement, for how they might have developed. As you
say –
One requirement I have placed on intrinsic control systems
is that they be inheritable. The reason for this is that
to get the hierarchy started from scratch, there must be a
reorganizing system in place and functioning before any
hierarchical systems exist.
So, to specify the prediction a bit finer: In the developing
nervous system, there would need to be local indicators
(perhaps controlled chemical variables?) such that the
structures that become comparators receive at least one
input each from the collaterals that later are seen to
originate in efferent and afferent pathways, respectively.
If that prediction is borne out by subsequent discoveries,
we should have no problem retaining this aspect of the
model that we are in the process of devising… <:->
The core of the argument is this, with emphasis on the if
it were possible:
So if it were possible for an inherited reorganizing system
to know which signals were going to be error signals, error
signals could function as intrinsic variables with respect
to the reorganizing system… If comparators can be built in,
the reorganizing system can have inherited connections to
comparators that detect their outputs. And that means that
large errors can cause reorganization even if large signals
of other kinds can’t.
A lot hinges on the uniqueness of comparators, (recognizable
to a developing nervous system, controlling something about
the local environment without recourse to “intelligent
overview”). That is why your statements about comparators in
your earlier post leaped out at me.
Why not just say that the reorganizing system acts to
reorganize the control system where the error is? The
simple answer is because that’s not enough to account for
all the kinds of learning we see. My stock example is
the bird… [etc.]
The other answer is that the phrase “where is the error is”
is exceedingly ambiguous. Error is a property of loops,
(primarily). So where on the loop should the reorganization
be directed? There’s a sensory portion, and a CNS portion,
and a motor output portion, and an environmental feedback
portion. Moreover, every higher level variable includes
a good portion of the lower levels to complete its loop.
It is tempting to call it a “virtual loop”, just to
emphasize this hierarchical feature – but, of course, its
connections are just as real, no matter how many other
loops it incorporates into its path.
Now in this discussion, we are starting to make error a
property of a specific location – i.e., whatever emerges
from an identifiable comparator. But I don’t think we want
reorganization targeted there, because a net change of
sign (inserting an inhibitory interneurone?) might lose
us negative feedback itself.
Your multiple-control-system model is a fascinating
exploration of one place to target the reorganization –
‘let’s make it the input function’.
I agree with your first conclusion, what I would call a
proof of principle –
Result: the input functions become more and more
orthogonal to each other… This shows that random
reorganization can lead to systematic independent
control – if each control system reorganizes separately.
This is quite a demonstration. “Systematic” can emerge
from what is “random”. It maybe implies, in passing,
that randomness is not something to fear – it does not
mean random outcomes, just random attempts.
The second thing I am struck by is the orthogonal aspect
itself. It seems that one way for each reorganizing
control system to minimize the effects of disturbances
from all the other controlling systems is to, in effect,
create maximal distance for itself in the perceptual
state space. We could say it creates a perceptual niche(!),
in which conflict with other control systems is minimized.
This seems to me an extremely significant finding.
I am further intrigued by the matrices you describe.
the matrix of input weights for each system approaches
the transpose of the matrix of output weights for each
system.
Would the transpose still happen if the outputs could
only affect a subset of the 500 environmental variables
(say, 200 or 300)? [I’m not sure if that would allow
enough degrees of freedom for stable control to be
established.] I have this vague image of the weightings
being mirror images, and am wondering if there is any
artifact coming from each system being derived from and
affecting all 500 variables. I probably do not understand
enough about transpose matrices, to even raise the
question. (When I ran a version of your simulation
on-line sometime last year, I got a little confused
about how to understand what was happening.)
More to the point, in the present discussion, is your
finding about localized reorganization –
…if each control system reorganizes separately. Trying
to make them all reorganize according to the global total
error does not work anywhere near as fast, if indeed it
works at all (I can’t tell yet). This says that we need
something to direct reorganization to the systems where
the error is.
This suggests there might be a decided evolutionary
advantage to such directed reorganization. Such that –
if it arose, it would tend to persist.
I also take this as one form of evidence, addressing my
earlier reservation –
Doesn’t this introduce an additional property (ripe for
exploitation!) into the model? What data does this allow
for,…?
You’ve inserted a single (and plausible) constraint –
let reorganization specifically affect “input functions
…according to the magnitude of [the control system’s]
error signal”. What emerges, in this demonstration,
are orthogonal perceptual niches. That is a very
intriguing finding.
At present, the only thing I know of that can focus on
systems with error is attention. So, in an intuitive leap,
I have guessed that the function of attention (central
awareness of specific control processes) is to direct
reorganization.
I have been guessing that emotion might play this role.
But I think your evidence may be better than mine. –
If you attend too closely to any skilled control action,
it tends to get disorganized. If there is a problem with
control, attention focuses on the systems with the largest
error, and they begin to reorganize.
Experientially, at least in general terms, this seems to
be true. Whereas I do not think we could make a similar
first-approximation generalization to situations associated
with emotion.
I still think there is some kind of semi-hard-wired
association between different emotions and the rate of
change of error (whether global or local, I’m not sure.)
And I note your cautions, such that –
The same relationships would be seen if emotions were just
the experience of errors together with the sensations of
getting prepared to deal with them. The idea that emotions
are causal or directive is unnecessary.
I agree there is no necessity. And I agree with you (and
James and Lange) that a big part includes the body’s “sensations
of getting ready to deal with” errors.
What I wonder about is the apparent “experience of errors” part.
With everything else, according to the postulates of PCT,
we only experience “perceptions.” And yet, here, when it comes
to emotion, I believe there is a more direct experience of
changing errors. This might be an illusion. It might all be
mediated through various kinds of proprioception. Or it
might be the one place where we experience more than our
perceptual inputs.
If so, if this latter intuition is possible, what might be
the evolutionary reason for such an arrangement? Is there
some kind of functional advantage, that would not mess up
a hierarchy functioning well in other respects?
My suspicion is that emotion might involve some kind of
reversible amplification feature. Driving the reorganization
of the intrinsic system is just not plausible, because (as
you note) that has to work prior to the appearance of the
hierarchy, to make the hierarchy appear in the first place.
And your intuition about awareness is a better candidate, for
most reorganization that does not explicitly channel through
the intrinsic hierarchy. Perhaps emotion has something to
do with your category of “learned reorganization”…
And we shouldn’t forget that there are probably other means
of reorganization that are learned: control by changing
parameters as opposed to changing reference signals is
certainly a means of reorganization, and can be acquired.
Some of my explorations in the past have certainly gone the
route of how the “gain” parameter might be adjusted, and
whether emotion might be tied in with that. That resonates
intuitively for me with both positive and negative emotions
– akin to “tuning up or tuning down” our responsivity in
different situations.
I whole-heartedly agree with your parting comments –
I think the main thing is to avoid introducing new features
into the model until we’re sure that the existing structure
can’t handle some phenomenon.
I think this methodological constraint has resulted in some
frustration for those who are looking for greater isomorphism
between the model and a) neurophysiological features on the
one hand, and b) phenomenological experiences on the other.
But the model does not have to look right, it has to act
right. And it has to do that generatively, without adding
ad hoc connections just to make a specific behavior turn
out right.
It’s easy to forget that the model is supposed to be a
lean pared-down functional diagram of only those connections
that are absolutely necessary. As you say –
The idea is to introduce new properties only when you MUST.
Not just to be creative.
Thanks for your detailed response. I hope these remarks
are comparably helpful.
All the best,
Erling
NOTICE: This e-mail communication (including any attachments) is CONFIDENTIAL and the materials contained herein are PRIVILEGED and intended only for disclosure to or use by the person(s) listed above. If you are neither the intended recipient(s), nor a person responsible for the delivery of this communication to the intended recipient(s), you are hereby notified that any retention, dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify me immediately by using the “reply” feature or by calling me at the number listed above, and then immediately delete this message and all attachments from your computer. Thank you.
<<<>>>
