Understanding error signals and imagination

CSGnet Archive CSG1999
1

[Martin Taylor 991022 15:48]

[From Bob Christensen (991022.1020 PT)]

Bill Powers (991021.0710 MDT)

A perception is not necessarily an error signal just because it represents
something that we don't like.

How is it that we know a perception is one we don't like or want?

Followed by a discussion of why we should consider that error values
should ordinarily be propagated up the hierarchy as inputs to the
perceptual input functions of higher-level units. It's worth contemplating
this idea, quite apart from whether it violates the standard HPCT model.

There are two ways to consider the question (at least):
   (1) The inputs to higher-level systems may be derived from both the
perceptual signals and the error signals of the lower-level systems;
   (2) The inputs to higher-level systems may be derived ONLY from the
error signals of lower-level systems.

In analysing the implications of these two possibilities, we have to
think also of the low-level reference signal, because e = r-p. If we
allow any two of the three signals to form part of the input to the
higher levels, the third is implicit, since the higher-level perceptual
function might be constructed to form it from the two that are available.
Therefore proposal (1) is formally identical to allowing the higher-level
perceptual input function to see both the lower-level perceptual signal
and the lower-level reference signal. Proposal (2) is different.

We can probably dismiss proposal (2), because there is no value in allowing
only the error signal to propagate upward. Here's why. Any elementary
control unit that functions properly will act until its error signal is
zero. If there is another unit trying to control the error signal to
a different value, there is inherent conflict between the two (it's a
different kind of conflict than the one we normally consider, but it's
conflict nevertheless). So nothing at any higher level can do more than
act to assist its own input to become zero--in other words, the higher
level system can only duplicate the function of the lower, and the
higher-level systems CANNOT have varying reference values for their
inputs. Their reference values MUST be zero, or they induce conflict at
the lower levels.

We cannot dismiss proposal (1) so readily. Indeed, with the addition of
a switch or some other non-linear combination scheme, it is identical
to the hypothesized "imagination connection." In the imagination
connection, the reference value sent down to a lower-level control
unit is substituted for its perceptual signal, as if the unit had
successfully controlled its perception, but without requiring the
lower unit to act on its environment.

Since the reference value for the lower-level control unit can
be derived from the values of its perceptual signal and its error
signal, the imagination connection _could_ work by propagating the
error signal back up the hierarchy. But it would be awkward, since
using such an error-based imagination signal would require a
switch to prevent the lower-level system from acting in addition to
the switches already inherent in the mechanism. It's possible, but
less plausible than the "standard" connection.

How, then does error get perceived? I think Bob hints at that:

For instance one cannot immediately make a
dent in one's car go away, but one might start imagining the dent not being
there. This is imagining the reference, isn't it? Now I can toggle back

and forth between Perception (dent), and reference (no dent) leading to a

sequence of the two perceptions stored in memory. Now I can perceive the
sequence simultaineously like when perceiving both the A and the B at the
same time in AB. Now I can perceive that the two perceptions (regular
perception, and replayed reference) are different, just like I can tell

that A is different from B.

This sounds very like the way the imagination connection is assumed to
work, except that the imagination connection _also_ allows the higher
systems to determine the results of lower-level action if the lower-
level controls were to work properly. It could well be, as Bob says,
that a discrepancy here between current perception and the downgoing
reference signal that is returned as an imagined perception could
indicate "what we don't like."

The gist of what I'm trying to say is that the "standard" imagination
perception gives the same formal results as does upward propagation of
error signals, and is simpler in an engineering sense.

ยทยทยท

---------------------------

BUT...

As Mark Lazare (991020 1145 am) pointed out, there IS a body of research
in which the error signals themselves are the subject of study. Tom
Bourbon studied a kind of adaptation in which one control system used
the error signal of another as input to its Perceptual Input Function.
The output of this error-perceiving control unit was not, however, to
the reference input of the other control unit. It affected (if I remember
correcty) the gain of the other unit. It was a local tuning system that
attempted to optimize the performance of the other system. Mark called
it a reorganizing control, but I think that this is an improper name
for it.

I mention this mainly to show that it _is_ reasonable to consider the
place within the HPCT mechanism for units that sense the error signals
of other units. But it is less reasonable to place the error-sensing
units directly within the main perceptual control hierarchy. If Bourbon
is right, they tune the momentary operations of the main hierarchy. It
is tempting to equate them also with attention control. But I won't
pursue that here.

Martin

2

[From Bob Christensen (991022.1500 PT)]

[Martin Taylor 991022 15:48]

The gist of what I'm trying to say is that the "standard" imagination
perception gives the same formal results as does upward propagation of
error signals, and is simpler in an engineering sense.

I like it. As I think more about the imagination toggle being used to place
perception and reference in a sequence level perception, it occurs to me that
perceptions and references at the event level occur in sequence level
perceptions all the time because sequences typically start where we are
(perception) and end where we want to be (reference). I'm thinking also that
considering how to find a sequence to get from event A to event B typically
involves going up to programs.

Bob C.

3

[Bruce Gregory (991024.1525 EDT)]

Martin Taylor 991022 15:48

I mention this mainly to show that it _is_ reasonable to consider the
place within the HPCT mechanism for units that sense the error signals
of other units. But it is less reasonable to place the error-sensing
units directly within the main perceptual control hierarchy. If Bourbon
is right, they tune the momentary operations of the main hierarchy. It
is tempting to equate them also with attention control. But I won't
pursue that here.

It seems to me that it is also possible that error-signals serve as input to
emotional systems--fear for example. This could replace the S-R approach to
fear as a reaction to "negative stimuli" with fear as the way we experience
certain kinds of error.

Bruce Gregory

4

[From Bill Powers (991025.0655 MDT)]

Martin Taylor 991022 15:48]
YMartin:

Tom
Bourbon studied a kind of adaptation in which one control system used
the error signal of another as input to its Perceptual Input Function.
The output of this error-perceiving control unit was not, however, to
the reference input of the other control unit. It affected (if I remember
correcty) the gain of the other unit. It was a local tuning system that
attempted to optimize the performance of the other system. Mark called
it a reorganizing control, but I think that this is an improper name
for it.

It was actually a reorganizing process using the E. coli principle. It
measured total absolute error and if the error was increasing it shortened
the interval to the next reorganization. A reorganization consisted of
randomly changing the direction in which the gain was being altered, so the
gain was either slowly increasing or slowing decreasing. Thus the gain
could increase, reducing the average error signal, until it became so large
that oscillations began to show up; then it would hover right at the
threshold of oscillation, maintaining the best possible control without
instability. The gain, starting from zero, would automatically take on the
correct sign.

I mention this mainly to show that it _is_ reasonable to consider the
place within the HPCT mechanism for units that sense the error signals
of other units. But it is less reasonable to place the error-sensing
units directly within the main perceptual control hierarchy. If Bourbon
is right, they tune the momentary operations of the main hierarchy.

You're absolutely right; this is the HPCT answer to adaptive control. If
there are systems that monitor the properties of control itself, or any
other global criteria, they can certainly adjust the parameters of
hierarchical control systems to achieve the desired performance. The
reorganizing system is just a very simple and basic adaptive controller.
Others are quite possible, and perhaps even learnable. The Kalman Filter
model is just one of an infinite number of such models, and probably one of
the least plausible for a brain.

Best,

Bill P.

5

[From Bill Powers (991025.0811 MDT)]

Bruce Gregory (991024.1525 EDT)--

It seems to me that it is also possible that error-signals serve as input to
emotional systems--fear for example. This could replace the S-R approach to
fear as a reaction to "negative stimuli" with fear as the way we experience
certain kinds of error.

I'd like to see that idea more fully worked out.

My own idea is that emotions are not "systems" but simply perceptions (by
the basic postulate about perception, anything we experience must be a
perceptual signal). What we call an emotion, I proposed in LCSII, is a
combination of somatic sensations and cognitive-level perceptions.

In a little more detail, when there is an error at some level in the
hierarchy, the error signals alter reference signals in lower levels, level
by level, the changes inducing error signals in the lower systems until
action alters the environment to make perceptions catch up with the
newly-set reference levels. At about the thalamus, there is a bifurcation
in the hierarchy, with one branch going to the motor systems and the other
passing inward to the hypothalamus and from there, via the vagus nerve and
the neurohypophysis of the pituitary (and maybe other pathways -- I'm no
expert), to the organ control systems. The reference signals entering the
organ control systems and the control systems operated via the pituitary
alter the configuration of physiological and biochemical states, causing
many changes that give rise to perceptual signals. The basic intensity
signals from the body give rise to identifiable sensations, then to
configurations of sensations that we recognize as feeling states. Like any
other perceptions, these bodily perceptions go through all the levels of
representation: transitions, events, relationships, categories/sequences in
some order, and so on. At some point the experience is given the name of an
emotion or feeling state, and we speak and reason about emotions as if they
were real objects.

But what causes an emotion? Basically it is an error at some high level in
the hierarchy: a goal and a perception that fails to match the goal.
Normally, such an error would cascade down through the hierarchy to the
motor systems, which would act on the world to alter perceptions at all
levels back up to where the error exists, lessening the error. At the same
time, the state of the physiological and biochemical systems would be
altered to a new pattern which would provide the energy backup for the
action taking place, or shut down the energy systems if the motor systems
were called upon to cease action. The action and its perceptual effects
would propagate back up the hierarchy and soon correct the error, removing
the drive to the biochemical systems and returning them to their default
configuration. This normally happens without any sense of feeling a change
of emotional state, although one has occurred; the internal sensations are
mixed with the sensation of motor action and are simply considered part of
"doing" things.

We feel a definite lasting emotion in the case where a higher-order error
has called for action, and has altered the biochemical state as appropriate
to the action, but the action for some reason can't take place or can't
accomplish the results that are called for (such as escaping from danger).
Then the high-order error remains large, and the shift in somatic state
persists, and we are left with a continuing feeling-state and a continuing
goal-related error. This combination, the shifted feeling state and the
persistent higher-level (cognitive) error, is what we label an emotion. The
label we put on the emotion (anger, fear, excitement, horror, love, and so
on) depends on just what feeling pattern we sense, and what goal is not
being met. If the goal is to flee, but we are unable to do so, we will
probably call the experience by some fear-related name. If the goal is to
attack, but we are unable to attack, we will probably name the emotion in a
way related to "anger" terms. In both cases, as is well known, the same
general somatic pattern exists, the so-called "fight-or-flight" syndrome.
And so on through the emotions; we are prepared somatically to support some
course of action, but the action is not affecting the high-level error the
way it is supposed to.

When errors are quickly corrected, as is normal, the somatic feelings mix
with the action feelings and are experienced simply as the way it feel to
act -- we don't tend to use emotion words when action is successful at
correcting errors.
We speak of emotions when action doesn't succeed in removing the error.

So that's a summary of the theory of emotion I have proposed. It's
different from yours in that mine proposes that emotion is part of an
active control process. You're seems to make it more a reaction to stimuli.

Best,

Bill P.

6

[From Bruce Gregory (991025.1156 EDT)]

Bill Powers (991025.0811 MDT)

So that's a summary of the theory of emotion I have proposed. It's
different from yours in that mine proposes that emotion is part of an
active control process. Yours seems to make it more a
reaction to stimuli.

Let's imagine that you are going to teach me how to do hammerhead stalls
this afternoon. It's morning and I am dreading the up-coming lesson.
What is this dread associated with? It appears to be associated with my
imagining that I will not be able to maintain control of the
airplane--it is dread associated with an imagined future loss of
control. Presumably I can eliminate this dread by acting--by phoning you
and saying that I not want to go flying this afternoon. But when I
imagine this, I experience relief _and_ disappointment. I asked you to
show me how to do hammerhead stalls because I want to master this skill.
If I cancel our flying session I will not realize this intention.
Clearly I am in conflict.

It seems to me that the emotions (dread, relief, and disappointment) I
have described can be seen as linked to my imagined ability to exercise
control. When I imagine not successfully controlling, I experience
negative emotions. When I imagine successfully controlling, I experience
positive emotions. If I read you correctly, you are saying that the
emotions I feel are the result of actions that cannot reduce error, but
I am not sure I know what the actions are in this example.

Bruce Gregory

7

[From Bill Powers (991025.2053 MDT)]

Bruce Gregory (991025.1156 EDT)--

Let's imagine that you are going to teach me how to do hammerhead stalls
this afternoon. It's morning and I am dreading the up-coming lesson.
What is this dread associated with?

Better to ask first, "what is this dread?" If you know about it, it's a
perception or set of perceptions, isn't it? I claim it has a somatic and a
cognitive aspect. Could you tell me what they are, for you?

It appears to be associated with my
imagining that I will not be able to maintain control of the
airplane--it is dread associated with an imagined future loss of
control. Presumably I can eliminate this dread by acting--by phoning you
and saying that I not want to go flying this afternoon. But when I
imagine this, I experience relief _and_ disappointment. I asked you to
show me how to do hammerhead stalls because I want to master this skill.
If I cancel our flying session I will not realize this intention.
Clearly I am in conflict.

It seems to me that the emotions (dread, relief, and disappointment) I
have described can be seen as linked to my imagined ability to exercise
control. When I imagine not successfully controlling, I experience
negative emotions. When I imagine successfully controlling, I experience
positive emotions. If I read you correctly, you are saying that the
emotions I feel are the result of actions that cannot reduce error, but
I am not sure I know what the actions are in this example.

That is not how I said it, I think. The emotions are the result of the
error, not the actions; if you are unable to act, or if the actions you
take do not reduce the error (both resulting in non-reduction of the
error), you are likely to feel an emotion. Correcting the error will remove
the emotion.

Conflicts are particularly likely to produce emotions because there are
_two_ errors involved, one in each side of the conflict. You get ready,
biochemically, to act, but you do not act and use up the energy because of
the conflict. The name you give to the emotion will reflect both the
feeling state and the cognitive aspects of the situation. Dread, as I
understand it, arises when you feel compelled to do something and at the
same time expect that doing it will result in disaster. If you disregard
the particular goals and expectations involved, and concentrate only on the
sensations, the physical feelings will turn out to be mundane; breathing
funny, muscle tension, sensations of vasoconstriction, and so on. I don't
know exactly what they would be for a given individual, but they're just
ways of feeling that are common to many emotions. The main difference
between emotions is in the details of the goals involved.

You may or may not feel an emotion when you imagine succeeding or failing.
You can, of course, imagine a remembered emotion, and if you imagine danger
vividly enough you can succeed in scaring yourself, or I suppose
exhilarating yourself. But such emotions are still the result of errors;
when you're imagining, you don't actually act, so the biochemical
preparedness is not dissipated by real action.

By the way, I wouldn't teach you how to do a hammerhead stall; it's too
hard on the airframe and engine mounts and we could both be killed. But I
can say that without any emotion; it's just a fact. It wouldn't start to
involve emotions until I started to suspect that I was going to _have_ to
teach you in spite of my determination not to.

Best,

Bill P.

8

[From Bruce Gregory (991027.1242 EDT)]

Bill Powers (991025.2053 MDT)

That is not how I said it, I think. The emotions are the result of the
error, not the actions; if you are unable to act, or if the
actions you
take do not reduce the error (both resulting in non-reduction of the
error), you are likely to feel an emotion. Correcting the
error will remove
the emotion.

This seems to be the case with negative emotions, I agree. It doesn't
seem to apply to positive emotions, does it?

Bruce Gregory

9

[From Bill Powers (991027.1633 MDT)]

Bruce Gregory (991027.1242 EDT)--

This seems to be the case with negative emotions, I agree. It doesn't
seem to apply to positive emotions, does it?

I don't know. It might. I know it would seem odd to associate positive
emotions with error signals, but if we can get over that proposition, maybe
it would make sense. It has been suggested that at least some positive
emotions go with a negative first derivative of a "bad" perception -- that
is, the bad perception is going away. Other positive emotions, like love,
may well involve error signals. "Love" is somewhat of a euphemism.

The term "emotion" is often used when we really aren't feeling much. Is
appreciation an emotion? Is adnmiration? Lots of other words indicate
states of mind and attitudes more than feelings, yet they're often spoken
of as emotions.

"Good emotions" are not necessarily good for us. The best state of mind, in
my experience, is not exhiliration or jubilation or even what people refer
to vaguely as happiness: rather, I think it's a state of tranquility, of
alert contact with the world, in which the body is in a rather neutral state.

That's obviously just my opinion.

Best,

Bill P.