Mastery is its own reward

[Martin Taylor 970508 11:45]

Bruce Gregory (970508.0950 EDT) to Bill Powers (970507.1536 MST)

> But as you surmise, mastery may be its own reward (isn't it amazing how many
> folk sayings translate directly into PCT -- and how few scientific
> descriptions do?).

Funny, I am continually amazed by this as well.

It hasn't been made explicit how "mastery is its own reward" translates into
PCT directly, and for myself I'm not clear how it does. I have a hypothesis,
though. Perhaps Bill P. will expand on his cryptic statement and suggest
where I'm going off the rails (though I probably won't see a response, since
I'm leaving tomorrow evening for a month in UK and Ireland, away from e-mail).

"Mastery" is no more obvious a perceived variable than is "uncertainty."
But it could be perceived, and it is easily translated into something that
is often cited as a likely intrinsic variable for the reorganizing system:
some function of absolute error magnitude and its derivative. High mastery
translates into low and decreasing error.

But it is clear that if there is a control for level of mastery, the
reference level cannot be perfection (as one assumes it is for the reorganizing
system). There is little enjoyment in succeeding in doing something that is
trivially easy. Nor is there much enjoyment in trying to control some
perception that you simply can't influence (yet). As Chris Cherpas said
(970507.1444):

+And, as I've noted before: what we've found empirically
+at CCC is that students who have an average error rate
+(e.g., errors in the conventional sense of "not the right answer")
+of about .3 enjoy a steeper trajectory in mastering cumulative
+sets of exercises than those with either > or < than .3.
+This is a kind of measure of that balance between the
+experience of mastery and challenge.

What is enjoyable (I guess that's what "its own reward" means) is therefore
not "mastery" but "the attainment of mastery". The acquisition of an ability
to control something well that could be controlled only poorly beforehand.
Once the ability to control well has been achieved sufficiently that
reorganization has more or less ceased in that area, the enjoyment has
gone out of the pursuit of mastery.

That is, the progress of reorganization is what is celebrated, not the
success of control.

Does this seem reasonable? Does it lead anywhere in respect of education?

See you in a month.

Martin

[From Bill Powers (970508.1121 MST)]

Martin Taylor 970508 11:45--

It hasn't been made explicit how "mastery is its own reward" translates
into PCT directly, and for myself I'm not clear how it does.

Nothing complicated: mastery can become a reference level for the perceived
degree of mastery. Complete mastery of tennis means winning every game
40-love. Complete mastery of bowling means bowling nothing but 300 games.
Conplete mastery of chess means winning every game. However, most people set
their reference levels for mastery somewhere below the maximum possible
amount -- that's what "level of aspiration" translates into.

Remember that "reward," in PCT, is simply whatever reference level you have
set for a perception. Things or events don't actually have any rewarding
power in themselves -- that's a different model. So mastery being its own
reward can translate only into perceived degree of mastery being a
controlled variable, with a specific degree of mastery as the reference
condition.

Indirectly responding to Bruce G: there is probably an application here for
the "universal error curve" we're talked about a few times on the net. For
small errors, output is proportional to error. As the error grows, the
output falls below a proportional relationship, and above some value of
error, the output begins to fall again. If the slope of the falloff is great
enough, a positive feedback effect ensues, with output dropping rapidly and
error increasing without limit. In other words, the control system appears
to "give up." When Wolfgang Zocher finishes his new version of Simcon, I
will try to show a simulation of that effect. Remind me.

Best,

Bill P.

[From Bruce Gregory (970508.1335 EDT)]

Martin Taylor 970508 11:45

I'm leaving tomorrow evening for a month in UK and Ireland, away from e-mail).

Enjoy your trip!

What is enjoyable (I guess that's what "its own reward" means) is therefore
not "mastery" but "the attainment of mastery". The acquisition of an ability
to control something well that could be controlled only poorly beforehand.
Once the ability to control well has been achieved sufficiently that
reorganization has more or less ceased in that area, the enjoyment has
gone out of the pursuit of mastery.

That is, the progress of reorganization is what is celebrated, not the
success of control.

It depends, I think, on the perceived difficulty of the control
being exercised. If it has taken a lot of work to achieve a
level of mastery, then exercising it seems to be cause for
celebration. At a certain level of mastery, one looks for
challenges in the form of other masters to contest. Sports is an
obvious example, but I think of the Zen Masters who seemed to
enjoy testing their wits against each other. Personally, I'm
thinking of challenging people to compensatory tracking
tasks...:wink:

Bruce

[Martin Taylor 970612 12:15]

Bill Powers (970508.1121 MST)]

Old stuff, being caught up.

Martin Taylor 970508 11:45--

>It hasn't been made explicit how "mastery is its own reward" translates
>into PCT directly, and for myself I'm not clear how it does.

Nothing complicated: mastery can become a reference level for the perceived
degree of mastery.
....
Remember that "reward," in PCT, is simply whatever reference level you have
set for a perception. Things or events don't actually have any rewarding
power in themselves -- that's a different model. So mastery being its own
reward can translate only into perceived degree of mastery being a
controlled variable, with a specific degree of mastery as the reference
condition.

"Mastery" presumably translates into accuracy of control--the degree to which
a particular perception tends to match its reference over some extended
period.

So what this statement seems to say is that the perceptual input
of one Elementary Control Unit (ECU) is derived in some way from either
(1) the error signal, or (2) both the perceptual and reference signals
of another ECU. Either way, this doesn't seem to be derivable from the
"standard" HPCT hierarchy, since the error signals are supposed to be
available to the reorganization system, but not to the perceptual hierarchy.
The perceptual hierarchy signals are supposed to be derived from the sensors,
or from the imagination-loop return signals (or from memory), at least as
I understand the "standard" hierarchy.

However, it does seem consistent with the view I have sometimes expressed,
and with which you have sometimes agreed, that perceptual signals can be
derived from any source within the hierarchy.

As for "reward" _being_ a reference level--that doesn't ring true at all.
The attainment of a reference level is (presumably) rewarding. But that
doesn't make the desire to score a perfect bowling game be by itself a
reward. I could reward myself tremendously by setting reference levels
for getting a Nobel Prize, winning a big lottery, and having everyone
around me become happy. But I suspect that if I did so, my failure to bring
my perceptions in line with those reference levels would be distinctly
_un_rewarding. Surely what is rewarding is the bringing of some perception
to its reference level _particularly when it has been difficult to do so_.

... most people set
their reference levels for mastery somewhere below the maximum possible
amount -- that's what "level of aspiration" translates into.

So, somewhere in the system there is the ability to perceive how well other
control systems are functioning, to set a reference level for how well they
are functioning, and to affect how well they are functioning.

That sounds a lot like some kind of specialization of the reorganizing
system: the ability to target reorganization toward a specific area of
control--and to perceive that this targeting is being done.

I still have a problem seeing how "mastery is its own reward" translates
into _standard_ PCT.

Martin

[From Bruce Gregory (970612.1415 EDT)]

Martin Taylor 970612 12:15

> Bill Powers (970508.1121 MST)

I share you puzzlement.

As for "reward" _being_ a reference level--that doesn't ring true at all.
The attainment of a reference level is (presumably) rewarding. But that
doesn't make the desire to score a perfect bowling game be by itself a
reward. I could reward myself tremendously by setting reference levels
for getting a Nobel Prize, winning a big lottery, and having everyone
around me become happy. But I suspect that if I did so, my failure to bring
my perceptions in line with those reference levels would be distinctly
_un_rewarding. Surely what is rewarding is the bringing of some perception
to its reference level _particularly when it has been difficult to do so_.

This would be my interpretation as well. This also make sense in
terms of how _I_ would design a nervous system :wink:

>... most people set
> their reference levels for mastery somewhere below the maximum possible
> amount -- that's what "level of aspiration" translates into.

This isn't clear to me. I doubt that people have
reference levels for mastery. I think we settle for less than
perfect mastery in many cases because we find that we cannot
achieve perfect mastery.

So, somewhere in the system there is the ability to perceive how well other
control systems are functioning, to set a reference level for how well they
are functioning, and to affect how well they are functioning.

That sounds a lot like some kind of specialization of the reorganizing
system: the ability to target reorganization toward a specific area of
control--and to perceive that this targeting is being done.

I agree. This does not seem to make sense.

I still have a problem seeing how "mastery is its own reward" translates
into _standard_ PCT.

My interpretation is that this means that we need no additional
reward beyond our ability to bring our perceptions to the
appropriate reference level to motivate us to exercise control.
Bill may have something completely different in mind.

Bruce

···

Martin

[From Bill Powers (970612.1341 MDT)]

Martin Taylor 970612 12:15--

So what this statement seems to say is that the perceptual input
of one Elementary Control Unit (ECU) is derived in some way from >either

(1) the error signal, or (2) both the perceptual and >reference signals of
another ECU. Either way, this doesn't seem to >be derivable from the
"standard" HPCT hierarchy, since the error >signals are supposed to be
available to the reorganization system, >but not to the perceptual hierarchy.

You're going about this the wrong way. Try to figure out how it works
_without_ going outside the normal HPCT hierarchy. The perceptual signal in
an ECU is _never_ (by hypothesis) derived from the error signal of any
other system, or its own error signal, or anything but a lower-level set of
perceptual signals. Both actions and their consequences are known _only_
via first-level perceptions and others built on them, or imagination. If
you find it impossible to represent the situation you're thinking of under
these limitations, then the model has to be changed. But first you have to
give it a good try!

However, it does seem consistent with the view I have sometimes
expressed, and with which you have sometimes agreed, that perceptual
signals can be derived from any source within the hierarchy.

I don't recall ever having agree to that. Maybe I agreed to something you
took to _mean_ that, but if so I meant something different.

As for "reward" _being_ a reference level--that doesn't ring true at >all.

The attainment of a reference level is (presumably) rewarding. >But that
doesn't make the desire to score a perfect bowling game be >by itself a
reward.

You're assuming something from a different theory. NOTHING is "rewarding"
in HPCT -- that is, there is NOTHING that acts on your nervous system to
make you want to repeat an action that has had a favorable result. If you
have set a reference signal to a high level, and manage to bring your
perception into a match with it, you may feel good as the error decreases,
but the feeling good is not what made you control the perception. It's a
consequence of controlling a perception you presumably couldn't control
before. Perhaps whatever you did to achieve control resolved a conflict,
and the good feeling is a side-effect of the conflict going away.

I could reward myself tremendously by setting reference levels
for getting a Nobel Prize, winning a big lottery, and having >everyone

around me become happy.

No, you couldn't, not under PCT. There is no reward in PCT. Actually, I
suspect that setting a reference level for getting a Nobel Prize (actually
doing so, not just contemplating it) would make you feel terrible until you
got the Prize. Big protracted errors are pretty unpleasant. But if you
actually knew what to do to get the Prize, and did it, and it worked, you
would probably feel pretty good -- "rewarded" -- although it would have
been the reference level, not the "reward", that led you to get it. The
nearest I can come to defining "reward" in PCT is "the experience that
follows from an error being corrected."

A reference signal is "its own reward" if it results in the corresponding
perception being brought into a match with it. That is, setting the
reference level creates an error, which is felt as "unrewarding", and
successful action then corrects the error, which is felt as "rewarding."
Except that there is no rewarding effect on the system, in PCT -- it's just
an experience with no special effect on behavior.

The PCT theory of emotion goes something like this: when you experience
error, your physiological systems are given new reference conditions to
back up the actions that will be used to correct the error. You sense those
changes in physiological state as a feeling, which together with the other
perceptions involved constitute an emotion. Emotions are most strongly felt
when the physiological preparations take place, but for some reason
(perhaps conflict) the action does not occur.

I still have a problem seeing how "mastery is its own reward" >translates

into _standard_ PCT.

Maybe I've now dropped a few hints about how to do it.

Best,

Bill P.

[From Bill Powers (970612.1420 MDT)]

Bruce Gregory (970612.1415 EDT)

Martin said:

Surely what is rewarding is the bringing of some perception
to its reference level _particularly when it has been difficult to >> do

so_.

and you said:

This would be my interpretation as well. This also make sense in
terms of how _I_ would design a nervous system :wink:

Well, I wouldn't design it that way. I would design it so that intrinsic
error caused reorganization to start, and when the intrinsic error was
corrected, the organization of the system would then persist in the form it
had after the last reorganization. Skinner, on the other hand, might have
designed it so that after a "rewarding" event, the probability of the
behavior that was occuring just when the reward happened would increase. So
if you went hopefully up to the ticker-seller at a popular rock concert,
and found that there was a ticket available, you would be tremendously
rewarded and would immediately get back in line to buy another ticket.

PCT gets along entirely without the concept of reward.

···

---------------

This isn't clear to me. I doubt that people have
reference levels for mastery. I think we settle for less than
perfect mastery in many cases because we find that we cannot
achieve perfect mastery.

If you set a reference level for perfect mastery (that is, if mastery means
to you the achievement of perfection), then the chances are that you will
never master anything to your satisfaction. I think the normal case is to
set a reference level for enough mastery of a skill that you can then use
it for whatever purpose you had in mind. I have mastered typing to a level
sufficient for my needs, although if I had some reason to want to type more
accurately and faster, I could raise my sights, practice more at the proper
finger positions, and get better at it. But I'd rather spend the same time
typing _about_ something.

Best,

Bill P.

[From Bruce Gregory (970612.1705 EDT)]

Bill Powers (970612.1420 MDT)

Bruce Gregory (970612.1415 EDT)

Martin said:
>> Surely what is rewarding is the bringing of some perception
>> to its reference level _particularly when it has been difficult to >> do
so_.

and you said:
>This would be my interpretation as well. This also make sense in
>terms of how _I_ would design a nervous system :wink:

Well, I wouldn't design it that way. I would design it so that intrinsic
error caused reorganization to start, and when the intrinsic error was
corrected, the organization of the system would then persist in the form it
had after the last reorganization. Skinner, on the other hand, might have
designed it so that after a "rewarding" event, the probability of the
behavior that was occuring just when the reward happened would increase. So
if you went hopefully up to the ticker-seller at a popular rock concert,
and found that there was a ticket available, you would be tremendously
rewarded and would immediately get back in line to buy another ticket.

Some misunderstanding here, because I agree with you! My point
was that we enjoy exercising control in the face of a challenge.
Which I think is the point Martin is making. (I realize
"rewarding" is a no, no.)

PCT gets along entirely without the concept of reward.

Yes, but not without the concept of pleasure!

If you set a reference level for perfect mastery (that is, if mastery means
to you the achievement of perfection), then the chances are that you will
never master anything to your satisfaction. I think the normal case is to
set a reference level for enough mastery of a skill that you can then use
it for whatever purpose you had in mind.

Yes, I agree.

I have mastered typing to a level
sufficient for my needs, although if I had some reason to want to type more
accurately and faster, I could raise my sights, practice more at the proper
finger positions, and get better at it. But I'd rather spend the same time
typing _about_ something.

You never _intend_ (at least I suspect not) to make some
percentage of typos. You'd be quite happy if you always hit the
key you intended to. Since that hasn't happened, you have
learned to live with some incidence of errors. The gain on the
"hit the right key" loop is not terribly tight. In this
sense, you have achieved a satisfactory level of mastery. No?

Bruce

[From Bruce Gregory (970612.1720 EDT)]

Bill Powers (970612.1341 MDT)

Emotions are most strongly felt
when the physiological preparations take place, but for some reason
(perhaps conflict) the action does not occur.

Then I hold a heterodox view. In many cases strong emotions are
associated with a loss of control (or intimations that
loss of control is imminent). You _are_ acting, but the error
is growing larger, not smaller. Pleasure is often associated
with exercising control in the face of a challenge. (I'm not
sure what this says about sex :wink:

Bruce

[Martin Taylor 970612 17:50]

Bill Powers (970612.1341 MDT)]

Martin Taylor 970612 12:15--

I still have a problem seeing how "mastery is its own reward"
translates into _standard_ PCT.

Maybe I've now dropped a few hints about how to do it.

No you haven't. You've made some comments about PCT not using the concept
of reward in the sense of Skinner, comments which change my understanding
not one iota. As in...

You're assuming something from a different theory. NOTHING is "rewarding"
in HPCT -- that is, there is NOTHING that acts on your nervous system to
make you want to repeat an action that has had a favorable result.

And you discuss something else on which there is no issue, as far as
I am concerned:

If you
have set a reference signal to a high level, and manage to bring your
perception into a match with it, you may feel good as the error decreases,
but the feeling good is not what made you control the perception. It's a
consequence of controlling a perception you presumably couldn't control
before. Perhaps whatever you did to achieve control resolved a conflict,
and the good feeling is a side-effect of the conflict going away.

There is one comment you make that surprises me:

However, it does seem consistent with the view I have sometimes
expressed, and with which you have sometimes agreed, that perceptual
signals can be derived from any source within the hierarchy.

I don't recall ever having agree to that. Maybe I agreed to something you
took to _mean_ that, but if so I meant something different.

If you _do_ repudiate that, then your prior comments about the _control_
of the level of mastery become even harder to understand. Let me repeat
the start of my earlier message, part of which you cited:

"Mastery" presumably translates into accuracy of control--the degree to which
a particular perception tends to match its reference over some extended
period.

So what this statement seems to say is that the perceptual input
of one Elementary Control Unit (ECU) is derived in some way from either
(1) the error signal, or (2) both the perceptual and reference signals
of another ECU. Either way, this doesn't seem to be derivable from the
"standard" HPCT hierarchy, since the error signals are supposed to be
available to the reorganization system, but not to the perceptual hierarchy.
The perceptual hierarchy signals are supposed to be derived from the sensors,
or from the imagination-loop return signals (or from memory), at least as
I understand the "standard" hierarchy.

Now I ask the question: If neither the error signal nor the reference
signal of one ECU are available as inputs to the perceptual function of
_any_ other ECU, then what is the perception of mastery, and what is the
output of the control unit that affects that perception?

You're going about this the wrong way. Try to figure out how it works
_without_ going outside the normal HPCT hierarchy. The perceptual signal in
an ECU is _never_ (by hypothesis) derived from the error signal of any
other system, or its own error signal, or anything but a lower-level set of
perceptual signals. Both actions and their consequences are known _only_
via first-level perceptions and others built on them, or imagination. If
you find it impossible to represent the situation you're thinking of under
these limitations, then the model has to be changed. But first you have to
give it a good try!

I thought I had. But, having utterly failed, both at the beginning of
May and now, I defer to the Master.

I could reward myself tremendously by setting reference levels
for getting a Nobel Prize, winning a big lottery, and having >everyone

around me become happy.

No, you couldn't, not under PCT. There is no reward in PCT. Actually, I
suspect that setting a reference level for getting a Nobel Prize (actually
doing so, not just contemplating it) would make you feel terrible until you
got the Prize. Big protracted errors are pretty unpleasant. But if you
actually knew what to do to get the Prize, and did it, and it worked, you
would probably feel pretty good -- "rewarded" -- although it would have
been the reference level, not the "reward", that led you to get it. The
nearest I can come to defining "reward" in PCT is "the experience that
follows from an error being corrected."

I agree with all this, of course, but it's irrelevant to the question
posed.

Martin

[From Bill Powers (970612.2044 MDT)]

Martin Taylor 970612 17:50--

There is one comment you make that surprises me:

However, it does seem consistent with the view I have sometimes
expressed, and with which you have sometimes agreed, that >>>perceptual

signals can be derived from any source within the >>>hierarchy.

I don't recall ever having agree to that. Maybe I agreed to >>something

you took to _mean_ that, but if so I meant something >>different.

If you _do_ repudiate that, then your prior comments about the >_control_

of the level of mastery become even harder to understand. >Let me repeat
the start of my earlier message, part of which you >cited:

"Mastery" presumably translates into accuracy of control--the >>>degree

to which a particular perception tends to match its >>>reference over some
extended period.

So what this statement seems to say is that the perceptual input
of one Elementary Control Unit (ECU) is derived in some way from
either (1) the error signal, or (2) both the perceptual and >>>reference

signals of another ECU. Either way, this doesn't seem to >>>be derivable
from the "standard" HPCT hierarchy, since the error >>>signals are supposed
to be available to the reorganization system, >>>but not to the perceptual
hierarchy.

All right, let's assume that: error signals are not available to the
perceptual hierarchy. What _is_ available to a higher-level system? A copy
of the reference signal (via the imagination connection) that would be sent
to the lower system if action were to take place. The perceptual signal
from the lower system that would be controlled if action were to take
place. In other words, your (2). If the perceptual signal were not equal to
the reference signal, as viewed by the higher system (a relationship
between two signals), control would be bad. You imagine achieving a
perception, and then you see if you actually achieved it. One imagined
perception, one real perception: i.e., two perceptions, being judged as to
their equality, the equality being a higher-level perception. NOT AN ERROR
SIGNAL.

Before you object that this perception is the equivalent of am error
signal, remember that there will be a reference signal for this perception,
and it need not be zero. That is, you could have the goal of perceiving the
real perception as being greater than, equal to, or less than the imagined
one: overachieving, achieving exactly the desired state, or underachieving.
With true error signals you do not have these options: the only correct
state for an error signal is zero.

Now I ask the question: If neither the error signal nor the >reference

signal of one ECU are available as inputs to the >perceptual function of
_any_ other ECU, then what is the perception >of mastery, and what is the
output of the control unit that affects >that perception?

The perception of mastery is just that, a perception. It is derived from
imagination (the desired state of a perception) and real-time perception
(what is actually achieved). So all the necessary information is available
from the channels that are already part of the HPCT model.

I did not say that perceptual signals can be derived from "any source in
the hierarchy." They cannot be derived from error signals. They can be
derived from the output of _the same system within which the perception
exists_ -- that is the imagination connection. They can be derived from
lower-level perceptual signals. They cannot be derived from any signals in
any higher system. That's the structure of the HPCT model.

I have worked out, more or less, how hierarchical control can work given
these connections and only these. If you start running other connections
through the system, I no longer know how the whole arrangement would work.
What happens if a higher input function receives not only the perceptual
signal from a lower system, but the error signal too, or instead of the
perceptual signal? What does the higher perception then mean? The higher
reference signal? The error signal in the higher system? What kind of
reference signal would be returned to the lower system, and what would it
accomplish? I can imagine how to draw these connections in a diagram, but I
can't imagine what the model would actually do with those connections in
place. Can you?

I have tried several times to see how error information might be used by a
higher-level system. All my ideas seemed plausible until I tried to get a
model to work, and couldn't. I still think about it from time to time, but
so far, no result. It's possible that some such model might work, might do
something recognizeable, but until that model is discovered, and
demonstrated to work, I will stick with the basic HPCT model because it's
the only one we have been able to get to operate in simulation.

You're going about this the wrong way. Try to figure out how it >>works

_without_ going outside the normal HPCT hierarchy. The >>perceptual signal
in an ECU is _never_ (by hypothesis) derived from >>the error signal of any
other system, or its own error signal, or >>anything but a lower-level set
of perceptual signals. Both actions >>and their consequences are known
_only_ via first-level perceptions >>and others built on them, or
imagination. If you find it impossible >>to represent the situation you're
thinking of under these >>limitations, then the model has to be changed.
But first you have >>to give it a good try!

I thought I had. But, having utterly failed, both at the beginning of May

and now, I defer to the Master.

Master-y is just a matter of looking at the available components and rules
and seeing how to use them to achieve a given result. It's hard to find any
realistic situation that the present HPCT model can't handle, without going
outside the definitions and structure. Of course that doesn't mean diddly
if you can think of a simpler structure that will allow achieving the same
result in a simpler way. But you know what I will say about any alternative
arrangement: show me that it actually does what you say it will do.

Best,

Bill P.

[Martin Taylor 970613 0935]

Bill Powers (970612.2044 MDT)

Now we may be getting somewhere!

All right, let's assume that: error signals are not available to the
perceptual hierarchy. What _is_ available to a higher-level system? A copy
of the reference signal (via the imagination connection) that would be sent
to the lower system if action were to take place.

That copy of the reference signal is what is needed. But to clarify this,
it is necessary that this copy be available _independently_ of the perceptual
signal it is imagining. Obviously it is, since we can consciously perceive
what we imagine. The very word itself says so. But this raises the same
kind of problem unsuccessfully addressed many times in different guises
on CSGnet. Just how are the imagined and real perceptions integrated within
the operation of the control loop? The question comes up in B:CP, with an
answer based on switches. It comes up in MCT in the building of the Kalman
filter. It comes up in your recent question "How is the output of an
Elementary Control Unit directed appropriately to the many possible
mechanisms available at lower control levels?" (in other words, how
do we choose to open a swinging door with a hand or a foot when carrying
a load?) It comes up in many guises.

The perceptual signal
from the lower system that would be controlled if action were to take
place. In other words, your (2). If the perceptual signal were not equal to
the reference signal, as viewed by the higher system (a relationship
between two signals), control would be bad. You imagine achieving a
perception, and then you see if you actually achieved it. One imagined
perception, one real perception: i.e., two perceptions, being judged as to
their equality, the equality being a higher-level perception. NOT AN ERROR
SIGNAL.

No, the difference is indeed a higher-level perception, but the value of
that difference is equal to the value of the error signal of the lower
level control unit.

Before you object that this perception is the equivalent of am error
signal, remember that there will be a reference signal for this perception,
and it need not be zero.

Too late:-) If the perception is _of_ an error signal, there can indeed be
a reference level for that perception, and an error signal based on the
difference between the reference [for lower-level error] and the perception
[of lower-level error].

That is, you could have the goal of perceiving the
real perception as being greater than, equal to, or less than the imagined
one: overachieving, achieving exactly the desired state, or underachieving.
With true error signals you do not have these options: the only correct
state for an error signal is zero.

Yes, but that's the business of the Elementary Control Unit being monitored,
not of the ECU doing the monitoring.

The perception of mastery is just that, a perception. It is derived from
imagination (the desired state of a perception) and real-time perception
(what is actually achieved). So all the necessary information is available
from the channels that are already part of the HPCT model.

Good. Now the next question is: if "mastery" of something is a controllable
perception, what is the mechanism whereby the action output of the mastery
control unit affects the mastery perception?

In PCT (or HPCT) learning is the business of a separate structure known
as the reorganizing system. This system is presumed to work by affecting
the parameters and interconnections of the perceptual hierarchy.

Presumably "mastery" control is an aspect of the reorganizing system,
targeted toward the reorganization of a sub-part of the perceptual hierarchy
that could be involved with the perception whose control is to be "mastered."
(But if the degree of "mastery" is perceptible, then at least that aspect
of the reorganization system is part of the _perceptual_ control hierarchy,
isn't it?)

···

---------------------

I did not say that perceptual signals can be derived from "any source in
the hierarchy." They cannot be derived from error signals. They can be
derived from the output of _the same system within which the perception
exists_ -- that is the imagination connection. They can be derived from
lower-level perceptual signals. They cannot be derived from any signals in
any higher system. That's the structure of the HPCT model.

Fair enough. I included the acknowledgment that you had pointed out the
possiblity that perceptual signals could be derived from any source because
I thought I got the idea from you, and if I failed to acknowledge that
fact you might think I was claiming unwarranted credit for a new idea.

But if you didn't say it, you didn't, and it's my memory that's at fault.

I have worked out, more or less, how hierarchical control can work given
these connections and only these. If you start running other connections
through the system, I no longer know how the whole arrangement would work.
What happens if a higher input function receives not only the perceptual
signal from a lower system, but the error signal too, or instead of the
perceptual signal? What does the higher perception then mean?

Its absolute value, low-pass filtered, means the level of mastery achieved
by the lower-level system.

You are in a cyclic argument here. First you say that the error
signal can't be used as an input to a higher control unit because
you haven't been able to see what the higher perception might mean;
and since for that reason the error signal can't be used as an input to
another perceptual function, it isn't available to be perceived as
"mastery," maning that you still have nothing that the error signal
could mean as a higher perception.

The higher
reference signal? The error signal in the higher system? What kind of
reference signal would be returned to the lower system, and what would it
accomplish? I can imagine how to draw these connections in a diagram, but I
can't imagine what the model would actually do with those connections in
place. Can you?

No. But then, I have trouble imagining in detail just what happens even in
the "classical" hierarchy. In principle, and in gross, I have no problem.
But I do have problems when trying to consider phase-shift effects and
nonlinear effects in a hierarchy with several units per level all connected
with all the units at the levels above and below.

But as you may have noticed, most of my contributions to CSGnet over the
years have been concerned with questions of what happens within a network
_as a whole_, rather than within one control loop. That's because the
behaviour of a single control loop is easy to understand, at least if
there's not too much nonlinearity. The behaviour, and more particularly
the adaptive (reorganization) behaviour, of the interacting control
systems in the hierarchy, or of interactions among control systems in
different hierarchies, is much harder to understand.

So I accept your desire to keep within the framework of simply structured
connections, even if from time to time other connections (such as gain
control signals) seem to be introduced in plausible and even desirable
ways.

I have tried several times to see how error information might be used by a
higher-level system. ...

Other than the reorganizing system, I presume you mean. You use it there.

...All my ideas seemed plausible until I tried to get a
model to work, and couldn't. I still think about it from time to time, but
so far, no result. It's possible that some such model might work, might do
something recognizeable, but until that model is discovered, and
demonstrated to work, I will stick with the basic HPCT model because it's
the only one we have been able to get to operate in simulation.

... It's hard to find any
realistic situation that the present HPCT model can't handle, without going
outside the definitions and structure. Of course that doesn't mean diddly
if you can think of a simpler structure that will allow achieving the same
result in a simpler way. But you know what I will say about any alternative
arrangement: show me that it actually does what you say it will do.

Fair enough. But the same could be said about much of the "standard"
arrangement. The Arm shows that a hierarchy works very well, and you have
done some experiments with particular forms of reorganization in a one-
level structure (or are there more studies I don't know about?). We
assert and assume that similar techiques would work in multi-level
hierarchies. We have good reason to believe that reorganization tends
toward both layering and orthogonalization of at least the output
part of the hierarchy, but I know of no studies (other than our aborted
"Little Baby" project) that attempted to show this happening in practice.

It's great to have an idealized position as a reference, but mastery of
HPCT, like mastery of typing, is something for which we all (including,
I suspect, you) have to accept some degree of errer:-)

Martin

[From Bill Powers (970613.1101 MDT)]

Martin Taylor 970613 0935--

That copy of the reference signal is what is needed. But to clarify >this,

it is necessary that this copy be available _independently_ of >the
perceptual signal it is imagining. Obviously it is, since we can

consciously perceive what we imagine. The very word itself says so. >But

this raises the same kind of problem unsuccessfully addressed >many times
in different guises on CSGnet. Just how are the imagined >and real
perceptions integrated within the operation of the control >loop?

Yes, a good question. My observation is that the imagined state of a
perception is never perceived at the same time as the present-time state --
those switches in B:CP reflect that observation. Try imagining your hand
turned palm up while you're seeing it turned palm down. I, anyway, can't do
it. This implies that when you consciously compare actual with imagined
performance, you alternate between the two. Imagined score: 100. Now look
at the paper: actual score, 72. Ouch.

A further implication is that judgments of this sort -- mastery -- are
pretty high level perceptions, involving categories, sequence, and logic.
This is NOT the reorganizing system we're talking about, but the operation
of the learned hierarchy.

In fact, I don't believe there is very much the learned -- conscious --
systems can do that will affect mastery. What can you do to improve your
golf swing? At the nuts and bolts level, not much. The actions that are
available have nothing to do with tweaking loop gain or adjusting dynamic
response: the most we can do is "pay attention" to such things as hand and
arm configurations. What we're supposed to be doing while we pay attention
isn't made clear in instruction manuals. We can think, "left arm straight,"
at the same time we're swinging with the left arm bent. The main thing we
can do consciously is to put ourselves in the right position and swing
again and again, and then again and again, while the swing gradually
approaches some asymptote. WE are not making it approach that asymptote.
We're just establishing the conditions in which reorganization or
adaptation can take place.

Good. Now the next question is: if "mastery" of something is a
controllable perception, what is the mechanism whereby the action >output

of the mastery control unit affects the mastery perception?

I think I've answered that: the "mastery control system" can act mainly by
establishing the conditions under which we get better at controlling,
rather than directly manipulating the parameters of control. The parameter
adjustments are made mostly through reorganization or adaptation. From our
seats in the learned hierarchy, about all we can do is keep asking for the
same perception, and hoping that something will change to make it more like
the perception we want. The way to Carnegie Hall is through practice.

Presumably "mastery" control is an aspect of the reorganizing >system,

targeted toward the reorganization of a sub-part of the >perceptual
hierarchy that could be involved with the perception >whose control is to
be "mastered."

I guess what I'm trying to say is that I don't think the actual achievement
of mastery is under the control of learned systems, which is where
consciousness resides. The reorganizing system is not part of them; it
works automatically, at all levels, in the same way every time. It's an
_unlearned_ control system, working outside the learned hierarchy and
outside consciousness.

(But if the degree of "mastery" is perceptible, then at least that >aspect

of the reorganization system is part of the _perceptual_ >control
hierarchy, isn't it?)

No, that doesn't follow. Mastery is a learned perceptual judgment. Not
everything we can perceive can be controlled. By setting reference levels
for our own control performance (and thus defining mastery), we create
error signals that, to the best of their current ability, the lower systems
correct. If control is not good, about all we can do is keep setting the
same reference signals. Usually what happens is that control improves. But
it doesn't improve through the actions of a "mastery control system." It
improves because there are built-in mechanisms for altering organization
until errors are as small as possible. We might appreciate experiencing an
improvement in control, but we can't take credit for the processes that
improved it.
About the most we can do is adopt principles of perseverance, patience, and
repetition.

Best,

Bill P.