reorganization

[Martin Taylor 930129 19:00]

Forgive me for this.

Here's a quote lots of people around here used to keep posted on their
boards. I just saw it used as a .sig in a posting, and it seems to me
to be quite appropriate to PCT reorganization issues. How do you prevent
it happening when "we" are ECSs?

···

---------------

"We trained hard, but it seemed that every time we were beginning to
form up into teams, we would be reorganized. I was to learn later in
life that we tend to meet any new situation by reorganizing; and a
wonderful method it can be for creating the illusion of progress while
producing confusion, inefficiency, and demoralization."
        --Petronius Arbiter, 210 BC

------------
Martin

[From Bruce Nevin (980721.2155 EDT)]

In conversation late last night with Peter Cariani I got some very
interesting and suggestive answers to a number of questions. Here's one.

Consider that what we call reorganizing is a normal activity of nerve
cells, which is inhibited when they are participating in a stable
(homeostatic) system, such as a control loop. In other words, the
conditions of embryonic development don't ever go away, they are just held
in check as cells act to stabilize systems that are working. An hypothesis
that could be tested is that a persistent error condition results in
neurochemical changes in the environment of cells involved in the
error-generating loop, either the diminution of some factor that
accompanies a stable system, or the increase of some factor that is present
during reorganization but reduced with stable control.

  BN

[from Mary Powers (980804)]

Bruce Gregory says (980802.0920 EDT)

"Another example of the limitations of PCT is revealed by the number of
times 'reorganization' is invoked {etc.,etc.,etc.]"

Haven't you noticed that the frequency of invoking reorganization is
inversely proportional to the invoker's familiarity with and understanding
of PCT?

"[PCT] isn't a model of everything. It's a model of control...the nasty
remarks I have made...are expressions of my frustrations that PCT is not a
theory of everything."

Yes, PCT is a model of control. That makes it real useful in studying
living control systems. If everything a living control system does is
control, then PCT is a (unfinished, improveable) model of everything a
living control system does. If the everything you are referring to includes
the inanimate world, astrophysics, etc., then PCT is certainly not a theory
of everything. What do you have in mind when you say "everything"?

Best,

Mary P.

[From Bruce Gregory (980805.1025 EDT)]

Mary Powers (980804)

Bruce Gregory says (980802.0920 EDT)

"Another example of the limitations of PCT is revealed by the number of
times 'reorganization' is invoked {etc.,etc.,etc.]"

Haven't you noticed that the frequency of invoking reorganization is
inversely proportional to the invoker's familiarity with and understanding
of PCT?

I don't feel that I'm in a position to judge that. I avoid the term and I
understand very little about PCT.

"[PCT] isn't a model of everything. It's a model of control...the nasty
remarks I have made...are expressions of my frustrations that PCT is not a
theory of everything."

Yes, PCT is a model of control. That makes it real useful in studying
living control systems. If everything a living control system does is
control, then PCT is a (unfinished, improveable) model of everything a
living control system does. If the everything you are referring
to includes
the inanimate world, astrophysics, etc., then PCT is certainly
not a theory
of everything. What do you have in mind when you say "everything"?

There is a huge step to the conclusion that everything a living control
system does is control. I don't think we have anywhere near the evidence we
would need to support this conclusion. Adopting it as a working hypothesis,
however, does make sense. I'm just not sure we can explain activities such
as deciding or choosing as examples of control processes. I'd love to see
this done, but it seems unlikely to happen in the near future.

Bruce Gregory

[From Bjorn
Simonsen (2005.09.25; 11,02 EST)]

From Bill
Powers (2005.08.19.1316 MDT)

That’s
true, but does it have to be true always? I’m thinking of

cases
where not all the information necessary to form a perceptual

signal
is present, yet we go ahead and control things in terms of

what is
present. What if we simply supply missing bits from

imagination
so as to get the error to zero? This would be true with a

single
signal is constructed from multiple lower-order signals; some

of
those lower-order signals might be imagined.

[From Rick
Marken (2005.09.02.1415)]

Yes.
This helps. I think this model might fit into my model of error. The

higher
level system wants to see a particular result, but if it’s taking too

long
to get that result, the higher level system somehow forces the lower

order
system – the one*(s) that can’t give
the higher level system what it

wants
– into imagination mode. Maybe an
error study is the way to see if

this
kind of “fill in” imagination actually occurs.

When I
first time studied the Reorganization model in BCP, I immediately experienced
an aha-experience. It was so self-evident that a lengthening of a child’s leg
during maturation demanded reorganization, etc.

It was
neither a problem for me to understand that learning how to solve a
second-degree equation demanded reorganization. But it was odd to me that this
reorganization was an effect of the intrinsic state of my organism. Maybe it
was a rise in blood pressure when I experienced the anticipation the first
minutes when the teacher taught about the second-degree equation. Maybe the
raise in blood pressure was the intrinsic signal that made an error with the
intrinsic reference (which I don’t understand what was) and initialised reorganization.
Who knows?

But your
comment, Bill, raise some associations within me.

If I enter
a sailboard and will teach myself how to handle it, I perceive some
perceptions. I tumble in the sea and start again. Now I imagine some
configurations, transitions and more and then I make it.

The
imaginations were perceptual signals directed upwards together with perceptual
signals initiated by extern quantities. These signals were different than the
signals directed upwards when I did not imagine how to handle the sailboard. Maybe these differences didn’t form error
signals some places and did make error signals other places. Maybe these
differences caused reorganization?

Where are
the interior signals and the interior references in this case?

bjorn

[From Bill Powers (2005.09.25.0840 MDT)]

Bjorn Simonsen (2005.09.25; 11,02 EST)--

When I first time studied the Reorganization model in BCP, I immediately experienced an aha-experience. It was so self-evident that a lengthening of a child's leg during maturation demanded reorganization, etc.

It was neither a problem for me to understand that learning how to solve a second-degree equation demanded reorganization. But it was odd to me that this reorganization was an effect of the intrinsic state of my organism. Maybe it was a rise in blood pressure when I experienced the anticipation the first minutes when the teacher taught about the second-degree equation. Maybe the raise in blood pressure was the intrinsic signal that made an error with the intrinsic reference (which I don't understand what was) and initialised reorganization. Who knows?

What do you include in "intrinsic state?" In B:CP I said that it includes variables like blood pressure, blood sugar, vasoconstriction, and other such physiological variables, but also I said that it might include higher-level variables. In Ch. 14, page 195, you will find:

···

=====================================================================
I have spoken about intrinsic reference levels as if they specify nothing more than the operating points of vegetative functions. That is almost certainly too limiting. As long as we do not try to invent reference levels having to do with aspects of behavior that could not be inherited (such as a reference level for car driving) we are free to try out any proposal. For example, it is feasible to think that the reorganizing system senses certain types of signals in the learned hierarchy, independent of specific perceptions of [or] behaviors. Total error signal would be such a piece of information. ... There may be intrinsic reference levels pertaining to rather abstract needs, such as the need for some degree of harmony or beauty in our perceptions, necessarily vague and general attributes."

As long as you can make a case for the inheritability of an intrinsic perceptual variable, you can propose a corresponding intrinsic reference signal. So to account for reorganization at higher levels such as the program level where you learn, for example, the strategies for solving equations, you can propose that when any kind of program is not working right, an intrinsic error will appear that can initiate reorganization. We have to say "any kind" because while we can assume that the ability to form program-level control systems in general is inherited, we can't assume that any particular program-level control system is inherited. That is, we can't assume that we inherit an intrinsic perceptual signal standing for the presence of Newton's Method for obtaining square roots, or a reference signal saying that a non-zero amount of that perception should be present. However, as we try to make a program work, we may experience a kind of error that signals something wrong at the program level, and that error might drive reorganization at that level.

The reason I didn't go more deeply into that sort of complication is simply that there is an endless number of proposals of this kind that one could make, but none that we have any way of testing. There is also the danger of creating an all-purpose explanation that can be brought out whenever there is the slightest difficulty, much as "evolution" or "history of reinforcement" is used as a garbage can into which to throw all difficulties that one's theories fail to cover.

But your comment, Bill, raise some associations within me.
If I enter a sailboard and will teach myself how to handle it, I perceive some perceptions. I tumble in the sea and start again. Now I imagine some configurations, transitions and more and then I make it.

Why would just imagining those perceptions allow you to handle a sailboard? You must learn to produce actions that alter real (that is, sense-based) perceptions at various levels in such a way that the sailboard is mastered. Imagining might lead you to realize that you must do certain things. You might realize that you can't hold onto the mast with both hands and manage the sail at the same time. In imagination, you might then work out a way of using your hands that is feasible. But then you must learn to actually make your real-time perceptions of your hands behave the way you imagined them to behave, and at first the real perceptions do not behave that way. You will be able to control the real perceptions only after some reorganization. Controlling imagined perceptions is easy, since as soon as you imagine what you want, it is accomplished. But controlling real perceptions is not easy because you have to adjust reference signals for lower systems, and they have to act through the outside world to alter all the lower-level perceptions involved.

This brings me to your objections to terms like "right" and "wrong." I think you're confusing different ways of using these words. If you say that it's wrong to kill someone, you're describing your own preference, and recommending that others adopt the same preference. But if you say it's wrong to shoot someone in the head and expect them to live, you're making a prediction: you're saying that in your experience, almost everyone who is shot in the head will die, so saying they will live is wrong. It's not morally wrong, but factually wrong, and you can cite evidence to show it's wrong.

It's in this second meaning that we can say perceptions or imaginings can be right or wrong. If you imagine shooting someone in the head and seeing that person walk away alive, your imagination is incorrect. If you actually try this, you will most probably discover the perceptions you imagined were incorrect -- they don't happen. If you actually perceive this happening rather than imagining it, you should seriously wonder if there is something wrong with your perceptual input functions, as you would do if you actually saw a lady being sawed in half.

This gets more complicated when you imagine things that do not exist even when not being imagined -- when you perceive them using your senses. I used the example of the taste of lemonade in B:CP. There is a certain mixture of oils, acids, and sugars that tastes "just right" to you. But these oils, acids, and sugars do not form some new chemical compound called "lemonade." They simply exist in the same volume, stimulating different sensory nerves. Your brain then constructs a perception you call "lemonade" by applying a certain input function to the signals coming from the individual sensory nerves. One state of this perception is the one you prefer. That's the "right" taste. If someone puts too much or too little sugar in the liquid, you get the "wrong" taste.

So there we have "right and wrong" in terms of preferences, reference levels. But now bring imagination into the picture. You make a glass of lemonade, adding just the right amount of lemon juice and sugar to the water. As you pick up the glass your mouth waters and puckers as you imagine the delicious tang of the cold drink. And when you take the first sip, you discover that what you thought was the sugar was really salt. Now you find that your imagined perception was wrong -- it was a wrong prediction of what you would taste if you actually ingested what was in the glass.

Yet if you imagined "correctly" -- that is, if the real taste turned out to be exactly what you had imagined -- that would still not mean that the taste was "right" in any objective terms. There is no such thing as an objective taste; tastes are sensations manufactured by perceptual input functions of second order as they receive first-order intensity signals.

The point of all this is that you have to distinguish between "right" as a moral judgement, "right" as a perception matching a reference level. and "right" as a prediction of a sense-based experience. There are eight combinations of these three kinds of right/wrong judgments, and each of them applies in some situation. Shooting someone in the head and killing them is morally wrong (I say); I perceive myself as not shooting anyone, and that is what I consider the right value of that perception; it is right to say that shooting people in the head will kill them (even if one or two might live). So that combination is wrong-right-right, or 011.

When we say that some theories are more correct than others, what we mean is that some theories are better at predicting what we will experience when we actually do something or observe something being done. Presumably, there is a real world out there, and theories that predict experiences more accurately than others do are probably, in some way, closer to being correct descriptions of the real world. We can't prove that, of course, but it has a logical appeal.

Since we don't know how correct any theory is (we'd have to have some way of looking directly at reality without using our senses to make that judgment), all we can really say is (a) whether one theory predicts better than another, and (b) whether or by how much any theory predicts incorrectly. It's easy to tell when, or by how much, a theory predicts incorrectly: it predicts something we don't experience, or predicts that we will not experience something that we do experience. More generally, there is always some degree of difference between what is predicted in imagination or by calculation (calculation being systematic imagination) and what is then experienced, If the difference is small, we say the prediction, or the means we used to obtain the prediction, is correct. If it's large, we say it's incorrect. "Small" and "large" allow some leeway for accepting less than perfect predictions, and rejecting predictions that to some trivial degree predict correctly (as for hypotheses that"explain" 10% of the variance).

I hope this rather long response clarifies some matters rather than making them more confusing.

Best,

Bill P.

[From Rick Marken (2005.09.25.0950)]

Bill Powers (2005.09.25.0840 MDT)--

The point of all this is that you have to distinguish between "right" as a moral judgement, "right" as a perception matching a reference level. and "right" as a prediction of a sense-based experience. There are eight combinations of these three kinds of right/wrong judgments, and each of them applies in some situation. Shooting someone in the head and killing them is morally wrong (I say); I perceive myself as not shooting anyone, and that is what I consider the right value of that perception; it is right to say that shooting people in the head will kill them (even if one or two might live). So that combination is wrong-right-right, or 011.

I hope this rather long response clarifies some matters rather than making them more confusing.

For me, it wasn't so much a matter of clarifying as it was a matter of nicely articulating what I already knew but had never before...er... articulated;-) I give it a 111: Explaining this is morally right, I perceive your explanation to be right inasmuch as it matches my reference for such an explanation, and it is right because there are really these three senses of being right.

Best

Rick

···

--
Richard S. Marken
marken@mindreadings.com
Home 310 474-0313
Cell 310 729-1400

[Martin Taylor 2005.09.25.14.03]

[From Rick Marken (2005.09.25.0950)]

Bill Powers (2005.09.25.0840 MDT)--

The point of all this is that you have to distinguish between "right" as a moral judgement, "right" as a perception matching a reference level. and "right" as a prediction of a sense-based experience. There are eight combinations of these three kinds of right/wrong judgments, and each of them applies in some situation. Shooting someone in the head and killing them is morally wrong (I say); I perceive myself as not shooting anyone, and that is what I consider the right value of that perception; it is right to say that shooting people in the head will kill them (even if one or two might live). So that combination is wrong-right-right, or 011.

I hope this rather long response clarifies some matters rather than making them more confusing.

For me, it wasn't so much a matter of clarifying as it was a matter of nicely articulating what I already knew but had never before...er... articulated;-) I give it a 111: Explaining this is morally right, I perceive your explanation to be right inasmuch as it matches my reference for such an explanation, and it is right because there are really these three senses of being right.

I second Rick's appreciation of the essay, but I can't go along with the 111 judgment. I'll object to each of the numbers. I think that all these values (and especially the last -- objectively right) should not be binary ones or zeros. To start with, nobody knows (as Bill said) whether a perception is objectively in correspondence with the "real world".

Bill said:

When we say that some theories are more correct than others, what we mean is that some theories are better at predicting what we will experience when we actually do something or observe something being done. Presumably, there is a real world out there, and theories that predict experiences more accurately than others do are probably, in some way, closer to being correct descriptions of the real world. We can't prove that, of course, but it has a logical appeal.

I do think PCT shows a way into the fog of philosophy about objective reality, but I'm not going into that here (and probably not in any near future, either). I'll just say that for me, the last number of the "111" is somewhere around 0.8 (but that's just my imagination, isn't it?). Most of it seems right to me; some is lost in the necessary ambiguity of language; probably what Bill wanted to say is ill-defined in his mind; and of what Bill asserts, almost certainly some will be found to be dubious or objectively insupportable. We can't know, so I can only go with my imagination of what seems right to me, multiplied by the effects of the caveats.

Now consider the first number of the triad. What does "right" mean as a moral judgment? Rick says (as if it were an statement of objectively verifiable fact): "Explaining this is morally right." It isn't. Explaining this is something that Rick perceives as _being_ morally right. And what does that mean? I interpret it as meaning that Rick imagines that if he had done the explaining himself, had he been in the position in which he imagines Bill to have been, the error signal in his control of perception of his own self-image would not have increased. (Or if not "perception of self-image" some other perception up at the principles level or thereabouts).

In other words, "morally right" seems to mean soemthing along the lines of correspondng to the reference value for a pretty high-level perception one is controlling in oneself. Or maybe it means reducing the error in one's self-image perceptual control, which is a rather stronger statement than just not increasing it. Or again, it could mean that if only a finite set of actions seem to be available, the most moral is the one that results in the least increase in the error in one's self-image control system. Whichever it might be, to judge someone else's actions a morally right depends on how one imagines one's sefl-image perception would have been affected had one performed those actions oneself, having been in the position of the other person.

That's a little different from the middle kind of "right", Bill's "perception matching a reference level". That does overlap "morally right" insofar as one of the perceptions that someone may be controlling for is that their actions should be morally right, or at least not morally wrong. But it applies at all perceptual levels. It's "right" to perceive one's hand grasping the glass of water from which one wishes to drink, but drinking or not drinking in seldom likely to have much influence on one's self-image. Since there are lots of perceptual levels involved in even such a simple act, the central "rightness" number should be vector-valued.

In the case of Rick observing Bill's essay, I don't think he is in a position to assign a middle "rightness" value, unless he has imagined what Bill was controlling for in publishing the essay, and has observed whether its publication affected Bill's perceptions as Bill intended.

After all that, I'm going to assign my own values in place of Rick's "111", as follows: 0.9, {x,x,...}, 0.8. (The "x's" are not totally arbitrary, though, since at the lower levels where the actions are things like keystrokes, or even word selection, it seems clear that their values should be very close to 1.0).

I hope I've confused things sufficiently :slight_smile:

Martin

[From Bill Powers (2005.09.25.1652 MDT)]

Martin Taylor 2005.09.25.14.03 --

After all that, I'm going to assign my own values in place of Rick's "111", as follows: 0.9, {x,x,...}, 0.8. (The "x's" are not totally arbitrary, though, since at the lower levels where the actions are things like keystrokes, or even word selection, it seems clear that their values should be very close to 1.0).

I hope I've confused things sufficiently :slight_smile:

Nicely confused; couldn't have done better myself. I agree that the "moral" right-wrong dimension is probably just a high-level reference signal. I was thinking, however, more in terms of translating from NormalSpeak into PCTspeak. Lots of people want moral judgments to have some objective or authoritative basis, so they can make pronouncements about what is REALLY right and wrong. My first classification was directed toward that idea. The idea of right and wrong as preferences, which as you say probably refers more to lower-order perceptions, is less controversial: "that piece of fish doesn't smell right to me." And the last, which has to do with predictions rather than objective reality, is just about science. I ended by putting the rightness-wrongness on a continuous scale, as you do, by introducing the idea of "large" and "small" errors of prediction. Perhaps you and I are the only ones who are not confused by this.

Best,

Bill P.

[From Rick Marken (2005.09.25.1800)]

Martin Taylor (2005.09.25.14.03) --

What does "right" mean as a moral judgment? Rick says (as if it were an statement of objectively verifiable fact): "Explaining this is morally right." It isn't. Explaining this is something that Rick perceives as _being_ morally right.

Good point. I should have been clearer because I don't really believe that such a judgment is objectively verifiable. I think that that first "right" in Bill's three tuple is just the setting of a reference from the behaving system's point of view. So this applies at all levels, not just at the level of morality (principles). What is "right" in this first sense of right, I believe, is the way the system itself wants (has references for) its experiences to be. So systems that want olives (which I can't stand) or war (which I also can't stand) are"right" when they say olives are good or war is justified, in Bill's first sense of "right", since they have references for olives and war.

Best

Rick

···

---
Richard S. Marken
marken@mindreadings.com
Home 310 474-0313
Cell 310 729-1400

[From
Bjorn Simonsen (2205.09.26,14:35 EST) ]

[From
Bill Powers (2005.09.25.0840 MDT)]

From BCP

“….For
example, it is feasible to think that

the
reorganizing system senses certain types of signals in the

learned
hierarchy, independent of specific perceptions of [or]

behaviors.
Total error signal would be such a piece of information.

… There
may be intrinsic reference levels pertaining to rather

abstract
needs, such as the need for some degree of harmony or beauty

in our
perceptions, necessarily vague and general attributes."

Yes,
reading your mail and BCP again I don’t see any difference between a perception
of physiological variables and perception referring to not extinct errors when
a perception is controlled. This control can result in reorganization on all
levels.

I think I
am able to explain reorganizing better today. Thank you.

But
your comment, Bill, raise some associations within me.

If
I enter a sailboard and will teach myself how to handle it, I

perceive
some perceptions. I tumble in the sea and start again. Now

I
imagine some configurations, transitions and more and then I make it.

Why
would just imagining those perceptions allow you to handle a

sailboard?
You must learn to produce actions that alter real (that

is,
sense-based) perceptions at various levels in such a way that the

sailboard
is mastered. Imagining might lead you to realize that you

must
do certain things. You might realize that you can’t hold onto

the
mast with both hands and manage the sail at the same time. In

imagination,
you might then work out a way of using your hands that

is
feasible. But then you must learn to actually make your real-time

perceptions
of your hands behave the way you imagined them to behave,

and
at first the real perceptions do not behave that way. You will be

able
to control the real perceptions only after some reorganization.

Controlling
imagined perceptions is easy, since as soon as you

imagine
what you want, it is accomplished. But controlling real

perceptions
is not easy because you have to adjust reference signals

for
lower systems, and they have to act through the outside world to

alter
all the lower-level perceptions involved.

I think
you misunderstood my comment above. I expressed myself not very well.

My point is
that I can sit in my chair in my house and imagine how it is to master a
sailboard. It may be easy to work out a way to use hands that are feasible. But
this is just imagination with none reorganizing.

When I
imagine the same on the sailboard in open sea, copies of imagined perceptions
travels to higher levels and initiates new outputs and new loops. Reorganizing.

And my
point, after your mail, is that imaginations put into real time may provoke
intrinsic signals from perceptions of not extinct errors when I train myself on
the sailboard.

Imagination
may via not extinct errors result in reorganization. Is that wrong?

This
brings me to your objections to terms like “right” and
“wrong.”

I
think you’re confusing different ways of using these words. If you

say
that it’s wrong to kill someone, you’re describing your own

preference,
and recommending that others adopt the same preference.

But
if you say it’s wrong to shoot someone in the head and expect

them
to live, you’re making a prediction: you’re
saying that in your

experience,
almost everyone who is shot in the head will die, so

saying
they will live is wrong. It’s not morally wrong, but factually

wrong,
and you can cite evidence to show it’s wrong.

Maybe you
are right, but aren’t you just talking about controlling perceptions on
different levels?

If I say
it is wrong to kill someone and will form a group of people thinking the same.
Then I or we control our perceptions at the System concept level.

If I say
it’s wrong to shoot someone in the head and expect them to live, I control my
perceptions at the Relationship level (I think).

It’s
in this second meaning that we can say perceptions or imaginings

can
be right or wrong. If you imagine shooting someone in the head

and
seeing that person walk away alive, your imagination is

incorrect.
If you actually try this, you will most probably discover

the
perceptions you imagined were incorrect – they don’t happen. If

you
actually perceive this happening rather than imagining it, you

should
seriously wonder if there is something wrong with your

perceptual
input functions, as you would do if you actually saw a

lady
being sawed in half.

In what you
call the second meaning, perceptions are controlled on lower levels, not
logical levels.

Of course
there are perceptions we can’t bring to our reference values. There are many,
many of them. Some times this results in reorganization. Some times we dye before
reorganization is effective (BCP).

I think
it is more PCT-ish if we say that it is difficulty to bring our perceptions to
the reference level than saying that some perceptions are wrong.

Let me
name another example. Mr: A controls his perceptions when he talks with Mr. B
standing next to him. 100 people standing around Mr. A can’t see nor hear Mr.
B. They say there is something wrong with Mr. A’s perceptual input functions.
They think as Aaron T. Beck in his "Cognitive Therapy and the Emotional
Disorders” and say “me doctor, you patient” and they are the one who
shall do the correcting.

I agree
with Martin when he say that PCT shows the way out of (my words) the fog of philosophy when we say that
people control their perceptions and don’t say that people have wrong
perceptions.

This
gets more complicated when you imagine things that do not exist

even
when not being imagined – when you perceive them using your

senses.
I used the example of the taste of lemonade in B:CP. There is

a
certain mixture of oils, acids, and sugars that tastes “just right”

to
you. But these oils, acids, and sugars do not form some new

chemical
compound called “lemonade.” They simply exist in the same

volume,
stimulating different sensory nerves. Your brain then

constructs
a perception you call “lemonade” by applying a certain

input
function to the signals coming from the individual sensory

nerves.
One state of this perception is the one you prefer. That’s

the
“right” taste. If someone puts too much or too little sugar in

the
liquid, you get the “wrong” taste.

Brewing
my lemonade, I will prefer to say that I control the taste of my lemonade. Of
course it takes some time and of course I meantime perceive tastes referring to
not extinct errors.

My point
is that I control my perceptions, I don’t have a wrong perception until it
reaches the reference level.

…………….There
is no such

thing
as an objective taste; tastes are sensations manufactured by

perceptual
input functions of second order as they receive

first-order
intensity signals.

Yes I
agree from the time I read BCP first time in 1998.

The
point of all this is that you have to distinguish between “right”

as a moral judgement, “right” as a perception matching a
reference

level.
and “right” as a prediction of a sense-based experience. There

are eight combinations of these three kinds of right/wrong judgments,

and
each of them applies in some situation. Shooting someone in the

head
and killing them is morally wrong (I say); I perceive myself as

not
shooting anyone, and that is what I consider the right value of

that
perception; it is right to say that shooting people in the head

will kill
them (even if one or two might live). So that combination

is
wrong-right-right, or 011.

I
understand what you say, but I don’t see the profit using the concepts “right”
and “wrong”. If you perceive yourself as not shooting anyone, that’s the way you
perceive yourself. If another person perceives himself as a gunner (shooting
terrorists is morally right (he says)), he controls other perceptions than you.

So is the
world.

I
understand that it would be problematic for people controlling opposite perceptions
to establish social contact. But that is another problem. PCT can teach them
how to handle conflicts.

When
we say that some theories are more correct than others, what we

mean
is that some theories are better at predicting what we will

experience
when we actually do something or observe something being

done.
Presumably, there is a real world out there, and theories that

predict
experiences more accurately than others do are probably, in

some
way, closer to being correct descriptions of the real world. We

can’t
prove that, of course, but it has a logical appeal.

Yes I
know you prefer classical mechanic before Quantum mechanics.

The way I
see it is that theories are our meanings about the extern world. I have my
theories and you and other people have your/their theories. When we control our
perceptions, the theories are references in our systems. Some of us control
perceptions referring to extinct error. Other people control perceptions
referring to not extinct errors. They sometimes reorganize, they sometimes die
or they some times live a conflicted life. Both parts control their perceptions
as well as they can. Which prise do we obtain by using the concepts “right” and
“wrong”? What is wrong today is maybe right tomorrow.

More generally,
there is always some degree of

difference
between what is predicted in imagination or by calculation

(calculation being systematic imagination) and what is then

experienced,
If the difference is small, we say the prediction, or

the
means we used to obtain the prediction, is correct. If it’s

large,
we say it’s incorrect. “Small” and “large” allow some
leeway

for
accepting less than perfect predictions, and rejecting

predictions
that to some trivial degree predict correctly (as for

hypotheses
that” explain" 10% of the variance).

I liked
this section better. Here you talk about differences and that is an adequate
concept for me.

I
hope this rather long response clarifies some matters rather than

making
them more confusing.

Yes I appreciated
your long response. I understand very well what you say and I can live with
“right” and “wrong”, “correct” and “incorrect”. But I become enthusiastic over
the way PCT/HPCT explains why people do as they do. They control their
perceptions. They neither control my perceptions nor the doctor’s perceptions.
They are on their way. Why shall we emphasize that their perceptions are wrong?

I hope
you understand that my repugnance against those adjectives is rooted in the way
I understand PCT. (Maybe that’s the problem)

bjorn

···

From Tom Bourbon [930902.0900]

The discussion between Hans Blom and Rick Marken includes some
interesting topics.

[Hans Blom, 930902]

(Rick Marken (930901.0730))

[Rick = >>]

Apparently "adaptive control" in control engineering is exactly the
same as "reorganization" in the reverse engineering of living systems
done by PCT.

[Hans = >]

···

Exactly, though it is not as random as PCT proposes. The basic diagram
of an adaptive controller is as follows:

     --------------------------------------------------
     > >
     > p ------------- ------------ |
     ---->|- basic | u | system | p |
r | controller|---------->| to be |---------
-------->|+ | | |controlled| \|/
          ------------- | ------------ ------------
               /|\adjust | ---------|comparator|
                > > \|/adjust ------------
                > > ------------ /|\
                > > > model | pm |
                > ---->| of the |---------
                > > system |
                > ------------ r = reference level
                > > u = action / behavior
                ------------------------- p = perception
                                                pm = predicted perception

The standard control setup is the top part. Notice a second controller
in the bottom part; it controls for maximum correlation between p and
pm. The model generates the adaptive controller's "expectation" of what
the reaction of the controlled system will be. The second controller,
the adaptation mechanism, compares the expectation with what is
observed. The "expectation error" adjusts the model of the system in
such a way that the model's output converges toward the system's output.
This convergence will generally not be exact, of course, since it
depends on the model's quality (amongst others, the number of parameters
/ degrees of freedom that can be tuned) and the signal to noise ratio.
The adjustment of the model is usually done by some least squares or
hill-climbing method. Given the model, the parameters of the "basic
controller" can be adjusted (for example, in case of an ECS, its gain
and slowing factor; in case of a PID-controller, its P, I and D
parameters).

Tom:

Hans, the standard diagram you presented, and your description of what it
does, are a little different from an adaptive PCT model I tried a couple
of years ago. The project was one I put aside during a period of career
changes and moves. I am a former experimental psychologist turned
tinkerer with PCT model parts, not an engineer or mathematician. I did
not even know. for example, that what I was playing with should be called
"adaptive control." (I mention those points only as a defense-in-advance
against obvious deficiencies in the language and terminology I might use
to describe my model, and as an invitation for you to help educate me on
the subject of adaptive control.)

I started work on the adaptive PCT model as part of my modeling of
interactions between control systems (people, models, person-model, etc).
In particular, I was working on tasks in which one system acts either to
aid, or to control the actions of, another. The interactions occurred
during a simple pursuit tracking task. First, I will describe the task,
then I will show you my diagram of the PCT models. Maybe you can tell me
what I should call it -- but only if the name can be repeated in public.

One person or PCT model (A) uses a control handle to keep a cursor (c)
aligned with a target (t). A second person (B) can also affect the
cursor, by using a second control handle. The position of the target is
determined solely by values from a table containing a smoothed series of
random numbers (the target function, T). The position of the cursor is
determined in a more complex manner by the sum of the scaled position of
control handle A, plus the present value from a second table of random
numbers (the disturbance, D), plus 1/2 of the scaled position of control
handle B: c = A + D + .5B. (Each set of random numbers contains 1800
values and the correlation between the 1800 pairs of values in the two
series is >= .2.) In the simplest case, described here, person A
attempts to keep the cursor aligned with the target on the computer
screen, that is to say, to keep c-t=0. That is the reference signal (r)
in a PCT model of person A shown below:

           PCT MODEL OF PERSON A

                   r = [c-t = 0]
                   !
                  \!/ c = cursor position
   present !!!!!!! t = target position
   value !!>! C !!!! T = function for t
   of c-t = p !!!!!!! e = p-r D = disturbance on c
             ! \!/ A = position of handle A
          !!!!!!! !!!!!!! r = reference signal
          ! I ! ! O ! e = error signal
          !!!!!!! !!!!!!! p = perceptual signal
          /!\ /!\ ! C = comparator
           ! ! ! I = input function
           t c <!! A <!! O = output function
          /!\ /!\ c := A + D
           ! ! A := A - k(e), where k = integration
           T D factor estimated from person's data.

(Hans, notice that I have rearranged the diagram of the control system in
canonical PCT form, one of the differences between PCT and standard
control engineering. At first glance, this difference in diagrams might
seem trivial, but I believe that I can show how differences in the
diagrams are suggestive of differences, some slight and others more
significant, in our interpretations of control.)

This model reproduces and predicts a person's tracking data to a high
degree of accuracy (correlations between 1800 pairs of model's and
person's handle positions >= .995). So much for the model of person A.

Let us consider the case in which person B attempted to "control" the
actions (handle movements) of person A, by adding an influence to the
position of c. Person A moves handle A to keep c aligned with t, while c
and t are each affected by independent random disturbances. Person B
intends to see the hand and handle of person A moving back and forth with
their position tracing a triangular function [/\/\/\].

The revised diagram:

            PCT MODEL OF PERSON B PCT MODEL OF PERSON A

                   r = A-[/\/\/\]=0 r = [c-t=0]
                   ! !
                  \!/ \!/
   present !!!!!!! present !!!!!!!
   value !!>! C !!!! value !!>! C !!!!
   of = p !!!!!!! e = p-r of c-t = p !!!!!!! e = p-r
   A-/\/\ ! \!/ ! \!/
          !!!!!!! !!!!!!! !!!!!!! !!!!!!!
          ! I ! ! O ! ! I ! ! O !
          !!!!!!! !!!!!!! !!!!!!! !!!!!!!
            /!\ ! /!\ /!\ !
             ! ! ! ! !
             ! ! t ! !
             ! ! /!\ ! !
             ! ! T ! !
             ! ! ! !
             ! !!!!!> B !!!!!!!!!!!!!!!> c <!! A <!!
             ! /!\ !
             ! D !
             ! !
             !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Two people can perform in this mode quite easily, as can two PCT models,
or a person interacting in real time with a PCT model. The cursor is now
disturbed by the sum of D and .5B and A keeps c-t=0. At the same time,
A=/\/\/\. B has ignored c and moved handle B, creating just the right
additional disturbance on c to require /\/\/\/\ from A. There is still
no adaptive control in the model diagrams, but I believe some of the
important distinctions between traditional and PCT models of interactive
adaptive control are already apparent. For one thing, person/model B has
no "model" of person/model A and no "expectations" of what the actions of
A will be . Instead, B (the "controller" of A) has an intended
perception of the actions of A. B does not directly affect the actions
of B; rather, B affects (disturbs) a variable that is part of a
relationship controlled by A. Rather than controlling for a correlation
between the actions of A and a model of A, B directly controls its own
perceptions of the actions of A, relative to B's intended perceptions of
those actions.

Adaptive control. When two people perform this task for the first time,
it is common for B to overdo, or underdo, it. In the first case, B moves
handle B so vigorously that A has no chance of controlling c-t. A either
goes "random," or moves handle A through a few small oscillations, or
abruptly stops tracking altogether, or stops after first doing a few
"test wiggles" of the handle to determine if it is still working. In the
second case, B moves handle B so little that A continues tracking with
handle movements determined by the two random functions acting on c and
t; B is not creating an "effective" disturbance on c. In either case, an
adaptive person B begins to adjust movements of handle B until there is a
match between B's perceived and intended movements of A. B does not
alter the reference perception (reference signal) for the intended
movements of A; rather, B changes the degree to which it moves handle B,
given a particular discrepancy between perceived and intended movements
by A -- B changes the gain on its output function, which, in the models I
use, is represented as a change in k. Can that adaptation be modeled in
a manner rigorously consistent with the PCT model? I believe it can.

In my initial attempts at modeling adaptation, I used a second control
loop in model B, monitoring the perceptual signal (which stands for the
positions of A) in the primary loop in B. Whenever p was >= X (where X
= an arbitrary upper criterion value), I systematically reduced k (in the
Output function of B) until e < X. Conversely, when p was <= Y (where Y
= an arbitrary lower criterion value), I systematically increased k (in
the Output function of B) until e > Y. I posted my results to Bill
Powers. Everything that occurred after that point is a direct result of
his reply to my post; I cannot claim anything that follows in the present
post as "my" idea. If I misrepresent Bill's insights, which I consider
brilliant, he should pounce on my errors and set them straight.

Bill asked if I had tried the "E. coli" method and used the "adaptor
loop" to monitor the error signal in B. I had not. By the "E. coli"
method, he meant the literal computational steps Rick Marken and he had
used in their paper: R. S. Marken and W. T. Powers, "Random-walk
chemotaxis: Trial and error as a control process." Behavioral
Neuroscience, 103, 1348-1355. (Reprinted in R. S. Marken (Ed.), _Mind
Readings_, available from CSG Publications, in care of Greg Williams, who
is on this net. No, I do not receive a royalty.)

As modified for my tracking tasks, the "adaptor loop" has a perceptual
function that computes the present change in the error signal in the
primary control lop in model B. In the adaptor loop, the computational
steps, which approximate the time integral of error, (1) calculate the
present CHANGE in the error signal, (2) add that change to the previous
total of changes, (3) determine if the new total > a criterion value, (4)
if the new total > the criterion, re-set the total to zero and produce a
random change in "something." In their chemotaxis paper, the change was
to a randomly selected new direction of travel for the simulated E. coli
bacterium; here, I made it a randomly-selected signed magnitude of change
in the integration factor, k, in the Output function of the primary
control loop in model B. Said another way, in B, when error in the loop
controlling A relative to /\/\/\/\ integrated to a criterion value, a
change, of random magnitude and sign, was added to the integration factor
in the Output function.

The revised diagram, including the adaptor loop in model B:

            PCT MODEL OF PERSON B PCT MODEL OF PERSON A

                   r = A-[/\/\/\]=0
                   !
                   ! r=[if q > Q, dk = rand dk]
                   ! \!/
                   ! !!!!!!!!!!!!!
                   ! ! i ! c ! o ! r = [c-t=0]
                   ! !!!!!!!!!!!!! !
                  \!/ /!\ ! \!/
   present !!!!!!! ! ! present !!!!!!!
   value !!>! C !!!! ! value !!>! C !!!!
   of = p !!!!!!! e=p-r ! of c-t = p !!!!!!! e = p-r
   A-/\/\ ! \!/ ! ! \!/
          !!!!!!! !!!!!!! dk !!!!!!! !!!!!!!
          ! I ! ! O !<!!!! ! I ! ! O !
          !!!!!!! !!!!!!! !!!!!!! !!!!!!!
            /!\ ! /!\ /!\ !
             ! ! ! ! !
             ! ! t ! !
             ! ! /!\ ! !
             ! ! T ! !
             ! ! ! !
             ! !!!!!> B !!!!!!!!!!!!!!!> c <!! A <!!
             ! /!\ !
             ! D !
             ! !
             !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

In the adaptor loop, q = present sum (integral) of the error signal, e;
dk = random change added to k (in O). B has become an adaptive
controller. In the runs I did, now a couple of years ago, B converged
within a few seconds on a value for k that kept A-/\/\/\ near zero, and A
kept c-t=0. I have not run the model since that time.

Now some of the differences between this particular adaptive PCT model,
and the model described by Hans seen even clearer to me. For one thing,
the equivalent of what Hans called the "second controller" in the
adaptive controller has no model for A and there is no expectation of
what A will do; rather, the "second loop" has a reference signal for the
value of the time integral of sensed error being < a criterion value, and
an output function that adds a random change to the integration factor of
the primary control loop. There is no "optimal" value of anything -- any
k that keeps the integral of sensed error below the criterion value will
"stick." For this loop, anything that is "in the ballpark" is good
enough. This implementation of the E. coli procedure in the PCT model
struck me then, as it does now in recollection, as a powerful
demonstration of random variation and selective retention as a possible
mechanism for learning (and as some on the net have speculated, for
evolutionary, or phylogenetic, change).

The second loop "knows" only the magnitude of the integral of sensed
error, but in this implementation, it did not, and need not, know that to
which the error pertains -- it doesn't need to know where the error comes
from or why it is there, and it does not know "what it is doing" when it
outputs a randomly selected value. The second loop is not modeled as
"intentionally" modifying the gain if another system, or as controlling
the actions of yet another system.

After that long oration, back to Hans's post.

Rick:

But I think the engineers are blowing it big time if they are basing
adaptive control on the correlation of outputs with inputs (as you
described them) since this correlation depends largely on the highly
non-linear and CHANGING environmental (feedback) function that connects
output back to input via the controlled variable.

Hans:

If the environmental function cannot be modelled (is purely random), no
adaptation (and no control) is possible. Otherwise some type of model,
possibly non-linear or changing in time, can be postulated a priori (by
the designer, given "partial knowledge" of the system's transfer
function) or can be discovered by the adaptation mechanism if the
initial model is sufficiently general (has enough knobs to adjust).

Tom:
I, Hans, that we have different definitions and understandings of
adaptive controllers; but from my position of ignorance (genuine lack of
experience and knowledge of your field), I think Model B adaptively
controlled the also randomly disturbed actions of model A. I imagine
"random" will be the key word to address in resolving this apparent
difference in our views. But in the application I described, the
designer (me, on the field, coached from the sidelines by the real
designer) got along without positing a "model of A" inside model B, yet
model B controlled its perceptions of the actions of A.

Rick:

                               The PCT reorganization schemes are
based on the correlation between reference signal (what engineers
usually call "input") and perceptual signal (what engineers often call
"output"). The goal of reorganization (adaptive control) is to keep
the correlation between r and p VERY HIGH (which is the same as keeping
r-p very small).

Hans:

Keeping the difference r-p small is the task of the "basic controller".
Keeping it small when even r changes FAST, such as in a step change, is
the task of the adaptation mechanism.

Tom:
In PCT models, keeping r-p small is the task of *every* controller. The
important distinction is in *which* r and *which* p. In the primary loop
inside model B, r and p have to do with the A and /\/\/\; in the second
loop in model B, r and p have to do with the integral of sensed error
signals.

Rick;

               The problem with using your input-output correlation
as the basis of adaptive control is that sometimes this correlation
MUST be low (because of the feedback function) if control (perception-
reference correlation) is to be high. So an adaptive control scheme
that just tried to maximize the input-output correlation would actually
be REDUCING control in some cases. In PCT terms, isn't adaptive
control (in both control engineering and PCT) necessarily an attempt to
maximize the perception-reference, not the input-output, correlation?

Hans:

I do not understand this argument. We are probably talking about
different correlations. The most accurate knowledge that can be obtained
(through observations) about the feedback function is collected into the
model's parameters through the process of maximizing the correlation
between expectation and reality (pm and p). The model may be nonlinear
and time-varying, yet at all times it ought to characterize the system
to be controlled in the best possible way. In the case of an organism,
the system to be controlled is the world, of course. If the correlation
between what you expect your action to achieve and what you observe as a
result of you action is low, even on average, the world reacts very
unpredictably. How is ANY type of control possible in that case?

Tom:
I believe this exchange between Rick and Hans reveals a huge differences
between PCT models and traditional models of control systems. (I am not
saying that is a good state of affairs -- just that it is a genuine big
difference.) Hans necessarily looks for large correlations between the
actions of the real system and those of the adaptive model's model of the
real system. In work to date on PCT, that procedure corresponds to our
looking for high correlations between the actions of our models and of
the real people we observe. *But in the PCT models themselves*, s for
the people, the actions of the person/model necessarily correlate hardly
at all with the state of environmental controlled variables: to fall
back on what must seem to be the only example I know, in a tracking task
with a disturbance acting on the cursor, handle positions correlate
hardly at all with the position of the cursor; but they correlate highly
and negatively with the values of the disturbance acting on the cursor.
That is the way things must be, else there is no control.

I believe the differences between comments from Rick and Hans in this
exchange stem from very different ideas about who, or what, must see the
various correlations they describe. When they act as designers, modelers
look for high correlations between the actions of real systems and model
systems; but viewed as control systems, the actions of designer-modelers
will correlate very poorly with the results of their actions and the
actions of the designer-modeler will correlate highly and negatively with
anything that disturbs the perception of "model's actions matching real
system's actions." In the traditional approach to adaptive modeling that
Hans described, the modeler seems to work from the assumption that an
adaptive system must be "smart," with a model of the controlled system;
in the PCT approach Rick and I are describing (let me know if you want
out of this assumed association, Rick!), we assume all parts of the
adaptive system should be "dumb," with each loop controlling only its own
perceptions.

This has probably gone on too long.

Again, Hans, if the words and names can be repeated on the net, I look
forward to learning what I was "really" doing!

Until later,

Tom

[From Bill Powers (921005.0900)]

Greg Williams (921003) --

Note that I have included BOTH "learning" AND "reorganization" in
3. Do you think we DON'T need to postulate something "less" than
reorganization in cases which Gary has raised, such as learning how
to multiply (where it appears that (1) the learner's control system
is altered and (2) there is no critical error triggering the
learning)?

Why does it appear that there is no critical error "triggering" the
learning of multiplication? I've looked at this sort of learning the
other way around: multiplication requires learning to control a new
pattern of perceptions; therefore reorganization must occur. If
reorganization occurs, what sort of critical variable departs from its
reference level and is restored by successful learning? This kind of
critical error would have to occur not specifically for learning
multiplication, but for learning anything of that sort, under the
conditions that exist at the time of learning.

There are some obvious answers and some not-so-obvious answers. The
obvious answers involve the coercive atmosphere of school, in which
not learning means confinement, disapproval, and punishment. For most
children the motivation for learning to multiply consists of all the
things that are done to them until they do learn -- more specifically,
the emotional consequences and the underlying state of internal
disorder resulting from the things that are done to them at school and
at home until the time when they demonstrate to someone else's
satisfaction that they know how to multiply.

The not-so-obvious answers would concern mainly those few children who
learn to multiply because they think this is a totally neat new skill
and they can't wait to find out how to do it. Critical variables, as I
have maintained from the start, are not confined to the vegetative
functions, although they are easiest to understand in terms of
biochemical functions that underlie hunger, thirst, and so on. All
that is required to define a critical variable is that it represent a
condition of the organism that is genetically specified, that it can
be detected by a built-in critical-variable detector, and that it lead
to reorganization when it departs from its inherited reference level.

The learning of multiplication would certainly not itself be a
critical variable; that is, there is no built-in reference level for
learning multiplication or any other specific cognitive skill. But
there could be built-in detectors and reference signals that are
satisfied when ANY new skill is acquired, when ANY previously-
experienced perception is brought under skilful control, at ANY level
of organization. That sort of critical reference level could be
inherited, so it could be operational from early during gestation. It
would account for a generalized urge to learn new skills, at any level
including the highest one currently under construction. And it would
satisfy my demand that evolution not be required to function on the
basis of future conditions of the environment that have only ephemeral
existence, such as a particular cultures, languages, or most-admired
sets of skills.

I've proposed one such critical variable, which is simply the absolute
magnitude of the error signal in any control system. It doesn't matter
what this error signal is about; all that matters is that it become as
small as possible.

If teaching relied on the built-in reference signals for critical
variables having to do with learning new skills, it would not be
necessary to force learning by induction of critical errors of
irrelevant kinds. Simply not knowing how to reproduce an observed
skill would be enough, if the skill seemed interesting. Unfortunately,
the educational system is so fundamentally coercive that few students
have time to discover the innate joy of learning. They are too busy
learning how to avoid the penalties of NOT learning that the
educational system arbitrarily imposes.

···

------------------------------------------------------------------
Best,

Bill P.

[Martin Taylor 921009 17:00]
(Bill Powers 921006.1530)

Just by trial and error, I found that the most
reliable method (so far) entails (roughly) making the rate of
reorganization depend on the rate of change of absolute critical error
TIMES the absolute critical error, so that the changes get smaller as
the remaining error diminishes. I realized only afterward that this
amounts to taking the derivative of the SQUARE of the error, which
neatly takes care of assuring positive values and that the result will
give the best least-squares fit to the optimum solution. You might try
this in addition to your (e*e + k*e*de)/G.
In other words, try d(e^2)/G.

Did you try varying the relative contributions of the value of e**2 and
of d(e**2)/dt? In other words, varying k in my expression between zero
and infinity? My formula is yours if k is large, but if k is moderate, then
reorganization could occur with a sustained error that does not change
value, which seems as if it would be a good thing to do.

Another tip. Rather than just make random corrections based on
critical error, I have found that it's best to randomly change a
parameter that determines the direction and speed of changes in
parameters. Between reorganizations, the changes then continue on each
iteration. This is strictly analogous to the E. coli method of
locomotion, in which, between tumbles, the organism continues to swim
in a straight line, continuously altering its relationship to a radial
gradient.

This sounds as though you are doing Hebbian learning (on the output link
weights?) rather than what I characterize as "reorganization" (the changing
of link sign or of the link targets). I can't see what this paragraph
would mean when applied to output links that had weights of only +-1 or zero.
I thought that the simple control nets you were working with all had this
restriction on the output weights, and that you were waiting to be forced
into allowing real-valued output weights.

My intuition has been that you would be forced into using real-valued
output weights when you got into massively parallel control nets in which
no single ECS could be responsible for the whole of any behaviour and there
was permanent tension in the net. But so far as I am aware, you haven't
studied that kind of net yet. Am I wrong? Or did you get into real-valued
weights when you found that sign-flipping had nasty effects? Or are you
talking about perceptual input functions here?

In the Little Baby project, the second (probably) experiment will >be
to start with a random set of output connections, and use Hebbian
learning on the perceptual input functions to see whether the baby
can learn to perceive the world in a way compatible with its (fixed
randomly assigned) outputs.

I can tell you already that this will work, using my method of
reorganizing outlined above, with up to 20 independent control systems
controlling through 20 shared environmental variables. The critical
variable is total squared error across all systems. Reorganization is
applied globally, not based on each system's error.

Again, I'm not 100% sure what you were varying. I had assumed that your
reorganization experiments were restricted to changing the output system
rather than the perceptual input system. If you have studied error-induced
modifications of the perceptual input functions, we would love to know more
about your results. We don't mind re-inventing wheels, but it is nice to
know that our wheels work like yours if we do. (By "we" I mean Chris and me,
though I expect the CSG-L readership would be interested, too).

Martin

[From Bill Powers (941225.1100 MST)]

Dennis Delprato (941220)--

Why is it necessary to posit reorganizing systems? In other
words, cannot the control systems reorganize without the need
for a separate set of reorganizing systems?

There may be several processes that we could call reorganization, but
the basic process which I attribute to a reorganizing _system_ was
proposed for reasons we haven't talked about much.

A basic question is how an animal can learn some completely arbitrary
behavior such as walking in a figure-eight or putting a token into a
slot as a means of getting fed. There is no natural connection between
the learned behavior and its effect on intrinsic variables. This makes a
purely local concept of reorganization (each control system reorganizing
itself) impractical; how can the control systems which grasp things and
put them into various places, or which move the body through spatial
paths, know that they should reorganize to control some variable with
which they are not concerned at all? A "grasping" control system can
optimize itself by means we can easily imagine, but the optimization
applies only to achieving the reference-state of grasping as well as
possible. What does such a control system know about grasping hot
objects, or grasping objects which are not only edible, but nourishing?

We also have to remember that reorganization has to work _before_ the
control systems in the hierarchy are present. A first answer to the
questions raised above is that a higher system reorganizes itself to
make the logical connection between doing a funny walk and receiving the
food needed to maintain a state of fed-ness. But that has to occur
before logic even exists -- even a baby chimp learns to root around on
the mother's breast to find the nipple and get milk. My answer was to
propose a very basic control system charged with maintaining the state
of the body. This system acts by bringing the behavioral systems into
being.

···

----------------------------------------------------------------------
Martin Taylor (941221.1410)--

Firstly, it is not clear that the result of learning IS typically the
maintenance of physiological variables in states required for survival.

I approached the question from the other end. Why is it that learning
can be produced by depriving an organism of something it needs in order
to survive, and then making acquisition of that something contingent on
performing some control process completely unrelated to it (except that
it has an arbitrary effect on acquiring the something)?

This seems to me to be a view that depends on the view of
reorganization that it is intended to support, in other words, a
circular proposition. "Living things are alive" is what it seems to
say, and what they have learned is not incompatible with their staying
alive.

This isn't how the concept arose. It arose because I realized that
organisms are not born understanding the physics and chemistry of the
environment or of their own bodies. Yet human organisms act as if they
know they have to eat and drink certain things in order to stay alive.
The problem becomes much simpler if we say they don't begin by knowing
this: that internal discomfort leads them to reorganize, and the end-
point of reorganization occurs when they have begun to control certain
variables that have a side-effect of keeping their internal nutritional
states near a set of inherited reference levels, removing the
discomfort. Since the supposed reorganizing system is not intelligent,
we don't have to explain how organisms understand their environments
well enough to know in advance what will cure pangs in the stomach. Nor
do we have to explain the logical connection between arbitrary and even
outlandish control processes that are learned and their ultimate
relationship to keeping fed (or whatever the intrinsic variable involved
may be).

In fact, it has seemed to me more than probable that there is no
separate reorganization system.

Then you would be hard-pressed to explain _simply_ how a rat can learn
to press a bar in order to turn off a source of gamma radiation, which
has no effects related to pressing a bar.

Now, this [reorganizing] control system is drawn a little differently
from the usual way of drawing a control hierarchy, because the
"physiological side effects" are signals like any other, though they
are probably chemical rather than electrical.

But they needn't be "signals." They can be actual physical effects like
burning yourself on a hot object or cutting yourself while trying to
close a clasp-knife. They can be effects that result from putting things
into your mouth. They can be effects that result from trying to lift
things that are too heavy. They can be effects of NOT doing things, such
as not staying close enough to a water supply or a food supply. They can
be effects of fighting with other organisms, or of not displaying your
feathers at the right time of the year. The way our behavior is
organized to control perceptions can have profound side-effects on our
physical well-being.

Your alternative diagram includes no paths through the external world.
So it leaves out all the effects on intrinsic variables that occur
because of the way the outside world is affected by behavior and by
independent causes.

So, there seems to be no obvious reason why signals relating to
physiological states must be kept separate from signals derived from
sensors of the outer world.

But you're supposing that all effects on intrinsic state are produced
internally, and that none are produced by disturbances in the outside
world. On the contrary, most effects on intrinsic state are direct
physical and chemical effects on the body. That is why the organism has
to learn to _behave_ in order to correct intrinsic errors. It has to
learn how to regulate the external world and its relationships to the
external world in order to control intrinsic variables.

Your suggestions about learning ignore these effects through the
external world.
-----------------------------------------------------------------------
I'm going to go and eat my Christmas candy and watch a Shirley Temple
movie with Mary. So take that, all you intellectuals.
-----------------------------------------------------------------------
Best to all,

Bill P.

[Martin Taylor 941228 11:40]

Bill Powers (941225.1100 MST)

Martin Taylor (941221.1410)

Bill posits some differences that are not really there, between the
"separate reorganizing system" construct and the "intrinsic reorganizing
system" construct.

In fact, it has seemed to me more than probable that there is no
separate reorganization system.

Then you would be hard-pressed to explain _simply_ how a rat can learn
to press a bar in order to turn off a source of gamma radiation, which
has no effects related to pressing a bar.

No harder pressed, and no less hard pressed than with the separate
reorganizing system. In fact, exactly the same, and with the same
simplicity. The randomness of reorganization is not entirely lost
in the intrinsic structure; it is somewhat more focused, but not lost.

Now, this [reorganizing] control system is drawn a little differently
from the usual way of drawing a control hierarchy, because the
"physiological side effects" are signals like any other, though they
are probably chemical rather than electrical.

But they needn't be "signals." They can be actual physical effects like
burning yourself on a hot object or cutting yourself while trying to
close a clasp-knife.

Oh, come now. You would never let ME get away with a statement like that!

The "physical effects" are turned into changes in the concentrations of
all sorts of body chemicals, by all sorts of processes involving both
neural signals and chemical reactions to oxidation products and the like.
All these chemical changes of concentration of reaction products
are variables that are potential "signals" in control loops. But the
"physical effect" of burning or cutting can never be an intrinsic variable.
Intrinsic variables are inside, somewhere. Intrinsic variables result
from physical effects, perhaps, but then so do all sensory variables in
the standard hierarchy.

Your alternative diagram includes no paths through the external world.

Sure it does. Exactly the same paths that the standard diagram includes.
The intrinsic variables are affected in exactly the same way in both
diagrams. I did, perhaps, make the "mistake" of mentioning that there do
exist known and plausible ways for the chemical variations to affect
variables in the neural signal paths, but that's something both diagrams
have to address at some point.

So, there seems to be no obvious reason why signals relating to
physiological states must be kept separate from signals derived from
sensors of the outer world.

But you're supposing that all effects on intrinsic state are produced
internally, and that none are produced by disturbances in the outside
world.

I don't know where you got that impression. It certainly was never in my
mind, because I was carefully making sure that the issue of what constitutes
an intrinsic variable, and how the intrinsic variable was affected, remained
identical in the two models, except for the single point that the "error"
variable is no longer included with the "intrinsic" variables.

In fact, the basic point at which my thinking leads to the "intrinsic"
reorganization structure, is that there is only one so-called "intrinsic
variable" for which the effects are produced internally: the globalized
error variable. This is why it seems reasonable to consider the globalized
error variable differently from all the other intrinsic variables. THEY
are directly affected by interactions between the organism and the external
world; IT is affected only indirectly, by the success or failure of the
organism to control its perceptions of the external (and internal) world.

On the contrary, most effects on intrinsic state are direct
physical and chemical effects on the body. That is why the organism has
to learn to _behave_ in order to correct intrinsic errors. It has to
learn how to regulate the external world and its relationships to the
external world in order to control intrinsic variables.

Precisely so. That's at the foundation of the evolution of control
hierarchies. It's where everything relating to life starts. Even a
paramecium controls variables relating to movement in order to control
its CO2 concentration, and its movement control reference values seem
to be connected with temperature. All very distinct variables, physically,
but the net result is the survival of the paramecium.

Your suggestions about learning ignore these effects through the
external world.

Not so; at least no more so than the standard separate reorganizing
structure does.

···

=====================

There is a problem with the intrinsic reorganizing structure, a problem
we discussed long ago. The problem is: "How does a new control system,
controlling a new perceptual variable, get inserted BETWEEN a higher and
a lower pre-existing pair of levels in such a way as to be useful and not
to destroy the control that already existed?" The separate reorganizing
system does not have this problem, because any new ECUs are built on top
of the existing hierarchy, providing variation for reference signals that
previously were only implicitly set at zero.

But in both cases, new ECUs are built mainly when the existing structure is
NOT controlling well, so the problem may not be as severe as it looks at
first sight.

===================

There is a difference of philosophy between the two structures, but this
difference may well not manifest itself as a difference in performance.
In the separate structure, a failure of an intrinsic variable to remain
under control cases an action that makes a change in the main hierarchy.
But where does this change occur? The "classical" answer is that it
occurs in some random part of the hierarchy. But there's a problem with
this answer: most of the hierarchy is controlling its perceptions quite
well at any moment, and almost all changes will be for the worse. There
presumably exists some part of the hierarchy in which some change will be
effective, but to find it by random variation is rather like using a gun
to open a door by shooting blindly at the building to which the door might
provide access. You may get in, but there may not be much house left
by the time you do. Better to shoot at the lock, if you can find it.

How do you find the lock, using the separate reorganizing structure?
Is there anything that can suggest that one part of the hierarchy is
better than another for trying a random (but no longer randomly placed)
alteration? One possibility is that regions of the hierarchy that are
not controlling their perceptions well might be better bets than regions
that are. But ALL the controlled perceptions have been derived randomly,
simply because having them has (so far but no longer) resulted in good
control of the intrinsic variables. There is nothing to suggest that good
or bad control in a region of the hierarchy is more likely to be related
to the particular intrinsic variable that is going out of bounds.
Turning Bill's question back:

Then you would be hard-pressed to explain _simply_ how a rat can learn
to press a bar in order to turn off a source of gamma radiation, which
has no effects related to pressing a bar.

Random variation will indeed find it, but will the rat be dead before the
random variation succeeds? Will it have lost control of its perceptions of
where its legs are, or of any of the many other "mechanisms" that enable it
to act in support of its higher-level perceptual control systems?

In our previous discussion, Bill suggested that to avoid this problem,
reorganization would typically work more strongly at the top of the
hierarchy than at the lower levels. Control would become established
at lower levels before the upper levels were built and effectively linked.
This should work, since it reduces the target area for the gun from the
whole building to the region of the door. But the implication seems to
be that if I am persistently hungry, or hot, or getting my skin abraded
by sandstorms, my principles should change, and not the means whereby
I act to control my principles. This doesn't seem very reasonable.
(I emphasize "persistently" here, because "Many means to the same end" is
still valid--but in the situation I describe, none of those means is
effective).

Consider now the intrinsic reorganization structure. In this structure,
the intrinsic variable control systems provide reference signals for lower
control systems, rather than providing outputs that change the structure
of a separate hierarchy. The values of the intrinsic variables are affected
by exactly the same things as in the separate reorganization structure,
but what the outputs of their control systems do is different.

One variable is distinct that is "intrinsic" in the standard structure.
There is no single variable that is based on error in the perceptual control
hierarchy. What there is, relating to error, is some mechanism that is
a property of each individual ECU, whereby if it experiences sustained,
and especially increasing error, something changes in its environment.
Specifically, it may add, subtract, or change the sign of some of its
links to other ECUs, or it may modify its input or output function.
The nature of this mechanism is no more precisely specified than is the
nature of the mechanism that causes these changes in the separate
reorganization structure model. (I would not be at all surprised if
there were some chemical effect that could also diffuse to physically
neighbouring regions--but perhaps that is one nesting of speculation
too deep for this discussion.)

If an intrinsic variable is not well under control, the associated ECU
will change its linkage with the rest of the hierarchy until it does
come under control. If these changes cause other parts of the hierarchy
also to lose control (which can happen, the physical environment being highly
nonlinear), then they, locally, may change their linkages, until everything
settles down again.

A probable consequence of this structure is that the lower levels, which
support all manner of different perceptual controls at higher levels, will
quickly stabilize so that they do not have to change when high-level variables
reorganize. What is proposed as an ad-hoc principle when the reorganizing
system is seen as a separate structure fall out naturally from the behaviour
of the intrinsic reorganizing system.

Or maybe not. If and when the stochastic Little Baby is fully developed,
we will know by simulation whether this actually works as suggested.
Till then, all I can ask is that the proposal be addressed for what it is,
not with some fatal flaw inserted so that it can be discarded. If it has
a fatal flaw that can be noted without the simulation, I'd like to know it
and save time and money.

Hope you all had a pleasant holiday. A vain hope ("all"), I know, but
hopes are always worth having.

Martin

[From Bill Powers (941228.1225 MST)]

Martin Taylor (941228.1140)--

Me:

Your alternative diagram includes no paths through the external world.

You:

Sure it does. Exactly the same paths that the standard diagram
includes.

Well, I must have misread your diagram. Here it is:

                intrinsic reference
                        >
                        >
       ---intrinsic-----C---intrinsic error---
      > perception |
      > intrinsic
physiological output
  variables function
      > /
physiological /
   effects /
       \ /
        ---------- ------------------
                  > >
                  ^ V
                  > >
                perceptual
                  control
                hierarchy
                  > ^
                  > >
                  v |
              outside world

When I looked at this diagram, it seemed that the "outside world" was
shown at the bottom, with no connections from the outside world to the
physiological effects with which the reorganizing system is concerned.
It looks as if the perceptual control hierarchy has effects either on
the outside world or on physiological variables, but with no direct
connection between the outside world and the physiological variables.
Perhaps I am misreading this diagram.

If you will look in B:CP p. 188, you will see how I drew the diagram.
The "environment" includes all that is not the nervous system, so it
includes the body as well as the "outside world."

Your remarks about the specificity of reorganization agree with my
thoughts. There is a problem, and I don't have a good solution. One
proposal I've made is that reorganization follows awareness (although
this does not mean it is confined to the highest-level systems --
awareness is not so confined).

The basic problem is that the _reason_ for reorganization may be far
removed, in the body and logically, from the system that needs to be
reorganized. How can a hungry pigeon learn a control system for walking
in figure-eights? I don't think I'm making this point very clearly:

It has to learn how to regulate the external world and its
relationships to the external world in order to control intrinsic
variables.

Precisely so. That's at the foundation of the evolution of control
hierarchies.

But if you look at the diagram above, you'll see there's no way in which
a variable in the outside world, being controlled by the organism, can
have a direct effect on a physiological variable without going through
the control hierarchy. What are you saying "precisely so" about? The
path I'm talking about may be there in your imagination, but it isn't in
the diagram. At least I can't find it.

···

------------------------------------------------

There is a problem with the intrinsic reorganizing structure, a problem
we discussed long ago. The problem is: "How does a new control system,
controlling a new perceptual variable, get inserted BETWEEN a higher
and a lower pre-existing pair of levels in such a way as to be useful
and not to destroy the control that already existed?" The separate
reorganizing system does not have this problem, because any new ECUs
are built on top of the existing hierarchy, providing variation for
reference signals that previously were only implicitly set at zero.

I can see where inserting a new control system in the middle of the
hierarchy can result in a conflict, but not why it must. The new control
system doesn't get used until a higher one selects it (via gain, or by
setting a reference signal). Learning something new often requires more
reorganization, both at lower and higher levels.

I don't recall saying that my concept of reorganization limits the
process to the highest level. In the B:CP diagram, the arrows are shown
coming from the reorganizing output function to all levels in the
hierarchy. I did sort of propose that maybe the locus of reorganization
follows awareness, and that awareness seems to be drawn to problem areas
(at any level), but that's not a very precise proposal -- it's just a
guess. What I did say was that during development of the hierarchy, the
locus of the most rapid reorganization probably moves upward as the
levels develop.

It seems to me that you proposed that the highest level developed first,
with intermediate levels being inserted afterward. I can see a
translation problem in going from a system concept error directly to a
reference level for muscle tension. I didn't (and don't) see how that
could work, but perhaps you have something in mind.

In the separate structure, a failure of an intrinsic variable to remain
under control causes an action that makes a change in the main
hierarchy. But where does this change occur? The "classical" answer is
that it occurs in some random part of the hierarchy.

If you want to call that an "answer." I have recognized the problem from
the start, but don't know how to fix it. In fact, for any given
intrinsic error signal, there are probably many places in the hierarchy
where a change of organization would fix the problem, so we certainly
can't go to the other extreme, proposing that for each intrinsic error
there is only one place to change or add to the hierarchy that will
work. How many ways can you think of to get something to eat?

The biggest problem is that there's no way to predict what control skill
will have to be learned in a given environment in order to correct a
given intrinsic error. I might have to learn to get my algebra homework
done every day in order not to be sent to bed without supper. I might
have to learn to treat grownups in a respectful way to avoid the pain of
a spanking. Almost anything can be linked, through the environment, to
intrinsic error, and the links need not be logical or "natural." That's
why, in my opinion, we need the centralized reorganizing system shown in
B:CP. There doesn't seem to be any other way for the organism to learn
what it needs to learn in order to control intrinsic variables.

But there's a problem with this answer: most of the hierarchy is
controlling its perceptions quite well at any moment, and almost all
changes will be for the worse.

That's the hallmark of reorganization: there's no guarantee that the
next change won't make matters worse: in fact, it usually does. The
trick is to set up the reorganizing process so that the changes only
persist for a short time if they make matters worse. Remember that
"better" and "worse" are not defined in the hierarchy, in terms of how
well variables are being controlled. Those terms are defined in terms of
INTRINSIC error, not ordinary perceptual error. If you're carrying out a
skilled behavior that is consistently getting you hurt, reorganization
will stop only when that skill disappears. It's only when insufficient
skill is causing intrinsic error that reorganization will end up
improving the skill. A clown learns to get what the clown needs by
learning how to behave clumsily.

In all my modeling of reorganization, I've found that simply randomly
changing parameters doesn't often work well. What works a lot better is
to introduce a small delta which is periodically chosen randomly between
positive and negative limits, and is then added to the value of the
parameter on every iteration. In other words, what is varied at random
when a reorganization occurs is the direction of change of the
parameter, not the value of the parameter. The amount of change goes
with the amount of intrinsic error. This prevents very large changes
from occurring during any one iteration, and forces continuity of change
onto the parameters. This greatly reduces the effects of unfavorable
outcomes of reorganization: another reorganization occurring immediately
will keep changes from going too far in the wrong direction.

Something like this seems to be working in evolution. The small changes
predominate: you don't often find a kitten in a litter that has decided
to grow an extra leg where its head should be. If recombinations were
truly random, you'd see a large proportion of monsters. There has to be
some principle that limits the size of changes.

I don't know. Perhaps the continuity requirement is enough to allow
random reorganizations all over the hierarchy. But probably not -- as
you have pointed out, when too many changes are made at the same time,
the chances of ending up with a favorable direction in hyperspace become
impracticably small. We need a principle that will not require massive
amounts of change just to hit upon one that is actually relevant, but
without making the process so specific that arbitrary environmental
links from behavior to intrinsic error can't be discovered.

I agree with your assessment of the problem of specificity. I don't have
any clever answers that will take care of it.

If and when the stochastic Little Baby is fully developed, we will know
by simulation whether this actually works as suggested.

Yes, that will tell us a lot. Keep in mind my suggestion about randomly
varying the direction of change of parameters instead of letting them
jump around by large amounts. That's made the difference between success
and failure in most of my models.

But you still need to fix up your diagram.
-----------------------------------------------------------------------
Best,

Bill P.

[Martin Taylor 941228 16:20]

Bill Powers (941228.1225 MST)

Martin Taylor (941228.1140)

Simple misunderstandings lead to big quarrels. Let this one not do so.

"Mr Speaker, I smell a rat; I see it floating in the air. But mark my
words, Mr Speaker, I shall root it out, I shall nip it in the bud..."
(Sorry, I forget the rest--it's from some 18th or 19th century speech
in the British Parliament).

Bill,

In my "localized reorganization" diagram, I put in only what Rick put into
his diagram of the separate reorganization system. (I'm using "localized"
today, rather than "intrinsic" to avoid confusion with "intrinsic variables").
Rick didn't put in the multitudinous pathways by which physiological
variables are affected, and neither did I. In both cases, to include
those pathways would serve only to distract from the main point,
which is that intrinsic variables are controlled, one way or another,
and that the control comes about through the action of the main perceptual
control hierarchy.

You might, I suppose, have interpreted my removal of "side" from Rick's
"physiological side effects" to mean that the physiological variables were
influenced only by neural signals from within the perceptual hierarchy.
I had no such intention. I wanted merely to maximize the parallel
between the two ways of looking at reorganization.

···

===================

Your remarks about the specificity of reorganization agree with my
thoughts. There is a problem, and I don't have a good solution. One
proposal I've made is that reorganization follows awareness (although
this does not mean it is confined to the highest-level systems --
awareness is not so confined).

The basic problem is that the _reason_ for reorganization may be far
removed, in the body and logically, from the system that needs to be
reorganized. How can a hungry pigeon learn a control system for walking
in figure-eights? I don't think I'm making this point very clearly:

We do agree on both the nature of the problem and its difficulty, or at
least your words are ones I could very well use.

But when you talk about "awareness," I get antsy. It's a construct of which
we are all aware, from personal experience. But it has (so far as I know)
no obvious counterpart in the signals and structures of PCT. Do you think
that the pigeon must be aware that it must walk figure-eights in order to get
food?

To me, this approach sounds very like a reference to program-level control,
rather than to reorganization. Perhaps program-level control and some form
of learning are related, but that kind of specificity seems to be different
from the kind of specificity that results (eventually) from e-coli-type
reorganization.

I can see where inserting a new control system in the middle of the
hierarchy can result in a conflict, but not why it must.

That's a little off the point that you made when this issue came up before.
Your point then was more or less that the inserted control system would be
redundant, and that the result of having it there would be that the old
output links would become level-skipping links, which would do nothing
that would not better be done by the intermediate level. When I said:

The problem is: "How does a new control system,
controlling a new perceptual variable, get inserted BETWEEN a higher
and a lower pre-existing pair of levels in such a way as to be useful
and not to destroy the control that already existed?"

I included both that aspect and the possibility that a randomly connected
new ECU might induce positive feedback in one or more loops, its own
included. So there's no "must" about conflict resulting from the
insertion of a new ECU between existing layers. It might work just fine.
But it might not.

I don't recall saying that my concept of reorganization limits the
process to the highest level.

No, I don't recall you saying that either, and if I gave the impression
you did, I'm sorry. What I recall you saying was that reorganization
would preferentially occur at higher layers, which is quite a different
thing. Does this say the same as:

What I did say was that during development of the hierarchy, the
locus of the most rapid reorganization probably moves upward as the
levels develop.

======================

The biggest problem is that there's no way to predict what control skill
will have to be learned in a given environment in order to correct a
given intrinsic error.
...

But there's a problem with this answer: most of the hierarchy is
controlling its perceptions quite well at any moment, and almost all
changes will be for the worse.

That's the hallmark of reorganization: there's no guarantee that the
next change won't make matters worse: in fact, it usually does. The
trick is to set up the reorganizing process so that the changes only
persist for a short time if they make matters worse. Remember that
"better" and "worse" are not defined in the hierarchy, in terms of how
well variables are being controlled. Those terms are defined in terms of
INTRINSIC error, not ordinary perceptual error.

Yes. In both versions of reorganization, when an intrinsic variable is
far from its reference, the control system for which it is a perceptual
signal is producing output. In the "localized" version, that output
changes the value of reference signals for other ECUs in the hierarchy,
eventually causing action that affects the intrinsic variable, with luck
in the direction to reduce the error in the intrinsic variable. It's a
normal control system within the hierarchy, and there are many actions
and types of action that could have the effect of reducing the intrinsic
error. Some, momentarily, might make matters worse, just as the error
gets worse momentarily after Rick flips a switch to reverse the handle-
cursor relation in a tracking task. But the normal action of the hierarchy
solves those kinds of problem, without reorganization (in the "localized"
structure).

When the control hierarchy that should support the control of an intrinsic
variable does not work properly, THEN is when reorganization should occur.
In the localized version, it is possible that the only control system
failing to control is the one for which the intrinsic variable provides
the perceptual signal. Then the reorganization would affect only what
the output of that one control system links to. But it the failure of
the control of the intrinsic variable was associated with other control
failures, then they, too, would reorganize. And locally, those reorganization
events would be random.

=====================

Now we come to a very interesting issue. Reorganization is supposed to
affect the linkages among ECUs--who provides reference signals to whom,
and with what sign. (It also affects the perceptual and output functions,
and maybe the function that collates the many signals that combine to form
the reference signal for an ECU, but none of these relate to the "interesting
issue.")

In all my modeling of reorganization, I've found that simply randomly
changing parameters doesn't often work well. What works a lot better is
to introduce a small delta which is periodically chosen randomly between
positive and negative limits, and is then added to the value of the
parameter on every iteration.

When ECU "H" provides output that forms part of the reference signal for
ECU "L", what kind of delta could apply? When H delinks from L and connects
to Q instead, and it makes matters a bit better, where is the next link
when the same delta is added?

What you describe is the aspect of reorganization to which, in past
postings, I have given the name "modification." In orthodox neural network
studies, it goes under names such as "gradient search." With a smooth
enough landscape of performance as a function of parameter variation, it
works pretty well. But it doesn't describe what happens in a structure
in which the outputs of higher ECUs add together to form the reference
for any lower ECU, some with positive and some with negative sign. There
have to be parameters with continuous possibility of variation, for this
approach to work, parameters such as those that define the perceptual
and output functions, or the output gain.

In earlier discussions, you talked about massive transients that occur when
links are reconnected. But there is a way to avoid this (two ways, actually,
and possibly a Spanish Inquisition number of ways, but only one I want to
deal with now). That is to consider that ECUs within a level are not
orthogonal to one another. Like muscles and muscle fibres, several pull
in neighbouring directions not quite the same, but when they pull together
they make one big pull in some specific direction. Then, if one gets
relinked with an opposite sign, so it pushes while the others pull, what
happens is a shift either in the magnitude or the direction of the big pull
(or both). Conflict, perhaps, increases, but the assumption is that the
ECU that changed its output sign was not controlling well, and under most
conditions that would mean that the others with which it was formerly
collaborating are also not controlling well. They would thus be likely
to undergo a reorganization event sooner than the one that recently did.
In this way, the "big pull" could smoothly and perhaps not too slowly
change to a "big push," or to a "big pull" in some other direction.

when too many changes are made at the same time,
the chances of ending up with a favorable direction in hyperspace become
impracticably small. We need a principle that will not require massive
amounts of change just to hit upon one that is actually relevant, but
without making the process so specific that arbitrary environmental
links from behavior to intrinsic error can't be discovered.

...[Little baby]...

Keep in mind my suggestion about randomly
varying the direction of change of parameters instead of letting them
jump around by large amounts. That's made the difference between success
and failure in most of my models.

I've asked a local mathematician to compute the probability that a move
of size S results in getting closer to a target in N dimensions, when
the original position is at distance D from the target, as a function
of S, N, and D. He says he has an analytical result, but the main point
is that only if S << D is the probability of improvement near 0.5 when N
gets reasonably large. If a jump has any chance of landing near the
optimum, then it will very probably make things worse rather than better.
Only if the best change is a slight improvement is there a reasonable chance
of making some improvement.

===================
I don't know whether localized reorganization would work any better than
a separate reorganization structure, but I do think that separating out the
role of error from the role of survival-based intrinsic variables has
something to recommend it.

===================

But you still need to fix up your diagram.

Still think so? Then get Rick to fix up his, too. I'll add whatever
he does.

===================

One of my reference perceptions for these three days is the sight of a
considerable proportion of my desk-top. It's a perception for which I
control only on the days between Xmas and New Year, every year. It conflicts
with other perceptual references that are more important most of the time.

So... back to the clean-up and throw-out!

Martin

[From Bill Powers (941229.0800 MST)]

Martin Taylor (941228.1620) --

In both cases, to include those pathways would serve only to distract
from the main point, which is that intrinsic variables are controlled,
one way or another, and that the control comes about through the action
of the main perceptual control hierarchy.

...

You might, I suppose, have interpreted my removal of "side" from Rick's
"physiological side effects" to mean that the physiological variables
were influenced only by neural signals from within the perceptual
hierarchy. I had no such intention. I wanted merely to maximize the
parallel between the two ways of looking at reorganization.

It wasn't leaving out a word that I remarked upon, but leaving out a
physical connection going directly from "outside world" to
"physiological variables." Leaving out essential functional pathways in
a system diagram may "maximize the parallel" between that diagram and
another one, but you have to be careful that you're not simply changing
the definition of the system. After all, people who omit feedback
pathways to emphasize the parallel between PCT and cognitive theories,
or who leave out reference signals to emphasize the parallel between S-R
theories and PCT, are rather missing the main point, aren't they?

In the case of the reorganization system, the phenomenon that is the
most difficult to handle is the one in which the system must learn to
control some particular variable in the environment at a certain
reference level because there happens to be a link between the state of
that variable and some intrinsic variable inside the organism. This is
the basic situation in animal learning experiments, where the
experimenter decides arbitrarily that the animal must perform some act
in order to get food, water, exercise, a mate, or something else. It's
like finding yourself in a situation where you must learn to wiggle your
ears in order to keep the oxygen concentration that you breathe at a
level where you're not gasping for air. You have to learn something
without any apparent rational reason for learning it, except that
learning it solves the problem.

I'm not sure that you are trying to solve this problem. It seems to me
that your localized method assumes that there is some goal in the
hierarchy and that when difficulties in reaching it are encountered, the
control system in question varies the reference signal paths connecting
it to other ECUs until it finds the connections that will correct its
error. This is like varying the weightings given to a set of pre-
existing paths, with a weighting near zero being equivalent to no
connection. I agree that this is a possible mode of reorganization, and
that we should see via simulations what it can accomplish. But it
assumes the presence of an already-organized and functioning control
system that is already receiving reference signals from higher systems.
It is already perceiving the environment in a particular way. Perhaps
what your proposal is doing should be called "tuning," not reorganizing.

A great deal of our differences depends on how much pre-organization is
assumed to exist in the brain of the neonate. I am trying to assume very
little; you appear to be assuming a great deal. I'm trying to account
for the evident relationship between the variables that must be
maintained at certain levels for survival and the learning of control
processes that relate to the happenstance world outside the organism.
I'm trying to see how this could happen if the organism has no _a
priori_ knowledge of the nature of the external world, and at a time
when the organism possesses no thinking or reasoning processes, and in
fact no systematic control capacities at all, at any higher levels.

Ideally, what I would like to see is a basic control process related to
fundamental critical variables that can bring neuro-behavioral control
systems into being -- or S-R systems, or motor programming systems, or
any other kind of system. I would like to demonstrate that control
systems would arise for the simple reason that they are the only ones
that will work in a variable environment.

Obviously, my assumption about the degree of preorganization is as
pessimistic as yours is optimistic. I don't think that either of us has
gone very far toward a workable model of reorganization, although we can
make our simple models work in specific circumstances.

···

-----------------------------------
Technical matters:

When ECU "H" provides output that forms part of the reference signal
for ECU "L", what kind of delta could apply? When H delinks from L and
connects to Q instead, and it makes matters a bit better, where is the
next link when the same delta is added?

The delta would apply to the weighting assigned to each path, increasing
or decreasing it. If a path has an error-increasing effect, then
eventually the weighting would hover near zero. Paths are changed by
changing the weightings. I neglected to say that when there are multiple
parameters being changed, there is a delta for each one, and each delta
is changed randomly when a reorganization occurs. If you think of the
set of parameters in n-dimensional space, the deltas describe a velocity
vector in that space. Since I multiply all the deltas by the absolute
value of the intrinsic error, the velocity decreases as overall error
goes toward zero.

This is a direct analogue of the E. coli situation. When I ran into
problems with changing the values of parameters directly at random, I
went back to E. coli and realized that this would be the equivalent of
changing positions in the gradient in jumps. So I went to the deltas in
a deliberate attempt to mimic the swimming velocity in 3-D space, and it
worked.

In my experiments for solving n equations simultaneously using
reorganization, I found that this works for n up to at least 50. So it
is possible for random reorganization to work on a respectable number of
parameters at the same time. Even though the probability of a favorable
direction decreases with the number of parameters, even a very small
probability, like 0.001, is enough to provide continual progress. A
large number of "tumbles" might have to occur between favorable moves,
but the unfavorable ones never change the parameters by much; they don't
last long enough, and for any given axis they average to zero as long as
tumbling is going on at the maximum rate.

You can see a similar effect in E. coli by setting the reference level
for rate of change of attractant to a value nearly at the maximum
possible value. What you see is a prolonged period of random tumbles
generating a cloud of traces that goes nowhere, and then suddenly the
spot breaks out of the cloud going almost directly toward the center of
concentration. It continues to the closest point of approach, and then
another prolonged period of tumbling occurs until the spot again heads
almost directly toward the target.

Note that the parameters are _always_ changing. In the pure E. coli case
this would mean continual reorganization even when the spot has reached
the target position. In my adaptation, I make the velocity vector
proportional to the intrinsic error, so the speed decreases as the error
gets smaller.

What you describe is the aspect of reorganization to which, in past
postings, I have given the name "modification." In orthodox neural
network studies, it goes under names such as "gradient search." With a
smooth enough landscape of performance as a function of parameter
variation, it works pretty well. But it doesn't describe what happens
in a structure in which the outputs of higher ECUs add together to form
the reference for any lower ECU, some with positive and some with
negative sign.

Yes it does. In my experiments, I had n control systems in an environent
of n variables. Each control system received inputs from all n
environmental variables with randomly assigned weights, and each control
system's error was converted into an effect on all n of the
environmental variables, with different weights for each connection. So
every environmental variable was affected by all n of the control
systems, and contributed to the inputs of all n of them.

I was able to get this system to reorganize either its output weights or
its input weights (but not both) on the basis of the sum of all the
error signals in the n systems: just one great big composite error
signal. For n = 10, this involved randomly varying 100 deltas applied to
100 weightings, 10 for each system. As I said, I got this to work pretty
well for n as high as 50 (meaning 2500 deltas being randomly changed, 50
for each system). Convergence took several hours at n = 50 and was far
from perfect, but since I was just working by cut-and-try I probably
didn't get close to the optimal design.
-------------------------------------

[A local mathematician] says he has an analytical result, but the main
point is that only if S << D is the probability of improvement near 0.5
when N gets reasonably large.

Ask him how large S/D can be for a probability >= 0.001. We don't need
anywhere near a probability of 0.5 for reorganization to work. Remember
that wrong directions lead immediately to another reorganization, while
right directions allow progress all the way to the next point where the
error rate goes positive again. A succession of consecutive
reorganizations has essentially no effect on the parameter values.
--------------------------------

But you still need to fix up your diagram.

Still think so? Then get Rick to fix up his, too. I'll add whatever
he does.

Here's Rick's:

intrinsic intrinsic
reference -->C<------ perception---------
             > ^
       intrinsic error |
             v |
          perceptual |
           control physiological |
          hierarchy---side effects-->physiological
           > ^ variables
           > >
           v |
        outside world

He, too, needs a direct connection from "outside word" to "physiological
variables." I would prefer to see this:

intrinsic intrinsic
reference -->C<------ perception---------
             > ^
       intrinsic error |
             v |
          perceptual |
           control |
          hierarchy physiological
           > ^ variables
           > > ^
           v | side-effects |
           world ------------------------
                      of control

.. where "world" includes all that is not nervous system.
--------------------------------------------------------------------
Best,

Bill P.

[Martin Taylor 941229 14:30]

Bill Powers (941229.0800 MST)

Last things first:

Let's put in the outer-world connections in the "reorganization" diagram.
Rick added them to his, as Bill suggests, and I agree with what Bill says:

Leaving out essential functional pathways in
a system diagram may "maximize the parallel" between that diagram and
another one, but you have to be careful that you're not simply changing
the definition of the system.

So here's a revised diagram.

                intrinsic reference
                        >
                        >
       ---intrinsic-----C---intrinsic error---
      > perception |
      > intrinsic
physiological output
  variables function
      > /
physiological /
   effects /
     \ \ /
      \ ---------- ------------------
       \ | |
        \ ^ V
         \ | |
          \ perceptual
           > / control
           > /hierarchy
           > / | | \
    -------^-^----^----v--v----physical effects-------
           >/ | | \
            \\ | | /
              outside world

This is my earlier re-arrangement of Rick's diagram, with three things
added: a line to indicate the physical-effect boundary with the outer
world, two extra lines to indicate the multiplicity of inputs and outputs
associated with the perceptual control hierarchy, and a line showing that
the physical interactions with the outer world can influence the physiological
variables without as well as through the perceptual systems.

In prior discussions, it has been noted that there seems to be no reason
why perceptual signals should not skip levels, and indeed the inclusion
of uncontrolled perceptions as part of the input to higher-level perceptual
control systems has been invoked in several discussions. That's what the
twin inputs to the "physiological effects" is supposed to represent. Some
of the effects come directly from external events, some from the perceptions
in the nervous system (such as the adrenalin burst that seems to accompany
a perception of emergency), and to be complete, one should add the effects
due to the chemical control systems that don't involve the nervous system as
well. I'm not too happy with Bill's:

.. where "world" includes all that is not nervous system.

though I could be persuaded to go along with it for the time being, for the
sake of maintaining parallelism with Rick's modified diagram.

···

==================

In the case of the reorganization system, the phenomenon that is the
most difficult to handle is the one in which the system must learn to
control some particular variable in the environment at a certain
reference level because there happens to be a link between the state of
that variable and some intrinsic variable inside the organism.
...
I'm not sure that you are trying to solve this problem. It seems to me
that your localized method assumes that there is some goal in the
hierarchy and that when difficulties in reaching it are encountered, the
control system in question varies the reference signal paths connecting
it to other ECUs until it finds the connections that will correct its
error.

I thought I was trying to solve the problem you pose. Yes, there is a
"goal in the hierarchy." It is to keep the intrinsic variables at their
reference levels. Normally I don't like using the word "goal" in this
context, because it carries a lot of freight, but it's the same goal that
the separate reorganizing system has. Both schemes serve as control
mechanisms for the intrinsic variables.

when difficulties in reaching it are encountered, the
control system in question varies the reference signal paths connecting
it to other ECUs until it finds the connections that will correct its
error.

The primary thing that is varied is, in the separate reorganizing structure,
the various signal paths (and the related I/O functions for the different
ECUs). In the localized system, the primary thing that is varied is the
set of reference signal LEVELS. Paths do change, but as a consequence
of failure to control, locally.

But it
assumes the presence of an already-organized and functioning control
system that is already receiving reference signals from higher systems.
It is already perceiving the environment in a particular way. Perhaps
what your proposal is doing should be called "tuning," not reorganizing.

No. Any variation in the hierarchy that can be imposed by the separate
reorganizing system is assumed to be available to the localized system.
In the Little Baby experiment, we plan to test three kinds of variation,
together or separately: (1) variation of linkage weights (as you suggest
below, but not as I previously thought you accepted), (2) variation of
perceptual function (changing WHAT is perceived by any particular ECU),
and (3) generation of new ECUs using Genetic Algorithms to create the
new ECU with characteristics combined from two parents. We propose to
try each mode separately at first, and then in combination. You suggest
that combining input and output variation doesn't work well. It would
be interesting to find out why not.

All of this depends on there being contract money to support the continued
work on LB, which is very far from being assured at the moment.

A great deal of our differences depends on how much pre-organization is
assumed to exist in the brain of the neonate. I am trying to assume very
little; you appear to be assuming a great deal.

Without having worked out the detail, my assumption is that the initial
stage is that the ECUs whose perceptual signals correspond to the intrinsic
variables exist initially, together with some means of affecting the
outer world. That, and the reorganizing mechanism, is all. Other
control systems grow in between the world and these "intrinsic variable"
ECUs, and are varied until they attain some kind of control. With luck,
the new ECUs assist the "IV" ECUs to maintain control, but if not, further
new ones are grown, and the linkages to the "IV" ECUs varied.

As the embryo grows, its environment changes, and control systems that
earlier were sufficient are no longer enough to permit the IV ECUs to maintain
control--and even the originally grown new ones may not be able to control.
So further ECUs will be grown both between the IV ECUs and the first "new"
layer, and between the first "new" layer and the outer world. And so on.
In the localized system, when ANY ECU fails to control, it varies its
linkages and functions, and possibly induces the generation of new ECUs
below it. (And above it???)

That's a hand-waving description of how I see the generation of a control
hierarchy through localized reorganization. It is not fleshed out, but
with luck it will be, and I could then do less waving and more pointing
at what worked and what didn't.

=========================

When ECU "H" provides output that forms part of the reference signal
for ECU "L", what kind of delta could apply? When H delinks from L and
connects to Q instead, and it makes matters a bit better, where is the
next link when the same delta is added?

The delta would apply to the weighting assigned to each path, increasing
or decreasing it.

Oh, fine. I thought, as I was writing the questions, that if I introduced the
concept of a variable weighting between output and lower reference input,
you would accuse me of adding something new to the structure of PCT. In
at least one earlier discussion, you talked about the difficulties associated
with the transients involved in flipping the sign of a link, and I guess
I assumed that you were restricting the weights to 1, 0, -1. Shifting
the weight was what I had in mind when I said that there were at least
two ways to get around the problem. Thinking from the viewpoint of synaptic
connections, incrementing or decrementing weights makes more sense.

But that still doesn't apply to delinking from L and relinking to Q, unless
these are seen as bringing one weight down to zero (and stopping there)
and independently bringing another up from zero. That's a reasonable way
of looking at it, especially if we remember that synaptic connections
probably do not go from excitatory to inhibitory, though they may get
stronger or weaker over time.

In my experiments for solving n equations simultaneously using
reorganization, I found that this works for n up to at least 50. So it
is possible for random reorganization to work on a respectable number of
parameters at the same time. Even though the probability of a favorable
direction decreases with the number of parameters, even a very small
probability, like 0.001, is enough to provide continual progress.

Yes, but there's a bit of a discrepancy between this statement and the
results that Jeff Hunter found when he tried increasing the dimensionality.
Maybe you were using a more efficient algorithm, but Jeff found that the
time to a solution increased so rapidly that overnight computation was
needed for more than about 10 dimensions. I don't know what algorithm
Jeff used, and he is no longer here to ask. Did you track the solution
time as a function of the dimensionality?

Note that the parameters are _always_ changing. In the pure E. coli case
this would mean continual reorganization even when the spot has reached
the target position. In my adaptation, I make the velocity vector
proportional to the intrinsic error, so the speed decreases as the error
gets smaller.

At some point, I'd like to pursue this aspect. It is a direct analogue of
the process of "simulated annealing" used in some neural network studies,
and relates directly to the notion of the entropy of the control network.
I don't want to pursue it now, but I'd raise a flag for later discussion,
along the lines you mentioned (but somehow I can't find the quote) that
the problem is to make the system both stable, so that it maintains
control, and fluid, so that it can adapt to changing circumstances.
I think that such a discussion will involve the Bomb in the hierarchy
and the concept of self-organized criticality. But I don't have time
to pursue it here and now, and in any case my ideas would benefit much
from seeing the Little Baby come to term and learn something.

===================

But it doesn't describe what happens
in a structure in which the outputs of higher ECUs add together to form
the reference for any lower ECU, some with positive and some with
negative sign.

Yes it does. In my experiments, I had n control systems in an environent
of n variables. Each control system received inputs from all n
environmental variables with randomly assigned weights, and each control
system's error was converted into an effect on all n of the
environmental variables, with different weights for each connection. So
every environmental variable was affected by all n of the control
systems, and contributed to the inputs of all n of them.

You varied the weights, or at least so you said earlier. So my statement
is not contradicted by yours. Mine referred to the condition in which
only the sign flipped or the link was disconnected, which I thought was
your view of the structural reorganization, as opposed to changes in
the perceptual function. If I didn't make that explicit, it was because
I took the wrong assumption for granted.

[A local mathematician] says he has an analytical result, but the main
point is that only if S << D is the probability of improvement near 0.5
when N gets reasonably large.

Ask him how large S/D can be for a probability >= 0.001.

When he gets back after the holidays I hope to be able to report on
just what he found. The answer to your specific question will depend
on the dimensionality.

We don't need
anywhere near a probability of 0.5 for reorganization to work. Remember
that wrong directions lead immediately to another reorganization, while
right directions allow progress all the way to the next point where the
error rate goes positive again. A succession of consecutive
reorganizations has essentially no effect on the parameter values.

No problem with this. Except for that word "immediately." In any
control loop there is a finite loop delay, and any step change is followed
by a transient that lasts at least as long as the loop delay. So if
another reorganization followed "immediately" after one that would have
been in the right direction, the good one will be lost. There has to
be enough time between reorganization events for at least one loop delay,
and probably time enough for the main body of the transient to die away.
It doesn't matter whether the change is small or big, the effect of the
change has to be given a chance to occur. Too frequent reorganization
events, just like too big ones, could be damaging.

By the way, that's in good part the problem with our electoral systems.
Policy changes often show their economic and social effects long after
the term of office of whoever initiated them, and even if the changes
are for the good, the situation may remain bad until after the election,
at which time the "bad guys" reverse the good changes, while reaping
their benefits at the following election. Electoral reorganizations
happen too often (and have too few degrees of freedom--is a binary
choice consistent with "the Land of the Free?-) And they sometimes
involve too large changes (as with our previous Conservative government,
and your promised Republican revolution).

================

But you still need to fix up your diagram.

Done. Hope it is satisfactory.

================

This is much more fun than cleaning up the office!

Martin