Exploration and Reorganization

CSGnet Archive CSG1992
1

[Martin Taylor 920831 17:20]

"All behaviour is the control of perception." But is it? I've been pondering
a bit on Bill's comment a while ago that he found in his reorganization
simulations that the flip of a sign in an output-to-reference connection
caused a bad transient that troubled the whole net. It leads to the question
"What is exploration?"

Exploratory behaviour is "what if" behaviour. One is not controlling for any
particular reference (at a higher level one is controlling for a percept of
knowing more than one does, but that's outside this point). One is just acting,
more or less randomly at the level at which exploration is being done. Of
course it isn't random at lower levels, and probably not at higher ones.

Exploration could be considered action without a reference, or action with a
severed connection between comparator and output, with a noise source
substituted for the error signal. "What happens if I give a weak positive
output?...OK, now what happens if I give a strong negative output?...OK, now
what..." and so forth. The results could be used in for reorganization,
in this case the adjustment of the signs of the output gain function or the
signs of randomly selected output links. Flipping signs has no effect on
the behaviour of the hierarchy if the signs are flipped in a loop with zero
gain.

Exploration is for learning, and that is not usually coupled with a desire
for the results to have particular values (unless you are a politician); so
the actions involved in exploration are carried out in the absence of (strong)
negative feedback. This makes reorganization safe, at the cost of providing
unpredictable (but compensable) disturbances to parts of the hierarchy that
are controlling.

In the Layered Protocol formulation, before I heard of PCT, I argued that the
purpose of casual conversation was for the partners to develop models of each
other that would permit them to communicate effectively when the need arose.
In the same vein, I suggest that the purpose of casual exploration is to
develop organizations that permit effective control when the occasion arises.
Exploratory behaviour, then, is NOT the control of perception, but the
discovery of how to control perception.

Martin

2

[From Rick Marken (920831.1930)]

Martin Taylor (920831 17:20) says:

"All behavior is the control of perception." But is it?

Being the official sloganeer on the net, I would say:

No. "All PURPOSEFUL behavior is the control of perception".

This slogan captures the fact that there are many aspects of
behavior that are neither the actions that are used to control
nor the outcomes that result from control of perception.
My favorite: the position of my body over time as it plummets into
the pool from the high dive. Other aspects of that behavior
may be controlled, but I can't do much about it's downward
acceleration (unless I'm holding a hang glider).

As to your question

"What is exploration?"

I don't know. I think that it is different than reorganization
because it is systematic; so it is a controlled behavior (some-
times). That is, I think it is a program level perception that
is carried out for some higher order principle. Thus, I don't
think that exploration itself is random; reorganization is.
Exploration and reorganization can "look" the same; but I think
you could tell the difference using the test.

Best regards

Rick

···

**************************************************************

Richard S. Marken USMail: 10459 Holman Ave
The Aerospace Corporation Los Angeles, CA 90024
E-mail: marken@aero.org
(310) 336-6214 (day)
(310) 474-0313 (evening)

3

[Martin Taylor 920902 11:15]
(Rick Marken 920831.1930)

Martin Taylor (920831 17:20) says:

"All behavior is the control of perception." But is it?

Being the official sloganeer on the net, I would say:

No. "All PURPOSEFUL behavior is the control of perception".

This slogan captures the fact that there are many aspects of
behavior that are neither the actions that are used to control
nor the outcomes that result from control of perception.

It also comes perilously close to tautology, whether you are a PCT proponent
or not. To have purposeful behaviour is to have a purpose or goal, with
behaviour calculated to lead toward achievement of that goal. And how can
that achievement or progress toward it be assessed other than by perception
(in the broad sense in which you intend the term)? Even in PCT terms, the
word "control" cannot be limited to smooth or monotonic results of monotonically
directed actions--things break abruptly after smoothly increased forces are
appied, for example. The breaking may be the achievement of the goal, or
a step toward it, or an unwanted step away from it. So it is not clear to
me what that slogan achieves, except to make people think about the importance
of the feedback loop.

Exploration and reorganization can "look" the same; but I think
you could tell the difference using the test.

They aren't even in the same domain, so I don't see how they could "look" the
same. Exploration involves some observable actions, reorganization is
totally internal, even though it may strongly affect subsequently observed
actions. I think of exploration as a means toward safe reorganization
(performed at zero gain), not as a form of reorganization.

Martin

4

[From Rick Marken (920902.0930)]

Martin Taylor (920902 11:15), in response to my slogan:

"All PURPOSEFUL behavior is the control of perception".

says:

It also comes perilously close to tautology

Well, slogans are slogans.

see Powers (920901.1000) where he says:

Rick said it: all control behavior is control of perception.

etc

I said:

Exploration and reorganization can "look" the same; but I think
you could tell the difference using the test.

And Martin said:

They aren't even in the same domain, so I don't see how they could "look" the
same.

I was not clear. I think of "exploration" and "reorganization" as words
describing potential mechanisms underlying an observed behavior; I think
of them as descriptions of models. I interpret your comment above to mean
that you think of exploration as a description of behavior and reorganization
as a description of a model. In this case, there is indeed nothing to compare.
And I would argue that much of the behavior that would be described
as exploration is actually the result of a reorganization process; other
behavior that would also be called exploration is actually systematic,
control behavior. For example, when a rat is placed in a test chamber
it "explores" -- it sniffs around, climbs the walls, pushes on stuff, etc.
It produces a lot of changing perceptions for itself; I think many people
would describe this as exploration. Now, what is causing this behavior?
One explanation is that it is caused by systematic variation in the
rats' references for various perceptions, this variation being caused by a
higher order system trying to achieve its own perceptual goal (which
might be verbally described as "a perception of high familiarity").
This explanation says that the exploration we see is a result of regular,
systematic control processes -- and this could be tested using versions
of the test for the controlled variable.

Another possible PCT explanation of the SAME behavior would be that it
is the result of reorganization. This explanation would be particularly
plausible if we knew that the rat had not eaten for a day before being
placed in the test cage. If the "exploratory behavior" is the result
of reorganization then we would expect to see the variability of the
behavior gradually eliminated as control of some particular aspect of
the enviroment had effects that reduced intrinsic error.

So that was the point of my comment; the SAME behavior (which could be
verbally described as "exploratory") could have different internal causes.

This whole discussion may not be relevant to what you were proposing --
which seems like it may have been that "exploration" is an alternative
to the reorganization model of learning. If so, then I would have to
say "never mind". But, I would suggst that, along with your new model,
you could tell us what observations it is meant to explain (that are
not handled by the control hierarchy and reorganization model as it
already exists) and how we might test your explanation vs the explanation
provided by the existing model.

I have often carried on about the importance of the phenomenon of
control -- and knowing what that is before getting to involved in
the modelling. I would just like to add one more caveat; it is also
important to understand the difference between phenomena and the
models that are designed to explain them. I know that you understand
that difference Martin. But I think that we all must be careful (when
we are talking about control phenomena) to be as clear as possible
(in our expression and our understanding) about the difference between
what we experience (phenomena) and what we imagine (models that we
hope will explain the phenomenon).

Best regards

Rick

···

**************************************************************

Richard S. Marken USMail: 10459 Holman Ave
The Aerospace Corporation Los Angeles, CA 90024
E-mail: marken@aero.org
(310) 336-6214 (day)
(310) 474-0313 (evening)

5

[Martin Taylor 920902 14:30]
(Rick Marken 920902.0930)

I was not clear. I think of "exploration" and "reorganization" as words
describing potential mechanisms underlying an observed behavior; I think
of them as descriptions of models.

Well, clarity, like everything else, is in the eye of the beholder, and
in the case of writing, the writer beholding(?) sees differently from most
others. I thought I was clear, too. But the rat example clearly fits
one view of exploration, which was not what I had intended.

For example, when a rat is placed in a test chamber
it "explores" -- it sniffs around, climbs the walls, pushes on stuff, etc.
It produces a lot of changing perceptions for itself; I think many people
would describe this as exploration.

Yes, they would, but it wasn't really what I meant, unless neither of your
proposed explanations apply.

Now, what is causing this behavior?
One explanation is that it is caused by systematic variation in the
rats' references for various perceptions, this variation being caused by a
higher order system trying to achieve its own perceptual goal (which
might be verbally described as "a perception of high familiarity").
This explanation says that the exploration we see is a result of regular,
systematic control processes -- and this could be tested using versions
of the test for the controlled variable.

I hadn't thought of this possibility. It comes close to an idea I had very
many years ago, that organisms control to maintain a preferred level of
variability in perception at all levels of abstraction. If things are
too familiar to the rat, look for something new. If you are put in a
sensory isolation chamber, turn up the gain in your sensory systems so that
the noise can be interpreted as perceiving something. At a high level,
don't allow yourself to get bored.

Another possible PCT explanation of the SAME behavior would be that it
is the result of reorganization. This explanation would be particularly
plausible if we knew that the rat had not eaten for a day before being
placed in the test cage. If the "exploratory behavior" is the result
of reorganization then we would expect to see the variability of the
behavior gradually eliminated as control of some particular aspect of
the enviroment had effects that reduced intrinsic error.

This would also account for the actions observed by a third party.

My account ties in with the alerting function discussion. When there are
"spare" degrees of freedom that could be used for control, and are not
because all the references are reasonably well satisfied, then some are
available for actions that find out "what is there." This is not to satisfy
any reference for reducing hunger (or "thirst for knowledge"), but to
discover the perceptual results of some action. That discovery permits the
assignment of the correct sign to the output that was/led to the action,
for a later occasion when the ECS in question has a reference that it must
satisfy with its percept. Reorganization is a consequence of exploration,
in this view, and it happens at zero gain.

I hope this is clear, but even if it is, the clarity may well be an illusion.

Martin