[Martin Taylor 960105 10:10]
Shannon Williams (960104) and Shannon Williams (960104)
I'm having a very hard time trying to understand what your vision of PCT is.
It is certainly wildly different from mine. Here's a brief take on my
understanding of PCT, followed by a repost of an old message of mine on
the different kinds of reorganization. Reorganization certainly isn't
an ignored issue, but it's hard to experiment with because it requires
an external environment that needs a control _hierarchy_ rather than an
elementary control _unit_ to achieve decent stability. Almost all the
studies of perceptual control that I know of deal with a single control
loop or two loops at the same level of perception. Our "Little Baby"
project was supposed to be an investigation into an area requiring multiple
levels of control--syntax recognition--but the money ran out and the project
is in abeyance.
Here's an abstract of my idea of PCT.
There is a world "out there" that is big enough that one can never know
what events might occur to influence things that affect one. Simple
thermodynamic considerations argue that a random event is more likely to be
damaging than helpful, which means that if one is to maintain one's internal
structure, one must act so as to counter or to avoid the effects of external
events. To do either, one must sense something that is at least correlated
with the important external event, compare the result of that "sensing"
with the state that is not damaging to oneself, and act so as to ensure
the maintenance of the non-damaging state. The result of "sensing" is what
is called "perception" in PCT.
The above applies to ALL living things, animal or vegetable, single-celled
or complex. Also, the above describes a control system, which is to say
that ALL living things must be built around control systems. A living thing
has an inside and an outside, defined by the "inside" containing the
structure that is stabilized by control, and the "outside" containing
those influences that might destabilize the inside structure. To "sense"
is to represent in some way some aspect of the outside, inside. (To "act"
is to turn that inside out 
Nothing in the general description says how complicated the "events" might
be that could destabilize the internal structure. They might be smooth
and continuous, such as a change in temperature, they might be sudden
and discrete, such as the impact of a falling rock, or they might be
sturcturally complex such as changes in the spatial arrangement of a
pack or potential predators. But if the organism is to survive--i.e. retain
its structure--those events that would destroy the structure have to be
countered. We are the descendants of organisms that survived long enough
to propagate their genes, all the way from the beginning of life. We are
control systems that have developed from control systems...
The core of PCT is this, that there are certain states within any living
organism that have to be maintained within a reasonable range of values if
the organism is to survive. There are classes of externally influenced
effects that can be sensed, and that would be damaging if they were not
countered. Evolution has weeded out those organisms that often failed
to counter important detrimental influences, and so we, along with all
other living organims, have a set of "intrinsic reference values," for
the "perception" of environmental conditions, that have proved to allow
our ancestors to survive if the perceptions are maintained at their
reference values. Our control systems, whether hierarchic or not, have
reference value settings that are not determined by environmental variation,
but have been found effective by the simple fact that organisms that
failed to maintain appropriate values for them don't now have descendants.
Perceptual Control Theory usually goes outside the core in two ways. Firstly,
although it is not necessary in principle, the Powers theory assumes that
all values in the organism are scalar. There are no vector values for
perceptual or reference variables. Secondly, it is asserted that simple
control systems are arranged in a hierarchy, in which the reference values
for lower-level systems are derived from the outputs of higher ones, and
the perceptual signals of higher ones are derived from the perceptual
signals of lower ones. Such a hierarchic arrangement is not absolutely
required by the scalar assumption, but it provides an enormous simplification
of the structure if the structure is to control in an environment of any
complexity--especially one that contains bifurcations. And the scalar
assumption is justified by the assumption that it is signals on neuronal
axons that correspond to the perceptual and reference variables required
by the core theory.
Hierarchic scalar PCT, or HPCT, is usually what is meant by "PCT" in postings
on this list.
HPCT is based on a very simple "black box" control system, that in its
plainest form has two inputs and two outputs. One input and one output
connect to lower levels (or to the external environment as sensing and
acting), and an input and an output connect to higher levels. The
output to a higher level is the sensory input for that higher level, and
the input from the higher level is the reference signal for the black box.
>to higher |from higher
> level | level
> >
----^---------------V-----
> out in |
> BLACK BOX |
> CONTROL SYSTEM |
> in out |
----^---------------V-----
> >
> from lower | to lower
> level | level
> >
O------<--------
>
>unpredictable environmental
>influences (disturbances)
>
There is a real and important difference between the connections to the
higher level and those to the lower level. The intrinsic reference signals
are not subject to environmental disturbances, whereas it is the unpredictable
nature of the perceptual signals affected by environmental events that is
the reason control is needed at all. The predictable aspects of external
events can be countered by planned and routine actions, without observation.
There is no theoretical need for these predictable aspects of the environment
to be represented in perceptual signals. It is to counter the unpredictable
influences of external events that we require perceptual signals that can
be compared with reference values.
The values of reference signals, like those of perceptual signals, cannot be
predicted from within the "black box" elementary control unit (ECU), but
neither can they be affected (at least not consistently) by the outputs of
the black box ECU. In contrast, the outputs of the ECU are tuned to the
external environment in such a way as to affect consistently the perceptual
signals that come into the ECU. From the viewpoint of the ECU, there is a
link between down-going output and sensory input, whereas the link between
upgoing output and incoming reference is inconsistent if it exists at all.
That's what distinguishes a reference signal from a perceptual signal, and
why you are wrong to say
Let us say that signals are sent to our muscle system:
1) Aren't these signals generated by the output of some control loops?
2) Don't these signals consist of input into the muscle's system?
3) Aren't these signals to the muscle system, the muscles system's
perception of its world?
No. they are not. They are the muscle system's reference signals, which
the muscle system cannot affect consistently by its tensing and relaxing.
4) Doesn't this perception directly generate output?
No it does not, at least not in the sense that a particular reference value
results in a particular output. The generated output depends also on the
external forces that play upon the muscle.
5) Doesn't every single control loop in the brain have to contribute to
this perception in order to have an influence on the muscle system?
Might do, but probably not. Depends on the actual structure of the hierarchy.
6) Doesn't every newly generated loop need to contribute to this
perception?
"Every", "need to"? No, not necessarily. Any newly generated loop might well
do so, however.
7) What part of the new loop would contribute to this 'overall'
perception- p, r, or e?
"e" is inside the black box ECU. It is neither a perception nor a contributor
to a perception, so far as that ECU is concerned (although any other ECU might
take the "e" signal as part of its sensory input, since sensory input can come
from anywhere; it isn't restricted to the perceptual signals of lower levels).
"p" is subject to external influences. The black box doesn't "know" the
difference between that and the unpredictable changes that occur in "r",
but it can affect "p" by its output and it can't affect "r", which
distinguishes "p" from "r".
What, by the way, do you mean by an "'overall' perception"?
What this means is that the current PCT model states that a loop generates
output when a reference level _is not_ met. What I am saying is that a
loop generates output when a reference level _is_ met.
Such a loop is possible, but it would not be a control loop in the ordinary
sense. A loop like that could be used to drive a perceived state _away from_
a reference value that indicated a local dangerous state. The characteristic
quote would be "I don't care where I am, but I don't want to be here." It's
a positive feedback loop of a particular kind.
Maybe you could give a similar description of how you view PCT?
Meanwhile, here's the old posting on the kinds of reorganization.
Martin
···
----------------------
[Martin Taylor 921005 11:30]
(Bill Powers 921005.0730)
This is in response to the discussion between Bill and Greg.
To start with, both have produced a set of principles, but neither gave a
functional description of an ECS and its place in the hierarchy. To have
a clear discussion of learning, we have to have agreement on what the
hierarchy looks like, at minimum. So here is what I have understood to
be more or less agreed. If any of this is wrong, it affects the discussion
and should be put right.
As I understand the hierarchy, the unit of which all is built is the ECS.
The ECS has two kinds of input function:
(i1) a perceptual input function, which combines all the many sensory
inputs according to some algorithm and produces a scalar value called the
perceptual signal. The word "sensory" includes the possibilities of direct
input from sensor systems and the perceptual signals produced by the perceptual
input functions of other (lower) ECSs.
(i2) a reference input function, which combines action output signals from
other (higher) ECSs. Typically, this function is considered to be a simple
summation, but it could be any algorithm. The output range of the reference
input function must not exceed that of the perceptual input function. The
output of the reference input function is a scalar value called the reference
signal of the ECS.
The perceptual signal is compared with the reference signal to produce a
scalar value called the error signal. The error signal is transformed by
an output function, typically but not necessarily an integrating amplifier,
to produce an output signal.
The ECS has two kinds of output that are distributed to other ECSs (except that
in the case of the lowest level ECSs the output goes directly to muscles of
other effector systems. In each case there is the possibility that the output
is weighted differently in transmission to each individual destination:
(o1) A set of perceptual outputs that consist of the perceptual signal possibly
mulitplied by some weight. These are the sensory inputs of other (higher) ECSs.
(o2) A set of action outputs that consist of the output signal possibly
multiplied by some weight. These are the reference inputs of other (lower)
ECSs.
The ECS has the possibility of storing one or more values of its perceptual
signal and of using a stored value in place of its reference signal (episodic
memory).
The ECS has the possibility of using an action output signal as one of its
sensory input signals (imagination).
The hierarchy consists of ECSs linked only by the connection of the (possibly
weighted) perceptual signals of lower ECSs to sensory inputs of higher ECSs,
and of the (possibly weighted) action output signals of higher ECSs to the
reference inputs of lower ECSs. Each kind of link is one-to-many.
If the foregoing is a correct description of an ECS and its place in the
hierarchy, what opportunities are there for learning? The following all
seem plausible:
Within a set of ECSs already in existence:
(1) Alteration of the perceptual function.
(2) Alteration of the perceptual-sensory link structure.
(3) Alteration of the output function.
(4) Alteration of the action-reference link structure.
(5) Alteration of the reference input function.
(6) Alteration of the content of the internal memory of the ECS.
And
(7) Incorporation of a new ECS into the hierarchy.
There are subclasses:
(1a) (3a) and (5a) Modification of the parameter values of the function
(1b) (3b) and (5b) Alteration of the form of the function.
(2a) and (4a) Alteration of the connection weights
(2b) and (4b) Alteration of which ECSs are linked.
All in all, this makes 12 logical possibilities for types of ways the hierarchy
can learn. Not all are effective, and the discussion has generally focussed
on only a couple.
Hebbian learning is a term usually denoting topologically smooth changes
of parameter values in a combining function. There are many forms, but
generally speaking they all involve changes based on some measure of goodness
of the present set of parameter values in relation to the data input to the
function. One way is to make more extreme a pattern that creates a large
output, and to make less extreme a pattern that produces a small output.
Another way is to move the input parameters in such a way as to increase
the output when some "teacher" asserts that the input pattern is one to
which the function "should" give a large response.
Within the ECS hierarchy, Hebbian learning has usually been discussed as it
applies to the perceptual input functions (1a), and then usually when the input
functions are seen as nonlinear compressions of weighted sums of their
inputs. If the hierarchy consisted only of ECSs having this form of perceptual
input function, the perceptual side would be a classical multilayer perceptron,
and Hebbian learning would suffice to allow it to perceive (and hence possibly
to control) any describable partitioning of the sensory input data space.
Other forms of perceptual input function, possibly involving delayed sensory
inputs, are possible and may be necessary for the control of dynamical
percepts such as sequences.
Hebbian learning could apply in classes 1a, 3a, or 5a. Because the success
of control does not depend much on loop gain if the loop gain is high enough,
one would not expect it to be very important in 3a, except in adjusting
parameters such as integration time constants. It might be important in 5a,
which affects the relative strength of different higher-level ECSs on the
reference signal of the ECS in question. No discussion (that I remember) has
considered this possibility, or how it might work if it happened at all.
Hebbian learning could apply in 2a and 4a as well, but numerically the
results would be indistinguishable from 1a and 5a applied in a different ECS.
The reason these possibilities are listed is that there is a potential
question about the location of responsibility for the alterations of weights.
"What does the ECS know, and when does it know it?"
Hebbian learning cannot be relevant to structural alterations in the hierarchy
(1b-5b), because the learning is not topologically smooth. This is the
province of "reorganization." It also cannot apply to classes 6 or 7, which
involve discrete events that change either the content or the structure of
the hierarchy.
(Powers)
Hebbian learning makes parameters depend on long-term effects of the
signals passing through the network that is being reorganized; that's
the Hebbian version of reorganization. For this to work in general, it
must be true that there is some "best" way of handling signals. This
assumption shows up in the postulate that the output of a neuron
somehow "strengthens" the effects of input signals that exist at the
same time as the output signal. The implication is that it is best for
the organism that all signals contributing to a given neural output
have the maximum possible effect. This sort of rule, I believe, is an
attempt (of which I approve) to get away from a "teacher" that already
knows how a neural function should be organized.
This is indeed one form of Hebbian learning, but the motivation is not to
get away from a teacher, so much as to provide the maximum discrimination among
input signals that is consistent with the variation in stimulus patterns.
Usually, weights on low-valued inputs decrease when weights on high-valued
ones increase, so as to sharpen the discrimination.
Any system in which there is a preferred kind of
input or output function loses the ability to adapt to environments in
which some other kind of input or output function is required for
successful control.
True. And I think that your "levels" of ECS accommodates that. But Hebbian
learning can operate within different kinds of input function, provided that
the function is such that small changes in parameter values cause small
changes in the function's behaviour.
I had intended to carry on this posting with a discussion of reorganization
possibilities, but I think it is long enough, and I have spent too long on
it, already. Maybe later. But if you accept my categorization of learning
possibilities, there seem to be 5 kinds of reorganization, of which perhaps
1 or 2 may be useful in learning. And then there are episodic memory learning
and the introduction of new ECSs. So there are many possibilities for ways
in which the hierarchy might change to develop new skills.
When there are so many possibilities, it makes good sense to see what happens
when only the most prominent are used. In this case, I think that the most
likely candidates for effective learning are (in no particular order),
1a, 2+4b (together), 6, and 7. These can be verbalized as "What can I perceive"
"What can I do to control what I perceive" "What have I perceived" and "I
can do nothing right--let's try something completely different."
Martin
