[From Bill Powers (960729.0730 MDT)]
Giving feedback:
Martin Taylor is right: if an unintended effect of an output feeds back
to affect the perception being controlled, that is feedback, too.
However, a given elementary control unit doesn't know how its output
affects its input anyway, so as far as its operation is concerned
there's no difference: it's all feedback. This is different from
unintended side-effects, which by definition do not ever affect the
controlled perception to a detectable degree.
···
---------------------------------------------
Hans Blom, 960725 --
[replying to Martin Taylor]
What my comment was most about, I guess, is that vector/matrix
analysis is a language built upon another language. A language that
makes it possible to talk about certain concepts that cannot
(easily) be talked about in the underlying language. By language I
mean a tool to create order in an otherwise rather chaotic whole.
Some tools are just handier than others for certain purposes.
Vector-matrix analysis is indeed a convenient language. However, when
you have to BUILD the system described by the vector-matrix language,
out of elementary components, you have to use the expanded form.
---------------------------------------------
Hans Blom, 960725b --
RE: optimality criteria
Critical is: ONE global measure. Thus, "modern" control theory
(MCT) is top-down: by explicitly describing the top-level goal, all
lower level goals are (implicitly) specified as well. In contrast,
HPCT looks more like a bottom-up approach. Even if it is not, your
major concern is with the lower levels of the hierarchy. MCT says
that you cannot design the lower levels if you have not specified
what they are to do for the higher levels.
I don't see why there has to be ONE global measure of optimality. Of
course you can always combine a set of measures into a single measure,
but that's conceptual, not a matter of how the system really works.
Also, the system itself might have a reference level for, e.g., energy
efficiency, but the engineer building it might also have a reference
level for cost of manufacture. The first reference level is internal to
the system; the second is external to it. Why combine such measures? And
even if there are several distinct criteria to be met (body temperature,
blood pH, etc.), why then propose that there is a single optimality
criterion composed of these individual measures?
Also: this global measure is something to be MINIMIZED (or
maximized, but that is the same thing with a different sign), not
something to be brought to a specific value.
Whether you use the values of variables or the difference between those
values and some set of reference values is unimportant. However, if you
speak only of minimizing, you might miss some optimality criteria:
optimizing body temperature is not a matter of making it as small as
possible, but of making it as near as possible to some fixed value. When
you "minimize" a variable you're just trying to get it as close as
possible to a reference value of zero. For most physical variables
involved in the functioning of organisms, the optimal value is not zero.
Furthermore, the plant is not conceived of as a modular system in
which the overall control problem can be broken down into smaller
problems, but as a whole which is optimized in one huge calculation.
Either the entire plant is controlled, or control fails.
Right. Multiple, maybe many, simultaneous sub-goals, all
interlinked, all serving the one supreme goal.
I don't think you read that paragraph correctly, or you wouldn't have
said "right." My impression has been that the MCT approach treats the
entire system as a single computation, not as a set of goals and sub-
goals.
These criteria are thought of as the "goals" of the control process.
They are specified not by the control system, but by someone or
something operating from outside the control system.
Right. Just like in organisms, something has prespecified which is
more important in the "cost function": to bring offspring into the
world, or to care for the offspring once it is there. Different
organisms have different prespecifications.
We still seem to disagree on the difference between goals and outcomes.
An outcome of the operation of an organism may be that it reproduces,
but that is only an outcome. The actual goals that lead to this outcome
must be much more specific and relate to things that the organism can
sense and affect.
There is (and can be) only one topmost optimality criterium. If
there were more, one would again have to give them weights in order
to pre- specify their importance.
You're still speaking like God deciding how to construct an organism. It
may be that among the top set of optimality criteria there are,
effectively, weights -- but there is no superordinate system assigning
those weights. The effective weights are simply the relative loop gains
involved.
RE: State vector
The concept "state" is the central notion in MCT. Briefly, the
"state" of a system is a memory vector which "remembers" all that
is important of the past and useful for (control in) the future.
Thus it provides for data compaction or redundancy elimination. All
information about the past that might be important for present and
future control but is not captured by the state is called the
"model error".
This is strictly the design engineer's view. What keeps slipping through
the cracks here is the fact that in an organism there is no design
engineer to know all these things. In modeling organisms, we have to do
away with the design engineer and try to see what the organism itself
can know and do without any external intelligence to guide it.
The error signals that result (absolute or
squared) then are the basis for calculating changes in u(t). The only
variable that is controlled in a negative feedback way is the
optimality measure.
The controller constantly tries to minimize it.
That's what I said: the controller tries to control the optimality
measure -- whether with respect to a reference value of zero or some
other value is only a matter of how you set up the equations.
For a person like me,
who wants to connect every variable and operation in a mathematical
representation to the physical system being represented, this is
intolerable: I get no sense of understanding how the system works.
What about if the system has 100,000 sensors and 10,000 actuators?
A matrix/vector description remains almost as simple as the scalar
description of a system with 1 input and 1 output. I would not be
able to understand such a complex system if NOT in vector/matrix
terms...
Yes, that's a problem, but there are ways of approaching it without
hiding the details in the vector/matrix notation. HPCT is such an
approach. At the first level of neuromotor control, we have 600 to 800
control systems concerned with controlling tendon tension and muscle
length. These systems are defined as functional units involving sets of
muscles and many parallel control loops. At the second level, we have
sensory signals representing weighted combinations of first-level
perceptual signals. In principle, a given second-level system receives
inputs from all first-level sensory signals (most of which are not under
control at the first level) and produces outputs that affect the
reference levels of all first-level control systems. In practice, a
given second-level system will be connected only to a subset of the
first-level systems, so the connection matrix is sparse.
It's nice to have the matrix methods available to handle this multiple-
input, multiple-output situation, but I like to know what's going on
before hiding the details by saying
p2 = Fi2(p1)
r1 = Fo(r2 - p2)
where all the variables are huge vectors and the functions are gigantic
matrices. Have you seen my Byte articles where I worked this out for a
simple three-variable two-level system -- using matrix notation?
The interesting thing about this bird's-eye view of modern control
theory is that it is not actually incompatible with PCT or HPCT.
If it were incompatible, then HPCT would be wrong and I wouldn't be
here ;-).
No comment.
In the HPCT model, the "plant" that is the external world is not
controlled in one huge chunk, but through layers of processes that
work from the simple to the complex.
MCT shows that in most cases the processes in these layers must be
interconnected in order to achieve control of a reasonable quality.
Except in special cases, where certain orthoganality requirements
are fulfilled. HPCT frequently neglects this aspect and talks about
SISO (single input single output systems) as if these can explain
everything.
You must be under some serious misapprehensions about HPCT if you think
it deals exclusively with SISO systems. The only reason we use SISO
models is that this is (a) the easiest way to communicate the principles
of negative feedback control, and (b) a sufficient way to reproduce the
behavior of a controlling person in a simple task. We are quite aware
that in the tracking model, the output fans out to many lower-order
systems that actually control joint angles, and that the perceptions
that are under control arise from multiple layers of perceptual
interpretation, each involving many variables. What is interesting is
that a SISO model can be defined to reproduce behavior quite accurately
even though the operations of all the lower systems are simply assumed
to take place perfectly.
The question is whether we have to wait until we can model every system
at every level in full detail before trying to apply the model to
specific cases. The answer seems to be that we don't: we can make sense
of specific behaviors at any level using a very simple SISO model. Aside
from the convenience of this fact, it also tells us something about the
organization of the brain: it is not all one big chunk, but is modular
in its organization, with relatively independent functions being carried
out in a way that is detectable from outside. In the present state of
PCT, we can't put all these modular systems together into a single
experimentally-defensible model, so our best approach is just to keep on
finding units of organization that can be identified and that are stable
over time. Eventually, I hope, we will begin to see how such units fit
together at different levels of organization. Maybe then we will be in a
position to write the vector-matrix expressions for the whole thing. But
that's a long way in the future, and we'll need the help of people
smarter than I am.
Thus, for instance, the discussions about "disturbances" as only
deteriorating the quality of control and disregarding their effect
on other simultaneous control loops and, for instance, in learning
a better model. I don't wish a personal crisis to anyone, but
people who have experienced one often emerge a lot wiser...
We have considered all these things; they don't constitute personal
crises for anyone who understands PCT and HPCT. It's just that we've put
off certain problems as not being within our capacity to test
experimentally. We work with what we can handle.
The most proximal variables, those easiest to control, are put under
local feedback control at the first level, the level where muscle
forces and lengths are generated. These loops are independently
stabilized, regardless of events that are more remote from the
organism.
That is a very good idea in some cases ...
Glad to hear that you approve. The case I am interested in is that of
the higher living organism, where this always appears to be the
principle of organization. I am not particularly interested in other
cases, such as stabilizing helicopters or controlling the effluent in a
oil fractionating plant.
So the world that is experienced by the higher systems
becomes both simpler and less subject to perturbation than it would
be without the first layer of feedback control.
That may not be the reason. The state vector approach offers a
similar simplification. But that simplification is global, not
layer by layer.
I'm not interested in the "reason" but in the effect. In the human
system, the effect of lower-order control loops is to defend higher
loops against disturbances and to simplify the dynamics of the higher
loops.
And so it goes, layer by layer,
each layer perceiving and controlling a world derived from the
simplified world of the level below it, further simplifying the world
presented to any higher systems and shielding the higher systems from
more kinds of disturbances.
Until at the top everything is simplified to one variable? Where do
you stop simplifying? How to _know_ when to stop? What to simplify
and what not?
No, until at the top we have the set of controlled variables that
actually exists in the real system. The model could be applied to a
system with any number of levels; we stop when we have explained all the
levels we can find in a given system. We don't have to design this
system; it's already there, working. Our problem is to model it, not to
design it.
Although I understand little of matrix mathematics, I have seen
suggestions that large problems in matrix manipulation can be greatly
simplified by suitable partitioning of the matrices, especially if
the matrices have favorable properties. In effect, the overall matrix
is broken down into a set of submatrices, each of which is much
easier to handle and demands much less computing power.
The computing power is not the problem, because everything can be
done in parallel.
I admire, but don't share, your child-like faith that no matter how
difficult the computation, the brain has plenty of neurons to do it.
When _you_ do these computations, you apply mathematical procedures that
you have learned over the years, and you have to write things down and
painfully manipulate each expression in full detail. In doing this you
are using learned skills for symbol manipulation according to rules (one
of the processes our model must eventually explain). But the brain
starts controlling without any of that learning, and it does so with
sloppy neurons that operate somewhere between the analog and digital
worlds. The only way the brain can do vector-matrix operations is with
pencil and paper (or by building a computer to do the same operations,
but faster). The only way I can see for the brain to do everything we
see it doing is for it NOT to do them in the ways we describe
mathematically, but to do them in some much simpler way. Parallel
operation is not a magical cure-all that makes anything possible.
-----------------------------------------
Some general comments:
In the discussions between Hans Blom and Rick Marken that took place
while I was gone, a clear theme has developed: MY theory says that this
is how it works (whether the proponent is referring to PCT or MCT). The
problem is that theories don't tell us how things work; how things work
is supposed to tell us what theories to use.
MCT developed out of a certain conception of how a control process ought
to work. If you can just find the inverse of an environmental plant
function, and apply a set of driving signals to the inverse process,
then its output, entering the environmental plant, will necessarily
cause the plant outputs to follow the driving signals. The basic
problems involve how to discover the form of the plant function, and how
to adjust the parameters of the inverse of that function to achieve
optimal control. This concept is very simple mathematically; what gets
complicated is implementing it. The fact that you can write a symbol for
the inverse of a function doesn't mean that you can calculate it.
The whole scheme of MCT has been aimed at making systems of this kind
work. I have no doubt that given enough effort and ingenuity, this can
be done. Along the way, the properties of systems of this kind will be
developed, and a whole way of thinking about control will evolve. No
matter what question you raise, somewhere in the mathematical system
there will be relationships that can be claimed to answer the question.
Much the same thing can be said about PCT. There is a certain basic
concept of negative feedback control, and a hierarchical model built
from basic control units. Given this way of thinking and a mathematical
way of talking about it, one can answer just about any question that is
raised.
What happens, then, when PCT encounters MCT? One person says "THIS is
what is happening," and the other person says, "No, you don't
understand; THIS is what is happening." Each person judges the other's
position in terms of the first person's own theory. But of course the
theories don't agree; they are based on different conceptions of how
control is achieved. Actually, if the same amount of development effort
and mathematical sophistication were devoted to PCT as has been involved
in MCT, the contest would be much more even: the MCTer wouldn't be able
to pull rank and say that the mathematics says such-and-such. The PCTer
of equal mathematical skill would say that the wrong mathematics is
being used; the right mathematics says something else. Of course the
mathematics would be correct in either case; the real argument is about
what conception of control to analyze mathematically.
So who is right? Neither one. In fact, we know very little about how
human beings achieve control of the things they control. We can model
simple control processes very well, but complex ones hardly at all. The
theory (whichever one) tells us what _should_ be going on, but that cuts
no ice in science. You can't prove anything by working out what your
mathematical theory implies. That only demonstrates internal
consistency.
What's necessary is to compare implications of a theory against
observations of what actually happens, in such a way that if the theory
is incorrect you will know it. That, and only that, will allow us to
defend a preference for one theory over the other.
-----------------------------------------------------------------------
Best to all,
Bill P.