another (?) Test for the controlled variable

[HANS_BLOM2]

------------------- ASCII.ASC follows --------------------
[Hans Blom, 970116]

On Another (?) Test for Control

Very succinct abstract: Testing whether a system is controlled is
mathematically equivalent to modelling that system.

How do we get to know the world -- or any other system -- out there?
The answer of the science called Systems Identification is: conduct
experiments. To be more specific: act on the thing ("system") that you
want to get to know and observe how it reacts. The hope is that the
relationship between the actions on and the reactions of the system
can be characterized by a few parameters only. If so, these parameters
characterize (the knowledge that is obtained about) the system. An
example: what is an object's color? An identification procedure might
be to shine monochromatic light of various wavelengths on the object
and measure the intensity of the reflected light. The amplitudes of
the reflections at different wavelengths collectively describe the
object's color.

There is probably not a single physical system that can be fully
described by just a few parameters: we are only able to construct
models, approximations of the real thing. Yet, these approximations
can be as complete and accurate as we want them to be, by modelling
ever more details. Yet, modelling these details conflicts with our
other goal of characterizing our knowledge with _as few parameters as
possible_. We prefer simple models, because we can _use_ them more
easily.

But how simple can the model be? In general, that depends on the
application, on what the model is used _for_. Frequently, to our
surprise, a simple model will do. Newton, for instance, demonstrated
that if it is our goal to know the gravitational attraction between
two "bodies", a model only needs the masses of those two bodies and
their distance, and _not_ those bodies' size, surface temperature,
luminosities or whatever.

Systems Identification provides a general answer (within a given
context): try out a range of models, and pick the "best" one using the
dual criteria of model fit (how well does the model describe the data)
and model economy (how many parameters does the model have). This is
most easily explained if we assume that the system is linear and time-
invariant (this assumption can later be tested, and if it is incorrect
a more complex model could be chosen). A schematic diagram of a
general single input single output (SISO) system is:

···

----------
   u (k) | | y (k)
   ----->| system |----->
         > >
         ----------

where u (k) is the system's input at time k and y (k) is the system's
response at time k. By sampling the input and output at regular time
intervals 0, 1, ..., k, k+1, ..., a general description results in the
form of a difference equation; if we observe continuously, we get
differential equations. A discussion in terms of difference equations
is simpler.

How do we model the system? Well, the general idea is that the
system's output somehow depends on its input. It is the "somehow" that
we have to determine. There are a lot of different methods, all
mathematically (more or less) equivalent, so I'll describe the
simplest, the "finite impulse response" (FIR) method. In order of
complexity from simplest to more complex, descriptions of possible
models would be

y (k) = a (order 0)
y (k) = a + b * u (k) (order 1)
y (k) = a + b * u (k) + c * u (k-1) (order 2)
y (k) = a + b * u (k) + c * u (k-1) + d * u (k-2) (order 3)

etcetera, where a, b, c, d, ... are constants. You get the idea. The
constants are initially unknown, but they can be estimated, e.g. using
a least-squares ("curve fitting") method. All we have to do is to
collect a sufficiently large set of input and output values and let
the least squares method do its work (either in a batch mode or real-
time/on-line). It will present us with the unknowns a, b, c, d... Once
we have these, we can use those same models to generate predictions,
using the formulas

ym (k) = am (order 0)
ym (k) = am + bm * u (k) (order 1)
ym (k) = am + bm * u (k) + cm * u (k-1) (order 2)
ym (k) = am + bm * u (k) + cm * u (k-1) + dm * u (k-2) (order 3)

where the constants am, bm, ... are the results of the least-squares
estimation procedure, and ym is the model's prediction. Depending on
the order of the model, the (average squared) difference between y (k)
and ym (k) will vary. If the order is high enough and the model is
good, differences will be small.

The order 0 model is probably pretty bad, at least if the input has a
significant and consistent effect on the output -- which is not
necessarily always the case. The order 1 model might be a little bit
better, and the order 2 model even better than that. Practice shows
that, starting from a certain "high enough" model order, the model
predictions hardly improves anymore. In that case we stop, and, if the
model's predictions are sufficiently accurate compared to the response
of the real system, we accept the model as our "knowledge" of the
system. If not, we pick a different type of model (e.g. one including
nonlinearities), construct a corresponding sequence of ever more
complex models, and try again. Once we have a good enough model, we
can use it, e.g. in model-based control.

All of this is probably pretty familiar to you. Now comes the thing
that may be new. In the above model, we have used the inputs u (k) as
the independent variables and the outputs y (k) as the dependent
variables. This is called a "forward" model, and in the example above
it described what is called the system's "impulse response". But there
also exists a "backward" model, in which independent and dependent
variables change places. The models to be tried out are now:

u (k) = a (order 0)
u (k) = a + b * y (k) (order 1)
u (k) = a + b * y (k) + c * y (k-1) (order 2)
u (k) = a + b * y (k) + c * y (k-1) + d * y (k-2) (order 3)

etcetera, where a, b, c, d are again constants (but different ones!).
These relationships now describe HOW u DEPENDS ON y! If it does not,
i.e. if u is truly independent of y, no higher order model will turn
out to be significantly better than a model of order 0. This is easy
to understand: if u does not depend on y in any (possibly time-
shifted) way, it will also not depend on earlier values of y (and,
moreover, constants b, c, d... will all turn out to be zero as well).

If, however, u DOES somehow depend on y, there is what we have learned
to call _feedback_ (from y to u) and thus we have a control system!
(Footnote: One might pose additional requirements for calling the
feedback thus discovered "control", such as a high enough loop gain).
Moreover, the estimates of the parameters a, b, c, d... present us
with the "control law", i.e. the way in which u (the control system's
"action") is "computed" based on y (what PCT calls the control
system's "perception"). The procedure to establish whether control is
present is obvious: establish whether higher order models give
improved predictions (note that u, the "action", is predicted in this
case!). Clearly, if the "action" cannot be predicted from the
"perceptions" in any way, there is no feedback/control. The procedure
to discover the governing control law is clear as well: increase the
backward model order until a higher order model does not give improved
(more accurate) predictions anymore.

The procedure to establish whether a variable y is controlled is thus
as simple as can be: collect a sequence of u/y values and find out
whether u depends on y. Or is this "nothing but" a formalization of
The Test?

The above is an informal description of some aspects of a paper by
Niu, Fisher, Ljung and Shah "A Tutorial on Multiple Model Least-
Squares and Augmented UD Identification" (1995), a Postscript version
of which can be found by connecting to

     http://ankeborg.isy.liu.se/cgi-bin/reports

This paper describes a method to simultaneously discover all forward
and backward models (of a variety of types) up to a maximum chosen
model order in a very efficient way. It describes both batch (off-
line) and real-time (on-line) algorithms. The latter can be used in
model-based/adaptive control.

A nice connection of papers on many aspects of Systems Identification
can be found at this address as well, including a general tutorial on
Systems Identification.

Greetings,

Hans

[Bill_Powers1]

[From Bill Powers (970116.1130 MST)]

Hans Blom, 970116 --

A lucid and interesting post, Hans. I agree with your opening statement:

Very succinct abstract: Testing whether a system is controlled is
mathematically equivalent to modelling that system.

Of course in the case of modeling an organism, we have the other side of the
same coin: Testing whether an organism is controlling a given system in the
environment is mathematically equivalent to modeling that organism.

The Test is nothing more than a common-sense way of checking to see whether
the behavior of the organism in relation to the observable environment fits
the definition of control behavior as given in PCT.

In truth, as your exposition so clearly shows, we must model BOTH the
organism and its environment. Usually the model of the environment is very
simple to achieve, because we can experiment with it directly and analyze it
using physical methods (or we created it ourselves for experimental
purposes). I would guess that when you speak of a "system" you are including
both the organism and the environment.

In your first model, you develop something that looks to me like the
convolution theorem:

y (k) = a (order 0)
y (k) = a + b * u (k) (order 1)
y (k) = a + b * u (k) + c * u (k-1) (order 2)
y (k) = a + b * u (k) + c * u (k-1) + d * u (k-2) (order 3)

This says that for an nth-order system,

          tau=n
y(k) = a + SUM[ u(k-tau)*f(tau)]
          tau=0

The constants b..z are the successive values of f(tau), which is the impulse
response of the system (as you later mention). The left-over _a_ is an
arbitrary constant. In general, a should be a function of time (or k),
because it represents the sum of all influences other than u that can
contribute to the state of y(k). Thus _a_ represents "the disturbance" in
the model of the environmental part of the whole system, and "the reference
signal" in the organismic part.

The
constants are initially unknown, but they can be estimated, e.g. using
a least-squares ("curve fitting") method. All we have to do is to
collect a sufficiently large set of input and output values and let
the least squares method do its work (either in a batch mode or real-
time/on-line). It will present us with the unknowns a, b, c, d... Once
we have these, we can use those same models to generate predictions,
using the formulas

ym (k) = am (order 0)
ym (k) = am + bm * u (k) (order 1)
ym (k) = am + bm * u (k) + cm * u (k-1) (order 2)
ym (k) = am + bm * u (k) + cm * u (k-1) + dm * u (k-2) (order 3)

where the constants am, bm, ... are the results of the least-squares
estimation procedure, and ym is the model's prediction. Depending on
the order of the model, the (average squared) difference between y (k)
and ym (k) will vary. If the order is high enough and the model is
good, differences will be small.

In my artificial cerebellum model, I use a simpler method for estimating the
various coefficients. The "system" I'm dealing with is just the output
function, but the principle is the same. The A-C model I sent you uses
somewhere between an 80th order and a 120th order model, in your terms. I
forget just how many values of f(tau) are maintained. The iterative method I
use is a very simple form of a "method of steep descent."

A practical problem arises if the plant or the organism function involves a
true integration, or a leaky integrator with a long time constant. There
must be coefficients representing influences of u(k-tau) over very long
times tau into the past, because a value of u from 1000 or more time units
ago can have almost as much influence on y(k) as the present value has. This
makes the problem of evaluating the coefficients by the method you describe
look very hard -- unless you have some way of doing it that avoids inverting
gigantic matrices.

When two integrations are involved, the problem becomes even worse. Each
term in y(k) must be a sum over a long time series of similar form.

The double integral form occurs whenever the "plant" is described by a
second-order differential equation (which is very common). A freely-moving
mass is the worst case, because the current position of the mass is a true
double integral of the forces applied to it in the past, over an indefinite
period of time.

All of this is probably pretty familiar to you.

Well, not as familiar as my words above may imply. I find this stuff pretty
difficult.

Now comes the thing
that may be new. In the above model, we have used the inputs u (k) as
the independent variables and the outputs y (k) as the dependent
variables. This is called a "forward" model, and in the example above
it described what is called the system's "impulse response". But there
also exists a "backward" model, in which independent and dependent
variables change places. The models to be tried out are now:

u (k) = a (order 0)
u (k) = a + b * y (k) (order 1)
u (k) = a + b * y (k) + c * y (k-1) (order 2)
u (k) = a + b * y (k) + c * y (k-1) + d * y (k-2) (order 3)

etcetera, where a, b, c, d are again constants (but different ones!).
These relationships now describe HOW u DEPENDS ON y! If it does not,
i.e. if u is truly independent of y, no higher order model will turn
out to be significantly better than a model of order 0. This is easy
to understand: if u does not depend on y in any (possibly time-
shifted) way, it will also not depend on earlier values of y (and,
moreover, constants b, c, d... will all turn out to be zero as well).

This isn't really new, is it? If your first set of equations describe the
"environmental feedback function", then what you call the "backward"
equations describe the "organism function." That's the standard PCT way of
analyzing a control system, although the analysis is much less sophisticated.

This is, by the way, one of the things I've chastized operant-conditioning
modelers about. They've tried to write equations for behavior using only the
environmental part of the expressions above, eschewing guesses about the
organism function. This is obviously futile, because there are two variables
(u and y) and you need two sets of equations to solve for them.

If, however, u DOES somehow depend on y, there is what we have learned
to call _feedback_ (from y to u) and thus we have a control system!
(Footnote: One might pose additional requirements for calling the
feedback thus discovered "control", such as a high enough loop gain).

Not to mention NEGATIVE feedback at all frequencies where the absolute loop
gain is greater than 1.

Moreover, the estimates of the parameters a, b, c, d... present us
with the "control law", i.e. the way in which u (the control system's
"action") is "computed" based on y (what PCT calls the control
system's "perception"). The procedure to establish whether control is
present is obvious: establish whether higher order models give
improved predictions (note that u, the "action", is predicted in this
case!). Clearly, if the "action" cannot be predicted from the
"perceptions" in any way, there is no feedback/control. The procedure
to discover the governing control law is clear as well: increase the
backward model order until a higher order model does not give improved
(more accurate) predictions anymore.

Very nicely put. I will have some comments later, but you have said this
very clearly.

The procedure to establish whether a variable y is controlled is thus
as simple as can be: collect a sequence of u/y values and find out
whether u depends on y. Or is this "nothing but" a formalization of
The Test?

It's the best way of doing the Test: set up a complete model, fit it to the
data, and see whether the loop gain is high enough to merit the term
"control system." All other ways of using the Test are only an approximation
of this way.

The above is an informal description of some aspects of a paper by
Niu, Fisher, Ljung and Shah "A Tutorial on Multiple Model Least-
Squares and Augmented UD Identification" (1995), a Postscript version
of which can be found by connecting to

    http://ankeborg.isy.liu.se/cgi-bin/reports

This paper describes a method to simultaneously discover all forward
and backward models (of a variety of types) up to a maximum chosen
model order in a very efficient way. It describes both batch (off-
line) and real-time (on-line) algorithms. The latter can be used in
model-based/adaptive control.

I'll look it up. However, I can't print Postscript, so I hope there's
another version available.

PCT, of course, is all about system identification. However, because we're
dealing with organisms up to human beings, we also have a problem that you
might call "variable identification." Just what is u, and what is y? This
isn't self-evident just from an examination of the physical variables involved.

The nature of y depends on the perceptual input functions that exist between
the environment and the place in the control system where it measures y. In
order for an observer to measure y, the observer must apply the _same_
perceptual input function to the physical variables in the environment.
Similarly for u: we can't tell just from measuring the physical outputs of
the system what _aspect_ of those outputs is relevant to having an effect on
y. We can measure, for example, the forces created by muscles and the way
they are transmitted to the bones and from there to the environment. But how
those forces get translated into the events that end up affecting y, where y
is the experienced sweetness of a glass of lemonade, is far from obvious. We
need to define u in more tractable terms, for example as the number of
teaspoons of sugar added to the pitcher of lemonade between sips. It's
hopeless to think of a model in which the nonlinear dynamics of arm control
are part of the model, and which includes such things as grasping a teaspoon
(after finding it!), dipping it into a bowl of sugar, and moving it, holding
it level, to the point over the pitcher where it can be rotated to dump the
sugar -- and then stirring it in. The output has to be defined in some way
that's compatible with the kind of controlled variable we're talking about.
We have to leave the details of HOW that output is produced for a different
level of discussion.

The problem with REAL model-based control is that we can't model most of the
things that go on; there are simply too many of them. Everything having to
do with behavior would, strictly speaking, have to include all the
second-order nonlinear differential equations that describe how the dynamics
of body movements arise in response to torques applied at all the joints in
the body -- and, of course, the detailed physics of the local environment.
This is basically why I don't buy model-based control as a _general_ model
of behavior. I KNOW I couldn't write such a model; I don't THINK you could,
and I very much doubt that the brain can run such a model, either. If there
is real model-based control going on in the brain, it's got to be of a much
simpler and more restricted kind.

Best,

Bill P.

[HANS_BLOM2]

[Hans Blom, 970120]

(Bill Powers (970116.1130 MST))

Very succinct abstract: Testing whether a system is controlled is
mathematically equivalent to modelling that system.

Of course in the case of modeling an organism, we have the other
side of the same coin: Testing whether an organism is controlling a
given system in the environment is mathematically equivalent to
modeling that organism.

My proposal was subtly different: don't model the organism/controller
-- that might be far too complex -- but (cor)relate its output/action
with its input/perception and check whether its action is somehow
"predicted" by -- or correlated with -- its perception.

In truth, as your exposition so clearly shows, we must model BOTH
the organism and its environment. Usually the model of the
environment is very simple to achieve, because we can experiment
with it directly and analyze it using physical methods (or we
created it ourselves for experimental purposes).

The funny thing is that we have exactly the same information about
the controller (organism) and the controlled system (environment):
input/output value pairs. In the one case, we have the forward model
y = f (u), in the other case the backward model u = f' (y). So, if we
do not have additional information, modelling one is equivalent to
modelling the other.

The left-over _a_ is an arbitrary constant. In general, a should be
a function of time (or k), because it represents the sum of all
influences other than u that can contribute to the state of y(k).
Thus _a_ represents "the disturbance" in the model of the
environmental part of the whole system, and "the reference signal"
in the organismic part.

In general, it is very advantageous to separate the model of the
environment y = f (u) and the model of the disturbance.

A practical problem arises if the plant or the organism function
involves a true integration, or a leaky integrator with a long time
constant.

A finite impulse response (FIR) model can handle only those cases
where the impulse response's tail starts to approach and remains at
zero. If a FIR model is not appropriate, another set of regressions,
of y (k) on y (k-1), y (k-2), ... is required. This does not,
however, change the algorithm. See the original article on this, as
well as on how to model the disturbance (another set of regressions).

If your first set of equations describe the "environmental feedback
function", then what you call the "backward" equations describe the
"organism function." That's the standard PCT way of analyzing a
control system, although the analysis is much less sophisticated.

The mathematical equivalence of y = f (u) and u = f' (y) may clarify
why I sometimes call the controller the environment's "inverse".

About the Postscript format: do a web-search for Ghostview, a free
(!) software package that presents Postscript files on the computer's
display. Ghostview is available for a lot of platforms, including the
one you're using. It includes a utility to transform the Postscript
text to standard ascii as well. When you're downloading, get gzip as
well; most Postscript files are zipped using this zipper, which is
free as well. Originally a Unix utility, it is also available for
many other platforms. Files compacted with gzip can be recognized by
an extension .gz or .z. Unzipping is done with gzip -d <filename>.

Greetings,

Hans

[Bill_Powers1]

[From Bill Powers (970120.0600 MST)]

Hans Blom, 970120 --

My proposal was subtly different: don't model the organism/controller
-- that might be far too complex -- but (cor)relate its output/action
with its input/perception and check whether its action is somehow
"predicted" by -- or correlated with -- its perception.

With a living controller, you can't do this. There's no way to get an
organism to behave (realistically) open-loop; the input must be affected by
the output at the same time the output is affected by the input.

Basically, you begin with observations of both input and output. Given those
observations, and given the form of the environmental connection from output
to input, you can (often) deduce a sufficient model of the organism's
input-output function for a specific task.

The observed input-output correlation for compensatory tracking tasks is
close to zero. The better the person is at controlling, the lower this
correlation gets. The reason is that the controller keeps the controlled
variable at its reference level within less than the system noise amplitude,
so the irreducible system noise predominates. The best you could do would be
to derive an _average_ input-output function, but that would not explain the
opposition to relatively rapidly-changing arbitary disturbances.

In truth, as your exposition so clearly shows, we must model BOTH
the organism and its environment. Usually the model of the
environment is very simple to achieve, because we can experiment
with it directly and analyze it using physical methods (or we
created it ourselves for experimental purposes).

The funny thing is that we have exactly the same information about
the controller (organism) and the controlled system (environment):
input/output value pairs. In the one case, we have the forward model
y = f (u), in the other case the backward model u = f' (y). So, if we
do not have additional information, modelling one is equivalent to
modelling the other.

Your statement is incorrect. We have ONE input-output pair that we can
observe. These two variables must simultaneously satisfy the organism
function and the environmental function. But the organism function is in
general different from the environment function. If the environment function
is a pure integrator, the optimum organism function is a simple
proportionality. That is why we have to solve the two equations
simultaneously, to find the parameters of each model that allow the
equations to be satisfied by the one pair of observed values.
...

A practical problem arises if the plant or the organism function
involves a true integration, or a leaky integrator with a long time
constant.

A finite impulse response (FIR) model can handle only those cases
where the impulse response's tail starts to approach and remains at
zero. If a FIR model is not appropriate, another set of regressions,
of y (k) on y (k-1), y (k-2), ... is required. This does not,
however, change the algorithm. See the original article on this, as
well as on how to model the disturbance (another set of regressions).

I'm really not interested in approaches that require modeling the
disturbance. I think that is an entirely impractical approach. A control
system that has to model the disturbance is going to be very limited in its
ability to control in a real world. If you make the adaptation operate fast
enough to handle an arbitrary disturbance (extrapolating only one iteration
ahead) it can no longer operate in the absence of perceptual input. If you
make it predict some distance ahead in time, adapting more slowly by
necessity, it can no longer handle arbitrary disturbances. I've shown you
how to change your model so it can have the advantages of both real-time
control and model-based control, but evidently you're not interested. If
you're not interested, then neither am I.

If your first set of equations describe the "environmental feedback
function", then what you call the "backward" equations describe the
"organism function." That's the standard PCT way of analyzing a
control system, although the analysis is much less sophisticated.

The mathematical equivalence of y = f (u) and u = f' (y) may clarify
why I sometimes call the controller the environment's "inverse".

The inverse of the environment function in your model is not between the
perceptual input and the output; it is between the reference signal and the
output.

In a PCT control system model, we can have

(Organism) : output = k*(reference - input)
(Environment): input := integral(output)

Where "input" and "output" are defined relative to the organism. Note that
the organism function is NOT the inverse of the environment function.

About the Postscript format: do a web-search for Ghostview, a free
(!) software package that presents Postscript files on the computer's
display. Ghostview is available for a lot of platforms, including the
one you're using. It includes a utility to transform the Postscript
text to standard ascii as well. When you're downloading, get gzip as
well; most Postscript files are zipped using this zipper, which is
free as well. Originally a Unix utility, it is also available for
many other platforms. Files compacted with gzip can be recognized by
an extension .gz or .z. Unzipping is done with gzip -d <filename>.

I had it already, and just the other day downloaded the latest (Aladdin)
version together with GSview which displays postscript files on the screen
in a very simple manner. The GSview program's installation also installs the
Ghostscript files, unzipping them for you! The problem is that it takes
close to a hour to print out a ten-page document (on my dot-matrix printer).
I haven't tried the PS to text program yet -- it wouldn't help much for
documents with figures and which use graphic methods of drawing mathematical
symbols.

Best,

Bill P.

[HANS_BLOM2]

[Hans Blom, 970120b]

(Bill Powers (970120.0600 MST))

My proposal was subtly different: don't model the organism/
controller -- that might be far too complex -- but (cor)relate
its output/action with its input/perception and check whether
its action is somehow "predicted" by -- or correlated with --
its perception.

With a living controller, you can't do this. There's no way to get
an organism to behave (realistically) open-loop; the input must be
affected by the output at the same time the output is affected by
the input.

We seem to have some misunderstandings. What you say is what I mean:
get the standard closed loop measurements. Only those will show
whether control goes on. Open loop measurements are no good.

The observed input-output correlation for compensatory tracking
tasks is close to zero. The better the person is at controlling, the
lower this correlation gets.

Yes, that is a real problem in identification. If control is perfect,
the correlation goes to zero. But we do need to get that correlation
if we are to identify. The recipe is: add some (very rapid, steplike)
disturbances that cannot be counteracted (almost) perfectly.

The funny thing is that we have exactly the same information about
the controller (organism) and the controlled system (environment):
input/output value pairs. In the one case, we have the forward
model y = f (u), in the other case the backward model u = f' (y).
So, if we do not have additional information, modelling one is
equivalent to modelling the other.

Your statement is incorrect. We have ONE input-output pair that we
can observe.

Linguistic misunderstanding. I meant that we have the pairs u(k) /
y(k), u(k-1) / y(k-1), u(k-2) / y(k-2) etc. Just one sample of each
is no good.

I'm really not interested in approaches that require modeling the
disturbance. I think that is an entirely impractical approach.

Remember an earlier demo of mine in which I took a disturbance that
varied between about +1000 and -1000 and modelled it so that the
prediction could "explain" everything but an error of about +/- 1? If
the disturbance is low frequency compared to the sample interval, it
can be well predicted (one or a few samples ahead).

Note that [in the PCT equations] the organism function is NOT the
inverse of the environment function.

Agreed. In the equations that I used it is. Might that have an
interpretation or should we disregard it altogether?

Again, don't believe me; get the original article...

Greetings,

Hans

[Bill_Powers1]

[From Bill Powers (970120.1025 MST)]

Hans Blom, 970120b --

We seem to have some misunderstandings. What you say is what I mean:
get the standard closed loop measurements. Only those will show
whether control goes on. Open loop measurements are no good.

I see - you seemed to be saying that we had to measure organism and
environment separately.

The observed input-output correlation for compensatory tracking
tasks is close to zero. The better the person is at controlling, the
lower this correlation gets.

Yes, that is a real problem in identification. If control is perfect,
the correlation goes to zero. But we do need to get that correlation
if we are to identify. The recipe is: add some (very rapid, steplike)
disturbances that cannot be counteracted (almost) perfectly.

I agree; the key is to apply disturbances that are not perfectly counteracted.

The funny thing is that we have exactly the same information about
the controller (organism) and the controlled system (environment):
input/output value pairs. In the one case, we have the forward
model y = f (u), in the other case the backward model u = f' (y).
So, if we do not have additional information, modelling one is
equivalent to modelling the other.

Your statement is incorrect. We have ONE input-output pair that we
can observe.

Linguistic misunderstanding. I meant that we have the pairs u(k) /
y(k), u(k-1) / y(k-1), u(k-2) / y(k-2) etc. Just one sample of each
is no good.

I don't see how you get that from the "forward" and "backward" relations you
described. The forward relation in the environment is not the inverse of the
backward relation in the organism.

I'm really not interested in approaches that require modeling the
disturbance. I think that is an entirely impractical approach.

Remember an earlier demo of mine in which I took a disturbance that
varied between about +1000 and -1000 and modelled it so that the
prediction could "explain" everything but an error of about +/- 1? If
the disturbance is low frequency compared to the sample interval, it
can be well predicted (one or a few samples ahead).

Yes, I have recently reminded you of it, but you said you didn't remember
it. This is the demo which, when I interrupt the perceptual input function,
loses control within a few iterations -- just as fast as the PCT model. This
is the demo which, if you trace the calculations out, turns out to deal with
the disturbance just as a PCT control system with a bit of first-derivative
feedback deals with it. When I pointed out that short-term and long-term
prediction involve a tradeoff, you disagreed, for reasons I have not yet
understood (although others agreed with me).

Note that [in the PCT equations] the organism function is NOT the
inverse of the environment function.

Agreed. In the equations that I used it is. Might that have an
interpretation or should we disregard it altogether?

In the equations you use, there is a world-model (which is an approximate
model of the _foward_ form of the environment), and a calculation involving
the _inverse_ of both the world-model and the environment, which is inserted
between the reference signal and the output signal. So y = f(u), but u =
f'(r),not f'(y).

Again, don't believe me; get the original article...

I finally did get it. I'm afraid that kind of mathematical treatment is over
my head. I hope you're not suggesting that this approach is the way living
control systems solve the problem. But I fear that you are.

Best,

Bill P.

[HANS_BLOM2]

[Hans Blom, 970122]

(Bill Powers (970120.1025 MST))

The forward relation in the environment is not the inverse of the
backward relation in the organism.

Words are so confusing ;-). The following, rather standard, block
diagram applies:

  r ------ -------
------>|+ | a | |
       > C |------->| E |------
  p -->|- | | | |

···

------ ------- |

    > >
    ------------------------------

where C is the controller, E the environment, r the reference, a the
action and p the perception. I can disregard random disturbances
without loss of generality: the least squares procedure averages
those away (and does a goood job at that if given enough time).

Now the "forward" equation (of the environment) is the familiar
expression

   p = E * a

where you can read the * as a multiplication, a convolution or a
(constant) function. If both p and a can be measured, E can be
established/estimated. In a system without dynamics, E is a "gain
constant"; in a dynamical system it is e.g. an impulse response
function.

The "backward" equation (also of the environment!) is

   a = F * p (= E^-1 * p if the system has no dynamics)

and similarly, if p and a can both be measured, F (or E^-1, if ...)
can be established/estimated as well. This is, of course, in addition
to the other (control) relationship between a and p:

   a = C * (s - p)

Clearly both equations apply.

Why would we want to consider this funny backward model? All right,
you don't want to consider it ;-). Anyway, the reason is that the
action a may somehow be "caused", in our case by a control system.
And -- the subject of my previous post -- we would like to know
whether control goes on without having to model the (maybe utterly
complex) "living controller".

This "causation" is expressed by the "backward" formula, which shows
that variations of perception p may "cause" variations of the action
a. Or maybe not. In the latter case, a is totally unrelated to p. And
that in turn demonstrates that no control exists.

Why? Well, it is a controller's goal to vary the action a such that p
meets certain requirements (e.g. bring p to s as well as possible).
If p varies, yet variations in a are totally unrelated to those in p,
we conclude that no control goes on. And that _without_ having to
model the controller! By just "backward modelling" the environment!

Well, I don't think I can put this more clearly...

Again, don't believe me; get the original article...

I finally did get it. I'm afraid that kind of mathematical treatment
is over my head.

Well, too bad that now we're not able to discuss this approach, its
possible merits and implications, and its analogy to what might go on
in an organism.

I hope you're not suggesting that this approach is the way living
control systems solve the problem. But I fear that you are.

Are you afraid of the complexity? I recently read that the computing
power of the largest and fastest computers that are now available
were compared to the computing power of the brain of a fly. The fly
still wins...

And we're modelling, remember? I have not the slightest idea how a
general "living control system", if such a generalization exists,
"really" solves problems. All I can do is present analogies (models)
of how systems whose internals we _do_ know solve similar or related
problems. And, maybe, to "derive" some (sometimes pretty abstract,
and only more or less valid) generalizations of what must go on in
terms of "data transformations" that allow a system to extract
"knowledge" (discover the "laws" that exist in the environment) out
of more basic variables, such as "raw" lowest level perceptions. Too
bad if those analogies mean nothing to you. They do to me.

Greetings,

Hans

[Bill_Powers1]

[From Bill Powers (970122.1100 MST)]

Hans Blom, 970122--

Words are so confusing ;-). The following, rather standard, block
diagram applies:

r ------ -------
------>|+ | a | |
      > C |------->| E |------
p -->|- | | | |
   > ------ ------- |
   > >
   ------------------------------

where C is the controller, E the environment, r the reference, a the
action and p the perception. I can disregard random disturbances
without loss of generality: the least squares procedure averages
those away (and does a goood job at that if given enough time).

What you're doing by averaging the disturbances away is getting rid of the
main tool we use in PCT for identifying control systems.

Now the "forward" equation (of the environment) is the familiar
expression

  p = E * a

where you can read the * as a multiplication, a convolution or a
(constant) function. If both p and a can be measured, E can be
established/estimated. In a system without dynamics, E is a "gain
constant"; in a dynamical system it is e.g. an impulse response
function.

Yes, this establishes the response of the environment to the output of the
controller. The same procudure is used in PCT for calculating the effect of
output (a) on input (p) in the absence of a control system.

The "backward" equation (also of the environment!) is

  a = F * p (= E^-1 * p if the system has no dynamics)

and similarly, if p and a can both be measured, F (or E^-1, if ...)
can be established/estimated as well. This is, of course, in addition
to the other (control) relationship between a and p:

  a = C * (s - p)

I think you meant to write (r - p), didn't you?

Clearly both equations apply.

This is getting confused. In your diagram, you show _a_ as C(r,p). Where is
the function a = f * p?

While you sort that out, let's go back to your elimination of the
disturbance. You show no disturbance in your diagram, so allow me to put it
in where it would appear in a PCT analysis:

                         d

                       >

                         v

r ------ -------
------>|+ | a | |
      > C |------->| E |------
p -->|- | | | |
   > ------ ------- |
   > >
    ----------------------------

Assuming that p is an environmental controlled variable, the first step of
the PCT test for control is to determine the effect of d on p with the
controller not acting (a = 0 or a = constant). We manipulate d and measure
the effects on p.

The next step is to restore the controller to action, and again measure the
effect of d on p. The basic (but not the only) requirement for concluding
that C is controlling p is that the effect of d on p be much less when the
controller is present than when it is absent.

In other words, the basic identifier of control is the suppression of
effects of disturbances on the variable assumed to be controlled -- effects
which, without the controller present, would be substantial.

This doesn't seem to have much relationship to your way of defining control.

Best,

Bill P.

[HANS_BLOM2]

[Hans Blom, 970127c]

(Bill Powers (970122.1100 MST))

                          I can disregard random disturbances
without loss of generality: the least squares procedure averages
those away (and does a goood job at that if given enough time).

What you're doing by averaging the disturbances away is getting rid
of the main tool we use in PCT for identifying control systems.

No. Although disturbances average out, _their effect_ is consistent
and deterministic: they are handled in a certain fixed way -- by a
controller.

I think you meant to write (r - p), didn't you?

Yes. Sorry.

This is getting confused. In your diagram, you show _a_ as C(r,p).
Where is the function a = f * p?

This function simply states that the action _a_ is some consistent
function of the perception _p_. This function can be established by
measuring both _a_ and _p_ and using _p_ as the regressor. Asking
where this regression function is is like asking where control is in
the loop.

Here you describe the PCT Test for the Controlled Variable:

Assuming that p is an environmental controlled variable, the first
step of the PCT test for control is to determine the effect of d on
p with the controller not acting (a = 0 or a = constant). We
manipulate d and measure the effects on p.

The Test that I proposed is not provocative and does not manipulate
d; it just uses the normally available measurements of action and
perception in order to establish how actions depend on perceptions.
This doesn't always work, alas: there must be ("naturally occurring")
disturbances that make control less than perfect; if not, there are
no useful correlations and the system is unidentifiable. But there
are no outright manipulations of the disturbance.

This doesn't seem to have much relationship to your way of defining
control.

Right. You have convinced me that these are different tests. We can
therefore remove the question mark from the title of the posts on
this subject and consider the matter settled.

Greetings,

Hans

[Bill_Powers1]

[From Bill Powers (970127.1000 MST)]

Hans Blom, 970127c --

This function simply states that the action _a_ is some consistent
function of the perception _p_. This function can be established by
measuring both _a_ and _p_ and using _p_ as the regressor. Asking
where this regression function is is like asking where control is in
the loop.

I guess I don't follow your test. Suppose the action _a_ is the position of
a mouse, and the perception _p_ is the perception of the position of a
cursor on a screen relative to a target (which we can take to be the same as
the actual position). You can measure _a_ and by some means determine the
function that makes _p_ dependent on it.

Now, how do you do the backward part of your test?

Best,

Bill P.