Adaptation, calibration, anticipation, models, reality and the environment

CSGnet Archive CSG1998
1

[From Rupert Young (980730.1204 BST)]

I�ve been having a look at Bill's artificail cerebellum (AC, armac demo) and
there's a number of interesting points which I would like to bring up,
particularly concerning Adaptation and Anticipation.

Here are my corresponding thoughts on these topics and I welcome comments or
corrections, particularly with reference to any misrepresentations of PCT.

1) Adaptation and calibration

In the AC system, control is initially poor but the function transforming the
error signal (and signals from recent past) to the output adjusts until it
gives the right transfer allowing good control. This adaptation process seems
to be deriving the correct correlation between the error and the amount
of output required to bring the perception in line with the reference. For
example, in eye fixation adaptation adjusts parameters representing how the
error transforms to the eye movement until there is a one to one correlation
between the amount of eye movement required for fixation and the movement
produced.

In the 3D computer vision work I am doing at the moment I am optimising camera
parameters (eg. focal length) until the error in image position of detected
objects (with known world position) and the re-projected image position is
minimised. To some extent the purpose is to determine the _real_ camera
parameters so that the world 3D position of objects can be computed from their
image positions. On the other hand it is enough to determine the _effective_
parameters as many unknowns can affect the actual values.

Anyway, there seems to be some parallels between this process of calibration
and that of adaptation. That is, that control systems are adjusting parameters
allowing them to perform better in the world. This suggests, to me at least, a
number of points,

a. The adaptation or learning process is, in some sense, determined or guided
by the environment. No, I'll rephrase that. The adaptation _process_ is
determined by the control system but the end result of the process, ie. what
the system learns to control, depends upon the environment. And so,
ultimately, who we are and our behaviour is determined by the environment. For
example, I speak English, and not another language, because of my (early)
environment. This is a view which doesn't seem to fit well with the PCT
thesis of control _from_ the system.

b. Although we can't directly experience any objective reality, adaptation is
a process of bringing our perceptions (measurements/observations of the
world) into line with that reality. If this were not the case, if our control
was not successful, we would still experience error.

c. Input functions which develop through learning can be viewed as a model of
the world or models of objective signals in the world.

2) Anticipation and modelling

The AC demo adapts its behaviour to a regular pattern of input such as a
square wave and ends up anticipating changes in reference signals.

I've been talking to a colleague about tracking with kalman filters (btw, if
anyone feels like giving a tutorial about kalman filters that would be
useful). It seems that kalman filters derive their tracking behaviour, where
to look next, from two sources, a predictive motion model of the object being
tracked and from extrapolating current observations of the object. When the
model predictions agree with the observations more emphasis is given to those
predictions and less to the observations. When predictions do not agree
with observations then more emphasis is given to the current observations for
purposes of extrapolation. (Does this make sense?)

The idea, I guess, is to use the model for anticipating where the object is,
if the observations are poor or intermittent. For example, for visually
tracking a plane when the light is poor, it's raining heavily and it keeps
going behind clouds.

Some more points,

d. In this situation it seems reasonable to say that we have a model of the
plane's behaviour, as we are able to estimate roughly where the plane will
appear from behind the cloud. But what do we mean by a model ? Is a vague
idea of what planes do, anything like a mathematical motion model ?

e. Perhaps adaptation is a process of building up a model of the input signal
so that it is possible to carry on control even though the _present_
perceptions are poor or intermittent.

f. If what I've been saying so far is valid I'm finding it difficult to see
the differences between the two approaches except that one controls output and
one input. But maybe that difference is enough ?

I think I'll leave things there for the moment as I'm sure the whole I've dug
for myself is quite sufficient.

Cheers,
Rup

2

[From Bill Powers (980730.0947 MDT)]

Rupert Young (980730.1204 BST)--

1) Adaptation and calibration

...

a. The adaptation or learning process is, in some sense, determined or guided
by the environment. No, I'll rephrase that. The adaptation _process_ is
determined by the control system but the end result of the process, ie. what
the system learns to control, depends upon the environment.

Not quite, because it is the organism's perceptual system that determines
what variables will be perceived, and the organism's reference signals that
determine the desired states of those variables. The environment determines
only what actions must be performed to control those variables. If you want
to loosen the cap on the jar, the environment dictates that you must turn
it counterclockwise. But the environment doesn't determine that you want to
get the cap off. Ultimately, it is the organism's inner control of its own
intrinsic state that determines how it will control the local environment.

And so,
ultimately, who we are and our behaviour is determined by the environment.

Sorry, I totally disagree. Ultimately, it is the local environment that is
determined by our intentions.

For
example, I speak English, and not another language, because of my (early)
environment.

And because of your desire or intention to communicate. You can decide to
move to France and speak French.

This is a view which doesn't seem to fit well with the PCT
thesis of control _from_ the system.

That's right, it doesn't.

b. Although we can't directly experience any objective reality, adaptation is
a process of bringing our perceptions (measurements/observations of the
world) into line with that reality. If this were not the case, if our control
was not successful, we would still experience error.

If control is not successful, we experience error. But the environment
doesn't determine what state of itself constitutes zero error. Different
organisms in the same environment have different perceptions of it and
different reference levels for it. Some people even like tofu, or claim
they do.

If you examine the demo program more closely, you will see that it is the
output function, not the perceptual function, that the system is adapting
to improve its own control over the environment.

c. Input functions which develop through learning can be viewed as a model of
the world or models of objective signals in the world.

I think that what we learn are the perceptions that benefit us when we
control them. Perceptions can be composed of any arbitrary functions of
detailed environmental variables; they don't have to have any objective
significance.

2) Anticipation and modelling

The AC demo adapts its behaviour to a regular pattern of input such as a
square wave and ends up anticipating changes in reference signals.

Yes. And when the square-wave turns into a sine-wave, the adaptation has to
be done all over again to achieve good control, and yet again when the
disturbances become random. The most generally useful adaptation is the
adaptation to a random disturbance, because it works very well for all
waveforms.

I've been talking to a colleague about tracking with kalman filters (btw, if
anyone feels like giving a tutorial about kalman filters that would be
useful). It seems that kalman filters derive their tracking behaviour, where
to look next, from two sources, a predictive motion model of the object being
tracked and from extrapolating current observations of the object. When the
model predictions agree with the observations more emphasis is given to those
predictions and less to the observations. When predictions do not agree
with observations then more emphasis is given to the current observations for
purposes of extrapolation. (Does this make sense?)

The concept makes sense -- this is the model-based control system that Hans
Blom was talking about. I don't know of any experiments with living systems
that would indicate use of such a method of adaptation, although it might
occur. It's chiefly something that engineers use in designing artificial
adaptive systems. I doubt that organisms use this method, since it's
extremely computation-intensive and requires high precision (because
there's no immediate feedback to correct small errors).

d. In this situation it seems reasonable to say that we have a model of the
plane's behaviour, as we are able to estimate roughly where the plane will
appear from behind the cloud. But what do we mean by a model ? Is a vague
idea of what planes do, anything like a mathematical motion model ?

That's a good question. A perceptual model of the plane's behavior can be
empirically derived from curve-fitting. But a different kind of model would
_simulate_ the plane's behavior as a function of underlying physical laws,
a much more complex process. The Kalman-filter approach seems to use the
latter method.

e. Perhaps adaptation is a process of building up a model of the input signal
so that it is possible to carry on control even though the _present_
perceptions are poor or intermittent.

Yes, that would be useful. However, in all cases I know of relating to
living systems, loss of real-time information leads very quickly to worse
and worse control. Maybe a little control is better than no control at all,
but one can't say that control continues unchanged when perceptions are
poor or intermittent. One of the biggest unsolved problems with this is how
the acaptive system can decide whether the perceptual system has really
started behaving poorly or whether the environment is actually behaving in
some peculiar way. If it's the environment that has changed its behavior,
you don't want to change the perceptual system, because it's still telling
the truth.

f. If what I've been saying so far is valid I'm finding it difficult to see
the differences between the two approaches except that one controls output

and

one input. But maybe that difference is enough ?

It's enough. Control systems control their inputs by _varying_ their
outputs, not controlling them.

Best,

Bill P.

3

[From Rupert Young (980731.1700 UT)]

Bill Powers (980730.0947 MDT)

>ultimately, [snip] our behaviour is determined by the environment.

Sorry, I totally disagree. Ultimately, it is the local environment that is
determined by our intentions.

By this do you mean that we shape our local environment according to our
intentions and desires, ie. move things around, boil kettles, make pizza,
build houses, roads etc ?

In the previous paragraph you had said "The environment determines only what
actions must be performed to control those variables". Are you distinguishing
between "behaviour" and "actions" ? If so, what are your definitions ?

I had said "And so, ultimately, who we are [snip] is determined by the
environment". Perhaps, here is a fundamental difference between PCT and
conventional cog sci (CCS), that CCS defines "who we are" by our behaviour
whereas PCT defines "who we are" by the variables we control, our intentions.
Though there has been quite a lot of discussion about Intentionality in recent
philosophy of the mind.

>For
>example, I speak English, and not another language, because of my (early)
>environment.

And because of your desire or intention to communicate. You can decide to
move to France and speak French.

Yes, though I didn't have much say in the matter when I was five, especially
as we were living in Bangkok. That I learned _a_ language was due to my
desire
to communicate, but the fact that the perceptual functions which developed,
developed as "english" perceptual functions was due to what was available in
the environment. Different perceptual functions develop in different
environments, surely ?

>b. Although we can't directly experience any objective reality, adaptation is
>a process of bringing our perceptions (measurements/observations of the
>world) into line with that reality.

If you examine the demo program more closely, you will see that it is the
output function, not the perceptual function, that the system is adapting
to improve its own control over the environment.

Ok, I think I'm getting my inputs and outputs mixed up. How about,
adaptation/learning can be a process of aligning our _actions_ with some
independent reality ? As in, if I'm learning to serve in tennis how my output
function develops depends upon the distance from the baseline to the receivers
box, which is independent of the control system (me). This is calibration,
methinks.

>e. Perhaps adaptation is a process of building up a model of the input signal
>so that it is possible to carry on control even though the _present_
>perceptions are poor or intermittent.

Yes, that would be useful. However, in all cases I know of relating to
living systems, loss of real-time information leads very quickly to worse
and worse control.

Wolpert et al, (1995) "An internal model for sensorimotor integration"
Science V. 269, 1880-1882.

Wolpert claimed, through his experiments, that errors in estimates of hand
location after loss of visual feedback did not become worse and worse. And he
proposed that an internal model, kalman filter type system could explain this.

The experiment went something like this. Subjects place hand on a device
which tracks its position. Lights are turned off. Subject moves hand. Subject
estimates hand position.
The error in estimation first goes up, to a peak (1 cm error) for movements of
1 second duration then _declines_ and levels off at 0.5 cm. Wolpert explains
this by an internal model which kicks in at 1 sec, if I read him right.
Presumably if control was getting worse and worse then so would the location
estimate error.

Control systems control their inputs by _varying_ their
outputs, not controlling them.

It's difficult to keep that in mind as well as its implications. CCS has
always worked with the benefit of 20/20 hindsight and can claim quite
reasonably and legitimately that the living system under study _did_ generate
a set of outputs.

Cheers,
Rup

4

[From Bill Powers (980731.13231 MDT)]

Rupert Young (980731.1700 UT)--

By this do you mean that we shape our local environment according to our
intentions and desires, ie. move things around, boil kettles, make pizza,
build houses, roads etc ?

Yes, that's what I meant.

In the previous paragraph you had said "The environment determines only what
actions must be performed to control those variables". Are you distinguishing
between "behaviour" and "actions" ? If so, what are your definitions ?

Yes. Actions are the outputs we use to affect the results or consequences
that we are perceiving and controlling. Since we're dealing with a
hierarchical model, to avoid confusion we have to stay focussed on one
level when we speak of inputs and outputs, so the "standard PCT block
diagram" applies. The input quantity qi is the controlled consequence, the
output quantity qo is the action of the system.

In common language, the word "behavior" does not distinguish between
actions (variable outputs) and their consequences (controlled inputs). We
can speak of swinging a hammer as a behavior, or of driving a nail into a
board as a behavior. Swinging a hammer is close to a description of an
action, while driving a nail into a board is more of a description of a
controlled consequence of the action. When we want to be precise in PCT, we
speak of actions and controlled consequences of actions. When we're just
referring loosely to what people "do," we can use terms like "behavior."

I had said "And so, ultimately, who we are [snip] is determined by the
environment". Perhaps, here is a fundamental difference between PCT and
conventional cog sci (CCS), that CCS defines "who we are" by our behaviour
whereas PCT defines "who we are" by the variables we control, our
intentions.

Though there has been quite a lot of discussion about Intentionality in
recent philosophy of the mind.

This goes way back to Brentano, who stole the word "intention" away from
those who used it to indicate purposive processes, and changed its meaning
to "aboutness" or "reference" or something vague like that. In that
parlance, a word such as "apple" would be intentional in that it is used to
refer to a round red object in the environment, or is in some sense "about"
that object. The reason he could get away with that is that most
philosophers of science had agreed that _real_ intentions, which imply
purposive behavior, did not exist, so the word was free for adoption for
other uses. Recent uses of the word that I have seen appear to be in line
with Brentano's usurpation of the term.

>For
>example, I speak English, and not another language, because of my (early)
>environment.

And because of your desire or intention to communicate. You can decide to
move to France and speak French.

Yes, though I didn't have much say in the matter when I was five, especially
as we were living in Bangkok. That I learned _a_ language was due to my
desire
to communicate, but the fact that the perceptual functions which developed,
developed as "english" perceptual functions was due to what was available in
the environment. Different perceptual functions develop in different
environments, surely ?

Yes, I agree that they do. But which perceptual functions we end up
retaining is determined by how controlling those perceptions helps us to
maintain control over our own intrinsic variables. The environment doesn't
care or know about our intrinsic variables.

If you examine the demo program more closely, you will see that it is the
output function, not the perceptual function, that the system is adapting
to improve its own control over the environment.

Ok, I think I'm getting my inputs and outputs mixed up. How about,
adaptation/learning can be a process of aligning our _actions_ with some
independent reality ? As in, if I'm learning to serve in tennis how my
output function develops depends upon the distance from the baseline to

the >receivers box, which is independent of the control system (me). This
is >calibration, methinks.

I don't know what "aligning my actions with reality" means. Since I can't
know about reality directly, I have no way to see if the actions are
aligned with anything. I think that what we do is (a) learn to perceive in
ways that produce well-behaved perceptual signals, and (b) learn to produce
outputs that reliably (although by unknown means) affect those perceptual
signals. In order for reliable effects to be produced, of course, we must
produce the right actions in the right directions and to the right degrees.
But there are many different actions that can produce the same results,
enabling us to control them by different means, and the existence of
independent disturbances in the environment means that we MUST vary our
actions if the consequences are to remain the same. I don't think that
"alignment" quite captures all the nuances of that picture.

Wolpert et al, (1995) "An internal model for sensorimotor integration"
Science V. 269, 1880-1882.

Wolpert claimed, through his experiments, that errors in estimates of hand
location after loss of visual feedback did not become worse and worse.

And >he proposed that an internal model, kalman filter type system could
explain >this.

There is, of course, kinesthetic feeback which tells us where our hands are
relative to our bodies in the absence of visual information. The Kalman
filter approach also requires feedback information, although it is used to
update an internal model instead of directly.

In a tracking task where the target is being driven by a smoothed random
disturbance, total loss of visual information will very quickly make
tracking impossible: errors will increase quickly to 100% (or more). This
will be true of the Kalman filter approach, too, because the KF requires
visual information for updating the internal model. In experiments where
loss of visual information does not eventually lead to loss of control,
there is some other feedback path being used in its place. Perhaps from
your familiarity with Wolpert's experiment you can figure out what it was.
I assure you that it's there.

Control systems control their inputs by _varying_ their
outputs, not controlling them.

It's difficult to keep that in mind as well as its implications. CCS has
always worked with the benefit of 20/20 hindsight and can claim quite
reasonably and legitimately that the living system under study _did_
generate a set of outputs.

Yes, of course. All control systems have to generate some kind of outputs.
But because of disturbances of various kinds, they achieve consistent
control of consequences through _variable_ outputs.

Best,

Bill P.

5

[From Oded Maler (980731)]

Bill Powers (980731.13231 MDT)]

>Though there has been quite a lot of discussion about Intentionality in
>recent philosophy of the mind.

This goes way back to Brentano, who stole the word "intention" away from
those who used it to indicate purposive processes, and changed its meaning
to "aboutness" or "reference" or something vague like that. In that
parlance, a word such as "apple" would be intentional in that it is used to
refer to a round red object in the environment, or is in some sense "about"
that object. The reason he could get away with that is that most
philosophers of science had agreed that _real_ intentions, which imply
purposive behavior, did not exist, so the word was free for adoption for
other uses. Recent uses of the word that I have seen appear to be in line
with Brentano's usurpation of the term.

I won't phrase it this way. The intentionality of Berntano is more
general than the intention you speak of. I is related to the connection
to the world. When I see or think about an apple, without any intention
to see it in some state, than the relation between my thought and some
object out there is this "aboutness". If in addition I control for
perceiving it in my hand, still this intentionality exist. Without it
all you could say about humans that they control for a firing rate of
x MHz in Neuron YYYY. The moment you (and others here) say that a person
is controlling for a perception that something is in some position,
you already invoke Berntano's intentionality.

--Oded

6

[From Rupert Young (980112.1300 UT)]

Bill Powers (980731.13231 MDT)

Thanks.

In common language, the word "behavior" does not distinguish between
actions (variable outputs) and their consequences (controlled inputs). We
can speak of swinging a hammer as a behavior, or of driving a nail into a
board as a behavior. Swinging a hammer is close to a description of an
action, while driving a nail into a board is more of a description of a
controlled consequence of the action. When we want to be precise in PCT, we
speak of actions and controlled consequences of actions. When we're just
referring loosely to what people "do," we can use terms like "behavior."

So it's correct to say

"actions (or outputs) control perceptions (or inputs)"

but incorrect to say

"behaviour controls perceptions"

hence

"behaviour: the control of perception"

referring to behaviour as encompassing everyting we do with respect to
_purposive_ behaviour.

>Wolpert et al, (1995) "An internal model for sensorimotor integration"
>Science V. 269, 1880-1882.

In a tracking task where the target is being driven by a smoothed random
disturbance, total loss of visual information will very quickly make
tracking impossible: errors will increase quickly to 100% (or more). This
will be true of the Kalman filter approach, too, because the KF requires
visual information for updating the internal model. In experiments where
loss of visual information does not eventually lead to loss of control,
there is some other feedback path being used in its place. Perhaps from
your familiarity with Wolpert's experiment you can figure out what it was.
I assure you that it's there.

Proprioception, I guess (is that the same as kinesthetic feeback?). It was
not clear to me why Wolpert needed to hypothesise an internal model when this
source of feedback was still available.

Regards,
Rupert

7

[From Bill Powers (980802.1427 MDT)]

Hi, Rupert --

Ah, the student is beginning to see through the teacher:

So it's correct to say

"actions (or outputs) control perceptions (or inputs)"

but incorrect to say

"behaviour controls perceptions"

hence

"behaviour: the control of perception"

referring to behaviour as encompassing everyting we do with respect to
_purposive_ behaviour.

The title of the book is stated in the terms nearest to popular meanings.
It is expected that a person who understands PCT will find some flaws in
this way of putting it.

Re: Wolpert:

Proprioception, I guess (is that the same as kinesthetic feeback?). It was
not clear to me why Wolpert needed to hypothesise an internal model when
this source of feedback was still available.

I don't understand it, either.

Best,

Bill P.