Feedforward Demo

[MartinT]

[Martin Taylor 2014.02.04.07.59]

Warren, your comment "this seems to be an imagination mode process",

together with Rick’s having induced me to reread Bill’s description
of the Artificial Cerebellum, has clarified for me something that
has been bugging me for many years, especially recently.
At CSG93, I discussed with Bill this configuration of the
imagination connection, which I had used in a presentation earlier
in the year – I quote a page from that presentation, which is on my
Web site at .
------------quote---------

···

On 2014/02/4 2:20 AM, Warren Mansell
wrote:

I think that the cloud and umbrella example really illustrates that we are talking about multi-componential phenomena here. First, putting up an umbrella is a behaviour to control the perception of staying dry.
But, given it only takes a second to put up an umbrella, you wouldn't do it until the rain started anyway.
Maybe a better example is whether you would put up a canopy at you garden party if rain was predicted.
Now, the actual putting up of the canopy is all controlled perception and the question is rather whether one takes the prediction of the perception of rain as the perception of rain.
So, this one seems to be an imagination mode process.
However I am sure that other apparent feed forward examples use a different combination of PCT components....
Warren

http://www.mmtaylor.net/PCT/DFS93/DFS93_1.html

Basic elements of an Elementary Control System (ECS) showing an

“imagination loop”

  The simple ECS is not complete in the form shown earlier. It is

capable of “imagining.” In PCT, imagination is conceived as
permitting the output to be connected directly back to the input.
Since the output usually serves as a reference for lower-level
perceptions that “should be” controlled to match their references,
feeding back the output is equivalent to receiving those
perceptions. It is the difference between a commander ordering
“Take that hill” and the commander imagining “What would it be
like if that hill were to be taken.”

  The ECS also can be augmented by the addition of a memory, which

allows it to recapture a perception previously obtained, either in
the form of a reference signal (which creates an error signal that
leads to output) or as a perceptual signal perhaps together with a
reference signal, leading to a perception but no error and thus no
output.

Imagining: Planning

  The imagination is not necessarily a direct connection between

the output and the perceptual input function, as shown in the
previous slide. It is likely to go through some kind of a world
model, particularly at the higher control levels where the
perceptual input function may be logical rather than analogue in
form. The world model may well be implemented in parts of the
complicated control network that are not currently being used for
active control (and which might be specialized for the imagination
function). It allows the output signal to have effects that are
distributed over time, rather than being concentrated at the time
the output signal is issued. The world model is built up through
experience with the world, as well as by training. Testing out the
likely effects of output by imagining through the world model is
“planning.”

http://en.wikipedia.org/wiki/Linear_filter

[wmansell]

Hi Martin, that’s a really exciting idea! I had thought that the cerebellum must have a very essential function in control but I wasn’t sure exactly what.

Although there is a bit of me that says that this function within imagination might not be sufficiently fundamental given the pervasiveness of the cerebellum to all animals with a CNS. Could it not be involved more directly in timing adjustments, whatever their use in the hierarchy?

I can see that when we imagine we do also often imagine the timecourse of disturbances to our reference values, not just our reference values themselves, and that if we imagined without the inclusion of disturbances, we would not be accurately modelling an ‘as if’ mental simulation of a problem. We would just solve it.

However, with my clinical experience, and my need for parsimony, I am also thinking that if two control loops in conflict at the same level enter the imagination mode, then one would perceive a disturbance. Also, some of the most important disturbances (and feedback functions) in the world are other people. So how would a conflicting goal within one’s own hierarchy that happened to conflict with the planned goal serve as a modelled disturbance? There is only one sandwich on the table - I want it but I can easily infer from this that the other person in the room wants it to, so I run both of these control systems together - opposing one another. I can see this might be a tough one mathematically, but I do think we are much more accurate if we model the ‘world’ as another control system rather than through other means, don’t you think? Having said that I can think of various physical obstacles to progress it is important to get one’s head around too!

So, I am definitely with dissecting the imagination mode and looking to understand how it models experience over time and that experience with the environment will form the foundations for that, just not sure exactly the solution yet…

Warren

···

On Tue, Feb 4, 2014 at 3:26 PM, Martin Taylor mmt-csg@mmtaylor.net wrote:

[Martin Taylor 2014.02.04.07.59]

  On 2014/02/4 2:20 AM, Warren Mansell

wrote:

I think that the cloud and umbrella example really illustrates that we are talking about multi-componential phenomena here. First, putting up an umbrella is a behaviour to control the perception of staying dry.
But, given it only takes a second to put up an umbrella, you wouldn't do it until the rain started anyway.
Maybe a better example is whether you would put up a canopy at you garden party if rain was predicted.
Now, the actual putting up of the canopy is all controlled perception and the question is rather whether one takes the prediction of the perception of rain as the perception of rain.
So, this one seems to be an imagination mode process.
However I am sure that other apparent feed forward examples use a different combination of PCT components....
Warren

Warren, your comment "this seems to be an imagination mode process",

together with Rick’s having induced me to reread Bill’s description
of the Artificial Cerebellum, has clarified for me something that
has been bugging me for many years, especially recently.

At CSG93, I discussed with Bill this configuration of the

imagination connection, which I had used in a presentation earlier
in the year – I quote a page from that presentation, which is on my
Web site at http://www.mmtaylor.net/PCT/DFS93/DFS93_1.html.

------------quote---------

Basic elements of an Elementary Control System (ECS) showing an

“imagination loop”

  The simple ECS is not complete in the form shown earlier. It is

capable of “imagining.” In PCT, imagination is conceived as
permitting the output to be connected directly back to the input.
Since the output usually serves as a reference for lower-level
perceptions that “should be” controlled to match their references,
feeding back the output is equivalent to receiving those
perceptions. It is the difference between a commander ordering
“Take that hill” and the commander imagining “What would it be
like if that hill were to be taken.”

  The ECS also can be augmented by the addition of a memory, which

allows it to recapture a perception previously obtained, either in
the form of a reference signal (which creates an error signal that
leads to output) or as a perceptual signal perhaps together with a
reference signal, leading to a perception but no error and thus no
output.

Imagining: Planning

  The imagination is not necessarily a direct connection between

the output and the perceptual input function, as shown in the
previous slide. It is likely to go through some kind of a world
model, particularly at the higher control levels where the
perceptual input function may be logical rather than analogue in
form. The world model may well be implemented in parts of the
complicated control network that are not currently being used for
active control (and which might be specialized for the imagination
function). It allows the output signal to have effects that are
distributed over time, rather than being concentrated at the time
the output signal is issued. The world model is built up through
experience with the world, as well as by training. Testing out the
likely effects of output by imagining through the world model is
“planning.”

---

Over the years, it has not been clear to me how the World Model

might be “built up through experience”. I have attributed part of it
to Bill’s context-specific content-addressable associative memory
that sets reference values for the next lower level, and part of it
to the reorganization of the levels below, which, when used in their
own imagination mode, should return perceptions that might be like
those that would occur if they had acted in the real world. But
neither of these, even at the sequence level of perception, which
gets close, satisfactorily seem to deal with the temporal
distribution of effects in the real world that should map onto
temporal effects in the World Model. The Artificial Cerebellum
structure could fill the gap.

As described by Bill, the Artificial Cerebellum allows the output of

a control system to be timed so as to compensate for the spectral
distribution of loop gain. In information-theoretic terms, this is
called “pre-whitening”, and optimizes the information-theoretic
efficiency of the channel (in this case the loop) in which it is
used. Bill’s example is control in an environment of a damped
mass-loaded spring. But it also works when it is not the
environmental feedback path but the disturbance that has a defined
spectrum.

The basic component of the Artificial Cerebellum (as of a

sequence-level perceptual function) is a shift-register. Bill’s
Figure 2 (p44) shows it used to create a World Model of some
arbitrary “Real System”. Since the signal values are all just scalar
numbers, the Real System is specified only by the waveform that
results when it is provided with an impulse as input. What comes out
is not necessarily an impulse (though it might be), but a waveform
extended over time. Bill trains the system by using as input not an
impulse (which has a flat spectrum but is unrealizable in practice)
but a white noise (which also has a flat spectrum, and can be
treated as a series of randomly sized impulses).

Since Bill called the technique an "Artificial Cerebellum", it is

reasonable to assume that he saw it as modelling the action of the
biological cerebellum. But in his demo, he used it as part or all of
the output function in each level of a two-level control system. It
is a general technique for dealing with time-based regularities. It
can implement an integrator, for example, by simply giving equal
weight to every sample along the shift register, or a leaky
integrator by giving declining weights along the length of the
register. In other words, it is a trainable linear filter. You can
read about linear filters in Wikipedia
http://en.wikipedia.org/wiki/Linear_filter.

Implemented as shown in the lower diagram of the extract from my

1993 presentation, the World Model could be an Artificial
Cerebellum, trained by comparing its output to the actual
perceptions coming from the “Real World” over the range of control
system outputs to lower levels (deleting the switch shown in the
figure, of course). Because of this generality, it might be better
not to identify it with the cerebellum, but simply to call it a
trainable finite impulse response filter (T-FIR).

Since the "Real World" environmental feedback function is likely to

be more consistent at lower that at high levels in the perceptual
hierarchy, the place of the World Model T-FIR is likely to be more
prominent at the lower levels, as compared to the context-specific
associative memory function, which is unlikely to be much use at low
levels because of the ubiquity of the contexts for low-level
perceptual control. Since reorganization presumably continues at all
levels, but is more prominent at the higher levels in a mature
system, the Artificial Cerebellum seems to fit very well what has
seemed to me to be a time-binding" gap in the imagination structure
of the hierarchy.

Martin

–
Dr Warren Mansell
Reader in Psychology
Cognitive Behavioural Therapist & Chartered Clinical Psychologist
School of Psychological Sciences

Coupland I
University of Manchester
Oxford Road
Manchester M13 9PL
Email: warren.mansell@manchester.ac.uk

Tel: +44 (0) 161 275 8589

Website: http://www.psych-sci.manchester.ac.uk/staff/131406

See teamstrial.net for further information on our trial of CBT for Bipolar Disorders in NW England

The highly acclaimed therapy manual on A Transdiagnostic Approach to CBT using Method of Levels is available now.

Check www.pctweb.org for further information on Perceptual Control Theory

[MartinT]

[Martin Taylor 2014.02.04.12.18]

Hi Martin, that's a really exciting idea! I had thought that the cerebellum must have a very essential function in control but I wasn't sure exactly what.

Although there is a bit of me that says that this function within imagination might not be sufficiently fundamental given the pervasiveness of the cerebellum to all animals with a CNS. Could it not be involved more directly in timing adjustments, whatever their use in the hierarchy?

Yes, for sure. That's what Bill intended, to judge from his paper and his naming of the device "Artificial Cerebellum". It was Rick's comment that the Artificial Cerebellum did not deal with temporal effects, and a long-ago comment from Bill that it only dealt with the way the environmental feedback path works, and not the spectrum of the disturbance, that made me put two and two together to see that the same mechanism could be used elsewhere as well.

I can see that when we imagine we do also often imagine the timecourse of disturbances to our reference values, not just our reference values themselves, and that if we imagined without the inclusion of disturbances, we would not be accurately modelling an 'as if' mental simulation of a problem. We would just solve it.

However, with my clinical experience, and my need for parsimony, I am also thinking that if two control loops in conflict at the same level enter the imagination mode, then one would perceive a disturbance.

Not necessarily. We often imagine that we can achieve two results at the same time, but when we try it in the real environment, we find that either the procedures or the controlled perceptions are in conflict. For example, lots of voters think that we can have low taxes and a good life, by imagining both at the same time.

Also, some of the most important disturbances (and feedback functions) in the world are other people. So how would a conflicting goal within one's own hierarchy that happened to conflict with the planned goal serve as a modelled disturbance?

The Trained Finite Impulse Response (T-FIR) filter doesn't model disturbances as such. It models only their spectra-- regularities over time. What you are talking about seems to be more closely associated with perceptual context, and thus the associative memory that we are assuming is the way reference values get set. That has nothing to do with regularities other than the regularity of finding in one perceptual context a metaphor for another in which specific reference values sent to lower level control units had resulted in error-reducing changes in one or more controlled perceptions.

There is only one sandwich on the table - I want it but I can easily infer from this that the other person in the room wants it to, so I run both of these control systems together - opposing one another. I can see this might be a tough one mathematically, but I do think we are much more accurate if we model the 'world' as another control system rather than through other means, don't you think?

Only if the perceived world has another control system in it. If you don't see that someone else might want the sandwich, there's no reason to imagine a conflict. The associative memory would not include in its addressing vector the perception of another person or a possibly opposing control system.

Having said that I can think of various physical obstacles to progress it is important to get one's head around too!

So, I am definitely with dissecting the imagination mode and looking to understand how it models experience over time and that experience with the environment will form the foundations for that, just not sure exactly the solution yet...

Nor am I. All this is just speculation using mechanisms that are known to work. As Rick has pointed out, there's no evidence that any of it exists in a biological organism. Nor, for the most part, does it exist in an implemented simulation. Next week, my speculation might be different. But I am encouraged that Bill did eventually invent for himself the idea of a World Model.

Martin

···

On 2014/02/4 11:28 AM, Warren Mansell wrote:

[wmansell]

Hi Martin,

Forgive my last message. Just because behaviour is the control of perception doesn’t mean that all perception is controlled. I can watch a plane fly overhead and have no control over where that plane is headed to. Bill recognised within the passive observation mode that we take in perceptions without controlling them. We must have a way of storing and imagining perceptions that are not an integral part of our own control systems as well as ones that are. Is that the perspective that filling in the ‘real world model’ provides for you? Rick, what do you think?

···

On 2014/02/4 2:20 AM, Warren Mansell
wrote:

I think that the cloud and umbrella example really illustrates that we are talking about multi-componential phenomena here. First, putting up an umbrella is a behaviour to control the perception of staying dry.
But, given it only takes a second to put up an umbrella, you wouldn't do it until the rain started anyway.
Maybe a better example is whether you would put up a canopy at you garden party if rain was predicted.
Now, the actual putting up of the canopy is all controlled perception and the question is rather whether one takes the prediction of the perception of rain as the perception of rain.
So, this one seems to be an imagination mode process.
However I am sure that other apparent feed forward examples use a different combination of PCT components....
Warren

http://www.mmtaylor.net/PCT/DFS93/DFS93_1.html

Basic elements of an Elementary Control System (ECS) showing an

“imagination loop”
<p4.ECU.jpg>

  The simple ECS is not complete in the form shown earlier. It is

capable of “imagining.” In PCT, imagination is conceived as
permitting the output to be connected directly back to the input.
Since the output usually serves as a reference for lower-level
perceptions that “should be” controlled to match their references,
feeding back the output is equivalent to receiving those
perceptions. It is the difference between a commander ordering
“Take that hill” and the commander imagining “What would it be
like if that hill were to be taken.”

  The ECS also can be augmented by the addition of a memory, which

allows it to recapture a perception previously obtained, either in
the form of a reference signal (which creates an error signal that
leads to output) or as a perceptual signal perhaps together with a
reference signal, leading to a perception but no error and thus no
output.

Imagining: Planning

<p4.Planning.ECU.jpg>

  The imagination is not necessarily a direct connection between

the output and the perceptual input function, as shown in the
previous slide. It is likely to go through some kind of a world
model, particularly at the higher control levels where the
perceptual input function may be logical rather than analogue in
form. The world model may well be implemented in parts of the
complicated control network that are not currently being used for
active control (and which might be specialized for the imagination
function). It allows the output signal to have effects that are
distributed over time, rather than being concentrated at the time
the output signal is issued. The world model is built up through
experience with the world, as well as by training. Testing out the
likely effects of output by imagining through the world model is
“planning.”

http://en.wikipedia.org/wiki/Linear_filter

[wmansell]

Thank you Martin, that makes sense and I hope my last message chimes with you.
Warren

···

Sent from my iPhone

On 4 Feb 2014, at 17:36, Martin Taylor <mmt-csg@MMTAYLOR.NET> wrote:

[Martin Taylor 2014.02.04.12.18]

On 2014/02/4 11:28 AM, Warren Mansell wrote:
Hi Martin, that's a really exciting idea! I had thought that the cerebellum must have a very essential function in control but I wasn't sure exactly what.

Although there is a bit of me that says that this function within imagination might not be sufficiently fundamental given the pervasiveness of the cerebellum to all animals with a CNS. Could it not be involved more directly in timing adjustments, whatever their use in the hierarchy?

Yes, for sure. That's what Bill intended, to judge from his paper and his naming of the device "Artificial Cerebellum". It was Rick's comment that the Artificial Cerebellum did not deal with temporal effects, and a long-ago comment from Bill that it only dealt with the way the environmental feedback path works, and not the spectrum of the disturbance, that made me put two and two together to see that the same mechanism could be used elsewhere as well.

I can see that when we imagine we do also often imagine the timecourse of disturbances to our reference values, not just our reference values themselves, and that if we imagined without the inclusion of disturbances, we would not be accurately modelling an 'as if' mental simulation of a problem. We would just solve it.

However, with my clinical experience, and my need for parsimony, I am also thinking that if two control loops in conflict at the same level enter the imagination mode, then one would perceive a disturbance.

Not necessarily. We often imagine that we can achieve two results at the same time, but when we try it in the real environment, we find that either the procedures or the controlled perceptions are in conflict. For example, lots of voters think that we can have low taxes and a good life, by imagining both at the same time.

Also, some of the most important disturbances (and feedback functions) in the world are other people. So how would a conflicting goal within one's own hierarchy that happened to conflict with the planned goal serve as a modelled disturbance?

The Trained Finite Impulse Response (T-FIR) filter doesn't model disturbances as such. It models only their spectra-- regularities over time. What you are talking about seems to be more closely associated with perceptual context, and thus the associative memory that we are assuming is the way reference values get set. That has nothing to do with regularities other than the regularity of finding in one perceptual context a metaphor for another in which specific reference values sent to lower level control units had resulted in error-reducing changes in one or more controlled perceptions.

There is only one sandwich on the table - I want it but I can easily infer from this that the other person in the room wants it to, so I run both of these control systems together - opposing one another. I can see this might be a tough one mathematically, but I do think we are much more accurate if we model the 'world' as another control system rather than through other means, don't you think?

Only if the perceived world has another control system in it. If you don't see that someone else might want the sandwich, there's no reason to imagine a conflict. The associative memory would not include in its addressing vector the perception of another person or a possibly opposing control system.

Having said that I can think of various physical obstacles to progress it is important to get one's head around too!

So, I am definitely with dissecting the imagination mode and looking to understand how it models experience over time and that experience with the environment will form the foundations for that, just not sure exactly the solution yet...

Nor am I. All this is just speculation using mechanisms that are known to work. As Rick has pointed out, there's no evidence that any of it exists in a biological organism. Nor, for the most part, does it exist in an implemented simulation. Next week, my speculation might be different. But I am encouraged that Bill did eventually invent for himself the idea of a World Model.

Martin

[rsmarken]

[From Rick Marken (2014.02.04.1000)]

W: Hi Rick and Martin, I don't know whether this will fit with both of you or either of you, but this isthe way I hope to use the illuminations you are providing. I am taking Martin's descriptions of feed forward as observable data that the majority of (non-PCT) psychologist will take to be examples of feed forward and considering the various forms that they take, including the apparent predictive information that is contained in the data. I am then assuming that behaviour is the control of perception and assuming that Bill's description of PCT as it stands will explain the if we find the right component (e.g. hierachies, reorganisation, memory) in the right context. They generally seem to. But then I will consider if any tweaks seem necessary, given that behaviour is the control of perception....

RM: Here's what fits with my point of view:

There is no such thing as feedforward. All cases of apparent
feedforward are simply actions protecting a CV from disturbance.
Whenever we see a response to a stimulus in a living organism we know
from PCT that there is a CV involved. Living organisms control
perceptions, they don't response to stimuli. So a response to a CS in
classical conditioning, getting an umbrella when the sky gets dark,
taking a different route when google maps says there is traffic in the
way, etc; all are simply negative feedback control of perceptions; we
just don't know what perceptions are being controlled. We can guess
but finding out what perceptions are controlled is what PCT research
is all about.

This all started, I believe, with your mention of people complaining
that negative feedback is too simple to explain behavior. What they
thought was needed was feedforward too. That's because these are S-R
psychologists who don't know that apparent feedforward phenomena are
explained by negative feedback control. Feedforward is just a version
of the behavioral illusion; the illusion being that a stimulus (like
the CS, dark sky or traffic message) is a stimulus for a response that
anticipates a future disturbance.

So I think the answer to those who say feedforward is needed to
supplement negative feedback as an explanation of behavior is "no it
isn't". Apparent feedforward is explained by the negative feedback
control model of PCT. There is no need to supplement PCT with S-R
theory; PCT already explains why behavior sometimes "looks like" S-R.

Best

Rick

···

On Mon, Feb 3, 2014 at 11:01 PM, Warren Mansell <wmansell@gmail.com> wrote:

--
Richard S. Marken PhD
www.mindreadings.com

The only thing that will redeem mankind is cooperation.
                                                   -- Bertrand Russell

[wmansell]

Hi Rick, I think this means you are agreeing with me as I see no need to invoke S-R theory to explain phenomena that people say we need feedforward to explain either. Indeed, I was pointing to the functional architecture of how perception is controlled by behaviour within PCT to explain them.
Great!
Warren

···

Sent from my iPhone

On 4 Feb 2014, at 17:56, Richard Marken <rsmarken@GMAIL.COM> wrote:

[From Rick Marken (2014.02.04.1000)]

On Mon, Feb 3, 2014 at 11:01 PM, Warren Mansell <wmansell@gmail.com> wrote:
W: Hi Rick and Martin, I don't know whether this will fit with both of you or either of you, but this isthe way I hope to use the illuminations you are providing. I am taking Martin's descriptions of feed forward as observable data that the majority of (non-PCT) psychologist will take to be examples of feed forward and considering the various forms that they take, including the apparent predictive information that is contained in the data. I am then assuming that behaviour is the control of perception and assuming that Bill's description of PCT as it stands will explain the if we find the right component (e.g. hierachies, reorganisation, memory) in the right context. They generally seem to. But then I will consider if any tweaks seem necessary, given that behaviour is the control of perception....

RM: Here's what fits with my point of view:

There is no such thing as feedforward. All cases of apparent
feedforward are simply actions protecting a CV from disturbance.
Whenever we see a response to a stimulus in a living organism we know
from PCT that there is a CV involved. Living organisms control
perceptions, they don't response to stimuli. So a response to a CS in
classical conditioning, getting an umbrella when the sky gets dark,
taking a different route when google maps says there is traffic in the
way, etc; all are simply negative feedback control of perceptions; we
just don't know what perceptions are being controlled. We can guess
but finding out what perceptions are controlled is what PCT research
is all about.

This all started, I believe, with your mention of people complaining
that negative feedback is too simple to explain behavior. What they
thought was needed was feedforward too. That's because these are S-R
psychologists who don't know that apparent feedforward phenomena are
explained by negative feedback control. Feedforward is just a version
of the behavioral illusion; the illusion being that a stimulus (like
the CS, dark sky or traffic message) is a stimulus for a response that
anticipates a future disturbance.

So I think the answer to those who say feedforward is needed to
supplement negative feedback as an explanation of behavior is "no it
isn't". Apparent feedforward is explained by the negative feedback
control model of PCT. There is no need to supplement PCT with S-R
theory; PCT already explains why behavior sometimes "looks like" S-R.

Best

Rick

--
Richard S. Marken PhD
www.mindreadings.com

The only thing that will redeem mankind is cooperation.
                                                  -- Bertrand Russell

[MartinT]

[Martin Taylor 2014.02.04.13.26]

[From Rick Marken (2014.02.04.1000)]

W: Hi Rick and Martin, I don't know whether this will fit with both of you or either of you, but this isthe way I hope to use the illuminations you are providing. I am taking Martin's descriptions of feed forward as observable data that the majority of (non-PCT) psychologist will take to be examples of feed forward and considering the various forms that they take, including the apparent predictive information that is contained in the data. I am then assuming that behaviour is the control of perception and assuming that Bill's description of PCT as it stands will explain the if we find the right component (e.g. hierachies, reorganisation, memory) in the right context. They generally seem to. But then I will consider if any tweaks seem necessary, given that behaviour is the control of perception....
RM: Here's what fits with my point of view:

There is no such thing as feedforward. All cases of apparent
feedforward are simply actions protecting a CV from disturbance.

What do you mean by "feedforward" today?

Whenever we see a response to a stimulus in a living organism we know
from PCT that there is a CV involved.

I do not consider myself as part of that "we", since I like to consider myself to be a scientist, and scientists do not "know" the truth about the world they investigate. PCT is a theory that has a premise. If the premise is true, it _defines_ the truth of what you say. But that's as close as we can come to "knowing" it. PCT is not a faith like Islam. The holy writ can be modified by observation of the world.

And in any case how does this comment relate to "There is no such thing as feedforward"? In the last day or two, you seem to have been slipping away from the meaning we had been using in this thread, that "feedforward" meant "uses prediction". Since there obviously IS such a thing as prediction, you can't be meaning that any more. So what do you mean today? And do you expect to mean the same thing tomorrow?

By the way, in case you don't believe there is such a thing as prediction, I make one here and now: In 2020, Barack Obama will not be the President of the USA. Here's another. Around midnight, the sky will be darker than it is right now (around 13.30).

  Living organisms control
perceptions, they don't response to stimuli.

So we believe until a counter-example is provably demonstrated.

  So a response to a CS in
classical conditioning, getting an umbrella when the sky gets dark,
taking a different route when google maps says there is traffic in the
way, etc; all are simply negative feedback control of perceptions; we
just don't know what perceptions are being controlled.

So we agree.

  We can guess
but finding out what perceptions are controlled is what PCT research
is all about.

This all started, I believe, with your mention of people complaining
that negative feedback is too simple to explain behavior.

Actually, I think that was Philip

  What they
thought was needed was feedforward too. That's because these are S-R
psychologists who don't know that apparent feedforward phenomena are
explained by negative feedback control. Feedforward is just a version
of the behavioral illusion; the illusion being that a stimulus (like
the CS, dark sky or traffic message) is a stimulus for a response that
anticipates a future disturbance.

This certainly makes it clear that you have quite drastically changed what you mean by "feedforward", but it doesn't help in understanding what you now mean by the word. What, for example, would be an example of "apparent feedforward phenomena"?

So I think the answer to those who say feedforward is needed to
supplement negative feedback as an explanation of behavior is "no it
isn't". Apparent feedforward is explained by the negative feedback
control model of PCT. There is no need to supplement PCT with S-R
theory; PCT already explains why behavior sometimes "looks like" S-R.

So it does. But who on CSGnet has ever suggested supplementing PCT with S-R theory? That's not a straw man argument, it's a snowman on the equator, and an insult to the general readership of CSGnet.

Martin

···

On Mon, Feb 3, 2014 at 11:01 PM, Warren Mansell <wmansell@gmail.com> wrote:

[wmansell]

So Martin, are you saying that this part of our brain sits at the back and perceives copies of the signals we send to muscles and it tries (through hierarchies & reorganisation??) to synchronously match it’s outputs to the inputs the rest of the brain gets from the environment? I like it!

Warren

···

On 2014/02/4 2:20 AM, Warren Mansell
wrote:

I think that the cloud and umbrella example really illustrates that we are talking about multi-componential phenomena here. First, putting up an umbrella is a behaviour to control the perception of staying dry.
But, given it only takes a second to put up an umbrella, you wouldn't do it until the rain started anyway.
Maybe a better example is whether you would put up a canopy at you garden party if rain was predicted.
Now, the actual putting up of the canopy is all controlled perception and the question is rather whether one takes the prediction of the perception of rain as the perception of rain.
So, this one seems to be an imagination mode process.
However I am sure that other apparent feed forward examples use a different combination of PCT components....
Warren

http://www.mmtaylor.net/PCT/DFS93/DFS93_1.html

Basic elements of an Elementary Control System (ECS) showing an

“imagination loop”
<p4.ECU.jpg>

  The simple ECS is not complete in the form shown earlier. It is

capable of “imagining.” In PCT, imagination is conceived as
permitting the output to be connected directly back to the input.
Since the output usually serves as a reference for lower-level
perceptions that “should be” controlled to match their references,
feeding back the output is equivalent to receiving those
perceptions. It is the difference between a commander ordering
“Take that hill” and the commander imagining “What would it be
like if that hill were to be taken.”

  The ECS also can be augmented by the addition of a memory, which

allows it to recapture a perception previously obtained, either in
the form of a reference signal (which creates an error signal that
leads to output) or as a perceptual signal perhaps together with a
reference signal, leading to a perception but no error and thus no
output.

Imagining: Planning

<p4.Planning.ECU.jpg>

  The imagination is not necessarily a direct connection between

the output and the perceptual input function, as shown in the
previous slide. It is likely to go through some kind of a world
model, particularly at the higher control levels where the
perceptual input function may be logical rather than analogue in
form. The world model may well be implemented in parts of the
complicated control network that are not currently being used for
active control (and which might be specialized for the imagination
function). It allows the output signal to have effects that are
distributed over time, rather than being concentrated at the time
the output signal is issued. The world model is built up through
experience with the world, as well as by training. Testing out the
likely effects of output by imagining through the world model is
“planning.”

http://en.wikipedia.org/wiki/Linear_filter

[MartinT]

[Martin Taylor 2014.02.04.14.29]

Warren,

You may like it, and you may be right to do so, but it was not my
idea. I’ll have to think a bit more about what you say. As I
understand it, in what we may call “cerebellar mode” the adaptation
of the FIR filter mechanism depends on the ability of the whole
control loop to control. The error signal drives the adaptation of
the filter.
In “Cerebellar mode”, Bill’s description substitutes a T-FIR filter
for the leaky integrator as the output function of each control
unit. The contribution of each value of delay is continuously being
updated. The updating isn’t shown in this diagram (Fig 4 of his
paper).
The boundary between organism and environment is at the level of the
three vertical arrows. Below the arrows is the simulated mass on a
spring. Above them is the two-level control structure.
I suppose those A.C. boxes might reside in the cerebellum, and
because of the name of the mechanism I assume Bill thought they did,
but I suspect they would be more likely to be found in the motor
cortex. However, I defer to anyone with a smattering of knowledge of
brain anatomy and function.
Martin

···

On 2014/02/4 2:16 PM, Warren Mansell
wrote:

    So Martin, are you saying that this part of our brain sits at

the back and perceives copies of the signals we send to muscles
and it tries (through hierarchies & reorganisation??) to
synchronously match it’s outputs to the inputs the rest of the
brain gets from the environment? I like it!

Warren

    Sent from my iPhone

    On 4 Feb 2014, at 15:26, Martin Taylor <mmt-csg@MMTAYLOR.NET        >

wrote:

[Martin Taylor 2014.02.04.07.59]

      Warren, your comment "this seems to be an imagination mode

process", together with Rick’s having induced me to reread
Bill’s description of the Artificial Cerebellum, has clarified
for me something that has been bugging me for many years,
especially recently.
At CSG93, I discussed with Bill this configuration of the
imagination connection, which I had used in a presentation
earlier in the year – I quote a page from that presentation,
which is on my Web site at .
------------quote---------

Over the years, it has not been clear to me how the World
Model might be “built up through experience”. I have
attributed part of it to Bill’s context-specific
content-addressable associative memory that sets reference
values for the next lower level, and part of it to the
reorganization of the levels below, which, when used in their
own imagination mode, should return perceptions that might be
like those that would occur if they had acted in the real
world. But neither of these, even at the sequence level of
perception, which gets close, satisfactorily seem to deal with
the temporal distribution of effects in the real world that
should map onto temporal effects in the World Model. The
Artificial Cerebellum structure could fill the gap.
As described by Bill, the Artificial Cerebellum allows the
output of a control system to be timed so as to compensate for
the spectral distribution of loop gain. In
information-theoretic terms, this is called “pre-whitening”,
and optimizes the information-theoretic efficiency of the
channel (in this case the loop) in which it is used. Bill’s
example is control in an environment of a damped mass-loaded
spring. But it also works when it is not the environmental
feedback path but the disturbance that has a defined spectrum.
The basic component of the Artificial Cerebellum (as of a
sequence-level perceptual function) is a shift-register.
Bill’s Figure 2 (p44) shows it used to create a World Model of
some arbitrary “Real System”. Since the signal values are all
just scalar numbers, the Real System is specified only by the
waveform that results when it is provided with an impulse as
input. What comes out is not necessarily an impulse (though it
might be), but a waveform extended over time. Bill trains the
system by using as input not an impulse (which has a flat
spectrum but is unrealizable in practice) but a white noise
(which also has a flat spectrum, and can be treated as a
series of randomly sized impulses).
Since Bill called the technique an “Artificial Cerebellum”, it
is reasonable to assume that he saw it as modelling the action
of the biological cerebellum. But in his demo, he used it as
part or all of the output function in each level of a
two-level control system. It is a general technique for
dealing with time-based regularities. It can implement an
integrator, for example, by simply giving equal weight to
every sample along the shift register, or a leaky integrator
by giving declining weights along the length of the register.
In other words, it is a trainable linear filter. You can read
about linear filters in Wikipedia .
Implemented as shown in the lower diagram of the extract from
my 1993 presentation, the World Model could be an Artificial
Cerebellum, trained by comparing its output to the actual
perceptions coming from the “Real World” over the range of
control system outputs to lower levels (deleting the switch
shown in the figure, of course). Because of this generality,
it might be better not to identify it with the cerebellum, but
simply to call it a trainable finite impulse response filter
(T-FIR).
Since the “Real World” environmental feedback function is
likely to be more consistent at lower that at high levels in
the perceptual hierarchy, the place of the World Model T-FIR
is likely to be more prominent at the lower levels, as
compared to the context-specific associative memory function,
which is unlikely to be much use at low levels because of the
ubiquity of the contexts for low-level perceptual control.
Since reorganization presumably continues at all levels, but
is more prominent at the higher levels in a mature system, the
Artificial Cerebellum seems to fit very well what has seemed
to me to be a time-binding" gap in the imagination structure
of the hierarchy.
Martin

        On 2014/02/4 2:20 AM, Warren

Mansell wrote:

I think that the cloud and umbrella example really illustrates that we are talking about multi-componential phenomena here. First, putting up an umbrella is a behaviour to control the perception of staying dry.
But, given it only takes a second to put up an umbrella, you wouldn't do it until the rain started anyway.
Maybe a better example is whether you would put up a canopy at you garden party if rain was predicted.
Now, the actual putting up of the canopy is all controlled perception and the question is rather whether one takes the prediction of the perception of rain as the perception of rain.
So, this one seems to be an imagination mode process.
However I am sure that other apparent feed forward examples use a different combination of PCT components....
Warren

http://www.mmtaylor.net/PCT/DFS93/DFS93_1.html

Basic elements of an Elementary Control System (ECS)

showing an “imagination loop”
<p4.ECU.jpg>

        The simple ECS is not complete in the form shown earlier.

It is capable of “imagining.” In PCT, imagination is
conceived as permitting the output to be connected directly
back to the input. Since the output usually serves as a
reference for lower-level perceptions that “should be”
controlled to match their references, feeding back the
output is equivalent to receiving those perceptions. It is
the difference between a commander ordering “Take that hill”
and the commander imagining “What would it be like if that
hill were to be taken.”

        The ECS also can be augmented by the addition of a memory,

which allows it to recapture a perception previously
obtained, either in the form of a reference signal (which
creates an error signal that leads to output) or as a
perceptual signal perhaps together with a reference signal,
leading to a perception but no error and thus no output.

Imagining: Planning

<p4.Planning.ECU.jpg>

        The imagination is not necessarily a direct connection

between the output and the perceptual input function, as
shown in the previous slide. It is likely to go through some
kind of a world model, particularly at the higher control
levels where the perceptual input function may be logical
rather than analogue in form. The world model may well be
implemented in parts of the complicated control network that
are not currently being used for active control (and which
might be specialized for the imagination function). It
allows the output signal to have effects that are
distributed over time, rather than being concentrated at the
time the output signal is issued. The world model is built
up through experience with the world, as well as by
training. Testing out the likely effects of output by
imagining through the world model is “planning.”

http://en.wikipedia.org/wiki/Linear_filter

[rsmarken]

[From Rick Marken (2014.02.04.1315)]

···

Martin Taylor (2014.02.04.14.29) –

In "Cerebellar mode", Bill's description substitutes a T-FIR filter

for the leaky integrator as the output function of each control
unit. The contribution of each value of delay is continuously being
updated. The updating isn’t shown in this diagram (Fig 4 of his
paper).

RM: The AC model was presented back in 1994 at the CSG conference in Wales. It’s a clever model; it defines the output functions as impulse response functions. The AC varies the weights of this function based on the difference between the actual error signal value and the desired error signal value (0) at different time delays from the present. There is no prediction involved; just adjustment of the transfer function weights based on the past (and the current) value of the error signal.

But I think that the AC model is now obsolete (it was pretty physiologically implausible also). Bill discovered that the kind of adaption that was the reason for inventing the AC model could be handled by a simple two-level control model that looks basically like the one shown above but without the AC components in the output. This “non-adaptive adaptive” model is described n Ch. 5 of LCS III and will be discussed in class when we get there.

Best

Rick

–
Richard S. Marken PhD
www.mindreadings.com
The only thing that will redeem mankind is cooperation.

                                               -- Bertrand Russell

[rsmarken]

[From Rick Marken (2014.02.02.1345)]

Martin Taylor (2014.02.04.13.26)--

MT: What do you mean by "feedforward" today?

RM: A situation where a stimulus (or the perception thereof) appears
to cause a response that preemptively compensates for a disturbance to
a controlled variable.

RM: Whenever we see a response to a stimulus in a living organism we
know from PCT that there is a CV involved.

MT: I do not consider myself as part of that "we"

RM: I know. But some of us know it because we know that we live in a
closed feedback loop; what we do depends on what we sense and, at the
same time, what we sense depends on what we do. And we know the sign
of the feedback in this loop is negative because behavior (control) is
generally quite stable. So what we do cannot be a response to
stimulation because the result of that stimulation is sensory input
that is being affected by what we are doing at the same time. So
whenever we see a regular relationship between stimulus and response
in a living system we can be sure that it is the result of the fact
that the stimulus is a disturbance to some sensory input (or,
actually, the perception thereof) that is under control because we
know (from PCT) that that's the way closed -loop negative feedback
systems work.

MT: And in any case how does this comment relate to "There is no such
thing as feedforward"? In the last day or two, you seem to have been
slipping away from the meaning we had been using in this thread, that
"feedforward" meant "uses prediction". Since there obviously IS such a
thing as prediction, you can't be meaning that any more. So what do you
mean today? And do you expect to mean the same thing tomorrow?

RM: See above. To me feedforward = S-R. Prediction is a whole
different thing. It's an imagined future state of affairs. We do that
a lot. But that's just control of imagination. I don't consider adding
the derivative of the perceptual signal in with the perceptuall signal
a prediction; if you do, then we have a communication problem. But
anyway, I re-wrote your model with the delay now between output and
CV and it comes out the same way; control of s(t)+k*d(st)/dt is
always better than control of s(t) alone when k>0. Is that what you
get?

MT: This all started, I believe, with your mention of people complaining
that negative feedback is too simple to explain behavior.

RM: Actually, I think that was Philip

RM: I believe it was Warren. He's the one who proposed writing the
paper on feedforward to show how it could be accounted for in terms of
negative feedback. I think? Is that right Warren?

MT: What, for example, would be an example of "apparent
feedforward phenomena"?

RM: A light stimulus classically conditioned to "elicit" salivation
prior to food in the mouth. Predictive landing control displays (very
similar to Bruce's demonstration of feedforward control).

Best

Rick

···

--
Richard S. Marken PhD
www.mindreadings.com

The only thing that will redeem mankind is cooperation.
                                                   -- Bertrand Russell

[MartinT]

[Martin Taylor 2014.02.04.17.26]

[From Rick Marken (2014.02.02.1345)]

Martin Taylor (2014.02.04.13.26)--
MT: What do you mean by "feedforward" today?

RM: A situation where a stimulus (or the perception thereof) appears
to cause a response that preemptively compensates for a disturbance to
a controlled variable.

OK. I'll call that the 2014.02.04 definition. But I'm not sure what

it means. It seems to describe a situation from the viewpoint of a
naive outside observer (“a stimulus appears to cause a response”)
with what hitherto has been a question about mechanism as described
by an analyst who models a control system. How can we reconcile
this? Other than by scrapping that attempt at a definition on the
grounds of internal inconsistency, I don’t know.

For what it's worth, let me try a definition that sticks to one

viewpoint: “To the analyst, an S-R process is one in which a given
input (labelled “stimulus”) will always produce the same
well-defined result (labelled “response”), regardless of context.”

Here's another definition, from a different viewpoint: "To a casual

observer, an observed pattern that appears to be sensed by an
organism and that is ordinarily followed by a specific observable
behaviour on the part of that organism can be labelled a
“Stimulus-Response” pair and the sequence of sensation followed by
action as “Stimulus-Response behaviour”."

This second definition says absolutely nothing about assumed

mechanisms, and it totally ignores the “Behavioural Illusion”. Fully
fledged idealized control systems with fixed reference values will
behave in that way, even though the “Response” incorporates all the
dynamics of the feedback loop.

I would prefer it if we stuck to the first definition, since it is

too easy to mix up viewpoints as you did, if we use terms in a
variety of different ways.

RM: Whenever we see a response to a stimulus in a living organism we
know from PCT that there is a CV involved.

MT: I do not consider myself as part of that "we"

RM: I know. But some of us know it because we know that we live in a
closed feedback loop; what we do depends on what we sense and, at the
same time, what we sense depends on what we do.

That is the world as we perceive it, I agree. But that's NOT what I

dissociated myself from. As one who tries to practice science rather
than religion, I dissociated myself from KNOWING that PCT is correct
and deciding what should be in the world BECAUSE PCT is correct. I
BELIEVE PCT is correct (though not necessarily in HPCT form), but I
don’t KNOW it to be so. In all of my postings I use the assumption
that it is correct.

MT: And in any case how does this comment relate to "There is no such
thing as feedforward"? In the last day or two, you seem to have been
slipping away from the meaning we had been using in this thread, that
"feedforward" meant "uses prediction". Since there obviously IS such a
thing as prediction, you can't be meaning that any more. So what do you
mean today? And do you expect to mean the same thing tomorrow?

RM: See above. To me feedforward = S-R. Prediction is a whole
different thing.

If I can requote you (quoted recently in
[Martin Taylor 2014.01.30.23.00], your first contribution to the

long “feedforward” thread started with this:“I do not like
predictive control, I do not like it on the whole. I do not like it
in on a roll, I do not like it in a bowl. I do not like it Warren I
am, I do not like green eggs and ham;-)” and finished with "But I
don’t think we’ll sly this predictive control dragon until we have
build a control of perception model that actually does what looks
like predictive control. But until then I’ll just have to reject
predictive (or feedforward) control based on simple little demos
like the one I describe above."

Today you say prediction is a whole different thing from

feedforward, as feedforward is now defined as S-R, and today we do
predict. What will you say tomorrow?

I don't consider adding
the derivative of the perceptual signal in with the perceptuall signal
a prediction; if you do, then we have a communication problem.

We know that without needing this example. But I'm interested in why

you think an estimate of the future value of the perceptual signal
is not a prediction.

But
anyway, I re-wrote your model with the delay now between output and
CV and it comes out the same way; control of s(t)+k*d(st)/dt is
always better than control of s(t) alone when k>0. Is that what you
get?

I haven't tried it. I've been trying to get rid of the bugs in the

Flexible Tracker experiment. Modelling is going to wait until I’ve
got rid of the bugs I know about. But I will try it. As a matter of
interest, how did you compute the derivative this time?

I think I will give up on this miasma of ever-changing meanings and

religious dogmatism, and just get back to science. There are people
on CSGnet who discuss PCT rationally and without continually
shifting definitions to avoid agreement or and who don’t just say
“S-R” when they encounter something they fail to understand or don’t
agree with.

But when you deal with models and their behaviour, I'll be happy to

collaborate.

Martin

[MartinT]

[Martin Taylor 2014.02.04.23.04]

The interesting thing is that except with one particular class of

weight distributions, this truly does embody prediction. Put
crudely, the prediction occurs because if there is any hint of
periodicity in the environmental impulse response, the value at the
next cycle will be related to the values at past cycles, and the
output will be affected.
If it was physiologically implausible (I can’t comment on whether it
was), the same applies to time-binding levels of perception, such as
sequence or event. As for being obsolete, it might have dropped out
of Bill’s thinking, but it was either reasonably right or reasonably
wrong in 1994, and whichever it was then, it is the same now. I
don’t know of any tests that have validated or invalidated it.
One reason why I don’t take this statement at face value is that the
AC mechanism can be adapted to handle a spring mechanism that
oscillates faster than a leaky integrator system could handle, even
if the cycle time of the ringing is fast compared to the transport
lag of the control loop. The ringing of the output can be set to
antiphase with the ringing of the environment. Of course when the
environment is randomly disturbed, the effects of the disturbance
can’t be countered in less than the transport loop delay, but the
loop response can be pre-whitened, which is half the battle.
The AC model is exactly the non-adaptive model (meaning one with a
leaky integrator output function) if it is set with a particular set
of function weights that implement leaky integration. The
integration may be implemented differently, but since it behaves
exactly the same, you can’t tell which implementation, if either,
corresponds to physiological reality, or even whether the same
implementation is used at different levels of control in a given
organism.
Normally, if adaptation is helpful, systems have evolved to allow
it. And whether or not the AC mechanism is used as the output
function of some, none, or all control units, the mechanism is still
useful elsewhere, as I discussed. Of course, this is not to say that
this particular mechanism is the way it is done. It’s an existence
proof that it can be done this way.
Martin

···

[From Rick Marken (2014.02.04.1315)]

Martin Taylor (2014.02.04.14.29) –

              In "Cerebellar mode", Bill's description substitutes a

T-FIR filter for the leaky integrator as the output
function of each control unit. The contribution of
each value of delay is continuously being updated. The
updating isn’t shown in this diagram (Fig 4 of his
paper).

            RM: The AC model was presented back in 1994 at the

CSG conference in Wales. It’s a clever model; it
defines the output functions as impulse response
functions. The AC varies the weights of this function
based on the difference between the actual error signal
value and the desired error signal value (0) at
different time delays from the present. There is no
prediction involved; just adjustment of the transfer
function weights based on the past (and the current)
value of the error signal.

            But I think that the AC model is now obsolete (it was

pretty physiologically implausible also).

            Bill discovered that the kind of adaption that was

the reason for inventing the AC model could be handled
by a simple two-level control model that looks basically
like the one shown above but without the AC components
in the output. This “non-adaptive adaptive” model is
described n Ch. 5 of LCS III and will be discussed in
class when we get there.

[amatic]

[From Adam Matic 2014.02.05.1230. CET]

Modeling what is called “predictive feedforward” might involve some kind of a pattern generator in the feedback loop, controlled by it’s own feedback loop. Bill commented R.K. Moore’s email to CSGnet titled “Evidence for predictive (non-sensory) control” back in 2012. explaining how this might look, here’s full quote:

[Bill Powers (2012.08.20.1006 MDT)]

I thought some members of the list might be interested in a paper that’s

just been published in Current Biology that shows gaze control in tadpoles
during swimming action using fast efferent (feedforward) predictive control,
as opposed to slower visual feedback.

The citation is …

Lambert, F. M., Combes, D., Simmers, J., & Straka, H. (2012). Gaze
stabilization by efference copy signaling without sensory feedback during
vertebrate locomotion. Current Biology.

And the weblink is …

http://www.sciencedirect.com/science/article/pii/S096098221200810X

I have no problem with organism-initiated actions involving feed-forward compensation. The organism can determine when the action is to be initiated and how large and in what direction it is to be. This means that delays may be present, but will not be apparent to an external observer because the compensations can be launched at the same time as or even earlier than the output is generated. Use of a central pattern generator makes this easy, since all the preparatory steps are available to help synchronize the final result. This would be true of swimming motions that disturb the direction of gaze, as in the article. The CPG can be in the output function of a higher-order control system.

The feedforward compensation in the vestibulo-ocular reflex is a little different, but still similar in that head motions are detected rather than predicted from efferent signals and the signal is added to the output to the extraocular muscles to counteract (to some degree) the effect of voluntary or involuntary head motion on the direction of gaze. There is some small delay in this system, but not a lot because the signal pathways are very short. But the most important and suggestive aspect of this reflex is that it has to be continually recalibrated, and that requires feedback information.

When image-stabilization techniques are used to alter the motion of an image on the retina relative to the head motions, the vestibulo-ocular reflex goes out of calibration and is no longer accurate enough to prevent blurring. But after only a few minutes of practice under the new conditions, the timing of the reflex changes until the eye movements are once again in the right relationship to head movements. The only possible basis for this retraining is the visual information, probably simply the blurring that results from miscalibration. I seem to recall that 20 seconds of practice is sufficient for recalibration, but that seems a bit short. The reorganization principle could easily explain how that works, as it does not rely on any information but the average error signal over some period of time.

I think the general rule must be that for every open-loop feedforward compensation, there must be a superordinate negative feedback loop that keeps recalibration accurate. It is not sufficient just to have qualitative feedforward; the feedforward must be timed and sized correctly to reduce the effect of the disturbance and not make it worse. This is especially true for visual effects, since the eye’s resolution is around one minute of arc or 1/60 degree, and any miscalibration greater than that will immediately worsen the quality of vision. Achieving that sort of precision in an open-loop reflex would be impossible without the feedback loop continually recalibrating the reflex.

When we recognize the need for the higher-order recalibration process, the whole problem reduces simply to the normal operation and adjustment of control processes involving more than one level. An example is the model for the behavior of extending a hand in a straight line away from the shoulder. To do this properly, it is necessary for a higher-order system to increase the reference signal for the vertical shoulder angle at one rate and simultaneously decrease the reference for the external angle at the elbow at twice that rate (assuming the upper and lower arm segments to have the same length). This will move the hand straight away from the shoulder if done perfectly, but of course perfection does not last long and reorganization is necessary to maintain good stabilization of the vertical angle of the hand as the reaching is repeated. This compensation can be done either on the perceptual side or the output side of the higher-order system. I have modeled it both ways.

These compensations are transient, and are needed only for a tenth of a second or so, during the short time before negative feedback control can become effective after a step-disturbance. We know that the delay can be compensated by increasing the time constant in the output function, and that after the initial perturbation the control system even with a delay can act very quickly and accurately. See the Live Block Diagram in the demos that go with LCS3.

This feedforward nonsense is just a play on words, motivated, I suspect, by the idea that if there is feed “back” there must also be feed “forward.” In a control system circuit there are signals going both ways, the feedforward signals simply traveling internally from input functions to output functions while the feedback signals go through the external environment to get back to the input function from the output function. There is actually only one direction of signal travel around the loop; there is no part of the circuit in which signals sometime go one way, and sometimes the opposite way. Axons can carry signals either way, but the only time they go backward (“antidromic” conduction) is when an experimenter injects electrical stimulation in the middle of the axon. Synapses work in only one direction.

OK, I found this: http://en.wikipedia.org/wiki/Feedforward_(management). This Wiki claims that Marshall Goldsmith originated this term in an article on management, which is hardly a good recommendation for a technical term.

Here’s a better one for a more technical, but clear, treatment:

http://www.clear.rice.edu/engi128/Handouts/Lec14.pdf

Best,

Bill P.

···

On Wed, Feb 5, 2014 at 5:27 AM, Martin Taylor mmt-csg@mmtaylor.net wrote:

[Martin Taylor 2014.02.04.23.04]

[From Rick Marken (2014.02.04.1315)]

The interesting thing is that except with one particular class of

weight distributions, this truly does embody prediction. Put
crudely, the prediction occurs because if there is any hint of
periodicity in the environmental impulse response, the value at the
next cycle will be related to the values at past cycles, and the
output will be affected.

If it was physiologically implausible (I can't comment on whether it

was), the same applies to time-binding levels of perception, such as
sequence or event. As for being obsolete, it might have dropped out
of Bill’s thinking, but it was either reasonably right or reasonably
wrong in 1994, and whichever it was then, it is the same now. I
don’t know of any tests that have validated or invalidated it.

One reason why I don't take this statement at face value is that the

AC mechanism can be adapted to handle a spring mechanism that
oscillates faster than a leaky integrator system could handle, even
if the cycle time of the ringing is fast compared to the transport
lag of the control loop. The ringing of the output can be set to
antiphase with the ringing of the environment. Of course when the
environment is randomly disturbed, the effects of the disturbance
can’t be countered in less than the transport loop delay, but the
loop response can be pre-whitened, which is half the battle.

The AC model is exactly the non-adaptive model (meaning one with a

leaky integrator output function) if it is set with a particular set
of function weights that implement leaky integration. The
integration may be implemented differently, but since it behaves
exactly the same, you can’t tell which implementation, if either,
corresponds to physiological reality, or even whether the same
implementation is used at different levels of control in a given
organism.

Normally, if adaptation is helpful, systems have evolved to allow

it. And whether or not the AC mechanism is used as the output
function of some, none, or all control units, the mechanism is still
useful elsewhere, as I discussed. Of course, this is not to say that
this particular mechanism is the way it is done. It’s an existence
proof that it can be done this way.

Martin

Martin Taylor (2014.02.04.14.29) –

              In "Cerebellar mode", Bill's description substitutes a

T-FIR filter for the leaky integrator as the output
function of each control unit. The contribution of
each value of delay is continuously being updated. The
updating isn’t shown in this diagram (Fig 4 of his
paper).

            RM: The AC model was presented back in 1994 at the

CSG conference in Wales. It’s a clever model; it
defines the output functions as impulse response
functions. The AC varies the weights of this function
based on the difference between the actual error signal
value and the desired error signal value (0) at
different time delays from the present. There is no
prediction involved; just adjustment of the transfer
function weights based on the past (and the current)
value of the error signal.

            But I think that the AC model is now obsolete (it was

pretty physiologically implausible also).

            Bill discovered that the kind of adaption that was

the reason for inventing the AC model could be handled
by a simple two-level control model that looks basically
like the one shown above but without the AC components
in the output. This “non-adaptive adaptive” model is
described n Ch. 5 of LCS III and will be discussed in
class when we get there.

[wmansell]

Hi Martin, oh that is different from what I was thinking but nonetheless very interesting. I can see how substituting the way that the leaky integrator carries over past signals in a more sensitive way than just reorganising the leakage makes sense… The idea that the cerebellum operates as a model of the physical world that trains itself against how the physical world interacts with the perceiving organism is a little different!

Warren

···

[From Rick Marken (2014.02.04.1315)]

Martin Taylor (2014.02.04.14.29) –

              In "Cerebellar mode", Bill's description substitutes a

T-FIR filter for the leaky integrator as the output
function of each control unit. The contribution of
each value of delay is continuously being updated. The
updating isn’t shown in this diagram (Fig 4 of his
paper).

              <mime-attachment.jpg>

            RM: The AC model was presented back in 1994 at the

CSG conference in Wales. It’s a clever model; it
defines the output functions as impulse response
functions. The AC varies the weights of this function
based on the difference between the actual error signal
value and the desired error signal value (0) at
different time delays from the present. There is no
prediction involved; just adjustment of the transfer
function weights based on the past (and the current)
value of the error signal.

            But I think that the AC model is now obsolete (it was

pretty physiologically implausible also).

            Bill discovered that the kind of adaption that was

the reason for inventing the AC model could be handled
by a simple two-level control model that looks basically
like the one shown above but without the AC components
in the output. This “non-adaptive adaptive” model is
described n Ch. 5 of LCS III and will be discussed in
class when we get there.

[wmansell]

Hi Adam, if the tadpole swam into some thick jelly I imagine it would have to rely entirely on negative feedback again?

···

On Wed, Feb 5, 2014 at 5:27 AM, Martin Taylor mmt-csg@mmtaylor.net wrote:

[Martin Taylor 2014.02.04.23.04]

[From Rick Marken (2014.02.04.1315)]

The interesting thing is that except with one particular class of

weight distributions, this truly does embody prediction. Put
crudely, the prediction occurs because if there is any hint of
periodicity in the environmental impulse response, the value at the
next cycle will be related to the values at past cycles, and the
output will be affected.

If it was physiologically implausible (I can't comment on whether it

was), the same applies to time-binding levels of perception, such as
sequence or event. As for being obsolete, it might have dropped out
of Bill’s thinking, but it was either reasonably right or reasonably
wrong in 1994, and whichever it was then, it is the same now. I
don’t know of any tests that have validated or invalidated it.

One reason why I don't take this statement at face value is that the

AC mechanism can be adapted to handle a spring mechanism that
oscillates faster than a leaky integrator system could handle, even
if the cycle time of the ringing is fast compared to the transport
lag of the control loop. The ringing of the output can be set to
antiphase with the ringing of the environment. Of course when the
environment is randomly disturbed, the effects of the disturbance
can’t be countered in less than the transport loop delay, but the
loop response can be pre-whitened, which is half the battle.

The AC model is exactly the non-adaptive model (meaning one with a

leaky integrator output function) if it is set with a particular set
of function weights that implement leaky integration. The
integration may be implemented differently, but since it behaves
exactly the same, you can’t tell which implementation, if either,
corresponds to physiological reality, or even whether the same
implementation is used at different levels of control in a given
organism.

Normally, if adaptation is helpful, systems have evolved to allow

it. And whether or not the AC mechanism is used as the output
function of some, none, or all control units, the mechanism is still
useful elsewhere, as I discussed. Of course, this is not to say that
this particular mechanism is the way it is done. It’s an existence
proof that it can be done this way.

Martin

Martin Taylor (2014.02.04.14.29) –

              In "Cerebellar mode", Bill's description substitutes a

T-FIR filter for the leaky integrator as the output
function of each control unit. The contribution of
each value of delay is continuously being updated. The
updating isn’t shown in this diagram (Fig 4 of his
paper).

              <mime-attachment.jpg>

            RM: The AC model was presented back in 1994 at the

CSG conference in Wales. It’s a clever model; it
defines the output functions as impulse response
functions. The AC varies the weights of this function
based on the difference between the actual error signal
value and the desired error signal value (0) at
different time delays from the present. There is no
prediction involved; just adjustment of the transfer
function weights based on the past (and the current)
value of the error signal.

            But I think that the AC model is now obsolete (it was

pretty physiologically implausible also).

            Bill discovered that the kind of adaption that was

the reason for inventing the AC model could be handled
by a simple two-level control model that looks basically
like the one shown above but without the AC components
in the output. This “non-adaptive adaptive” model is
described n Ch. 5 of LCS III and will be discussed in
class when we get there.

[amatic]

As I understand it, yes, it would have to calibrate the pattern generator all over again to make the eye movements compensate for body movements correctly. My attempt of translating the expression "calibration of the feedforward pattern generator" to PCT terms would be "reorganization of an output function in the sequence level".
Adam

···

On Wed, Feb 5, 2014 at 5:29 PM, Warren Mansell <<mailto:wmansell@gmail.com>wmansell@gmail.com> wrote:

Hi Adam, if the tadpole swam into some thick jelly I imagine it would have to rely entirely on negative feedback again?

[wmansell]

Yep, Quite possibly!

···

On Wed, Feb 5, 2014 at 5:29 PM, Warren Mansell wmansell@gmail.com wrote:

Hi Adam, if the tadpole swam into some thick jelly I imagine it would have to rely entirely on negative feedback again?

As I understand it, yes, it would have to calibrate the pattern generator all over again to make the eye movements compensate for body movements correctly. My attempt of translating the expression “calibration of the feedforward pattern generator” to PCT terms would be “reorganization of an output function in the sequence level”.

Adam