Feedforward Demo

[Abbott]

[From Bruce Abbott (2014.02.01.1040 EST)]

[Martin Taylor 2014.01.31.23.53] –

[From Bruce Abbott (2014.01.31.1915 EST)]

The recent discussion of feedforward versus feedback control systems, together with Rick Marken’s prompting, led me to create a demonstration program that shows off the advantage of feedforward control. The program was modified from the TrackAnalyze demo that is included in the suite of programs that accompany LCS III, and works in exactly the same way with one small difference: There is now a pair of radio buttons that can be set for either feedback or feedforward control.

MT: Bruce, I’m afraid I don’t see the point of this demo. What it looks like to me is that in the “feedforward” phase you are tracking normally with no lag, because the output and disturbance are both being delayed equally. The fitted model may have a controlled variable that is delayed, but so far as I understand it, the human does not. I don’t see where feedforward or prediction comes into it. Maybe I’m not looking at it the way you see it.

MT: Am I missing something, that you and Rick both think the problem ought to arise when there’s no environmental feedback delay and the effects of both output and disturbance are equally delayed in perception? That would be a most unusual situation in the real world, where perception usually is much faster than the effect of output on the environmental variable. In the computer-simulated world, we often make the “cursor” respond instantaneously to the mouse, but that’s not the usual case in everyday life.

I hope the diagram below will clarify:

In Ashby’s feedforward controller (top diagram), the potential disturbance to a variable, x, is sensed before it reaches x. A mechanism (left box) converts the disturbance waveform into an appropriately scaled action waveform. The disturbance impinges on x after some delay and the counteracting action is applied to x at the same time. The controlled variable is simply x plus any residual error.

This feedforward system will work even if there is essentially no delay, so long as the action can be applied approximately in phase with the disturbance (similar to the delays present in the output of a feedback control system relative to variations in the disturbance). For the demo, I used a 1 second delay to emphasize the advantage of having advance information about the disturbance when delays are present.

For the demo, I did not have a way to get inside the participant’s nervous system and create the function shown in the left box of the Ashby controller. So instead, I had the participant act as a negative feedback control system to keep the cursor tracking the target. It’s important to keep in mind that in this case the target is supposed to represent the disturbance that is being sensed by the participant in advance of its arrival at the variable x that is to be controlled. To the extent that the participant accurately follows the target, the participant’s cursor movements will provide output actions that can be subtracted from x just as in the Ashby feedforward controller. The primary variable to be controlled here is x; this is achieved by acting to control the cursor position relative to the disturbance. Controlling the latter is the only the means by which x is controlled. The negative feedback controller (left box) is here being treated as equivalent to the input-output device of the Ashby-style feedforward control system.

In retrospect, I should have made the variable x explicit in the demo, to make it clear that it is x that is the variable to be controlled; controlling the target-cursor relationship is just the means through which this control over x is achieved. I could have made x follow its own waveform, completely different from that of the disturbance, while the participant’s actions almost completely oppose the disturbance’s effect on x.

In the “feedback” mode provided by the demo, the actions produced by the control system are delayed to simulate a situation in which control is sluggish. By the time the participant’s actions begin to have an influence on the CV, the disturbance value has changed, so that the action is not properly opposing the disturbance. The feedforward controller, by sensing the state of the disturbance in advance of its effects on the CV, allows actions to be developed early enough to compensate for the delay in the effects of those actions on the CV, thus allowing for good control despite these delays.

Bruce

[wmansell]

[From Bruce Abbott (2014.02.01.1040 EST)]

[Martin Taylor 2014.01.31.23.53] –

···

[From Bruce Abbott (2014.01.31.1915 EST)]

The recent discussion of feedforward versus feedback control systems, together with Rick Marken’s prompting, led me to create a demonstration program that shows off the advantage of feedforward control. The program was modified from the TrackAnalyze demo that is included in the suite of programs that accompany LCS III, and works in exactly the same way with one small difference: There is now a pair of radio buttons that can be set for either feedback or feedforward control.

MT: Bruce, I’m afraid I don’t see the point of this demo. What it looks like to me is that in the “feedforward” phase you are tracking normally with no lag, because the output and disturbance are both being delayed equally. The fitted model may have a controlled variable that is delayed, but so far as I understand it, the human does not. I don’t see where feedforward or prediction comes into it. Maybe I’m not looking at it the way you see it.

MT: Am I missing something, that you and Rick both think the problem ought to arise when there’s no environmental feedback delay and the effects of both output and disturbance are equally delayed in perception? That would be a most unusual situation in the real world, where perception usually is much faster than the effect of output on the environmental variable. In the computer-simulated world, we often make the “cursor” respond instantaneously to the mouse, but that’s not the usual case in everyday life.

I hope the diagram below will clarify:

<image003.jpg>

In Ashby’s feedforward controller (top diagram), the potential disturbance to a variable, x, is sensed before it reaches x. A mechanism (left box) converts the disturbance waveform into an appropriately scaled action waveform. The disturbance impinges on x after some delay and the counteracting action is applied to x at the same time. The controlled variable is simply x plus any residual error.

This feedforward system will work even if there is essentially no delay, so long as the action can be applied approximately in phase with the disturbance (similar to the delays present in the output of a feedback control system relative to variations in the disturbance). For the demo, I used a 1 second delay to emphasize the advantage of having advance information about the disturbance when delays are present.

For the demo, I did not have a way to get inside the participant’s nervous system and create the function shown in the left box of the Ashby controller. So instead, I had the participant act as a negative feedback control system to keep the cursor tracking the target. It’s important to keep in mind that in this case the target is supposed to represent the disturbance that is being sensed by the participant in advance of its arrival at the variable x that is to be controlled. To the extent that the participant accurately follows the target, the participant’s cursor movements will provide output actions that can be subtracted from x just as in the Ashby feedforward controller. The primary variable to be controlled here is x; this is achieved by acting to control the cursor position relative to the disturbance. Controlling the latter is the only the means by which x is controlled. The negative feedback controller (left box) is here being treated as equivalent to the input-output device of the Ashby-style feedforward control system.

In retrospect, I should have made the variable x explicit in the demo, to make it clear that it is x that is the variable to be controlled; controlling the target-cursor relationship is just the means through which this control over x is achieved. I could have made x follow its own waveform, completely different from that of the disturbance, while the participant’s actions almost completely oppose the disturbance’s effect on x.

In the “feedback� mode provided by the demo, the actions produced by the control system are delayed to simulate a situation in which control is sluggish. By the time the participant’s actions begin to have an influence on the CV, the disturbance value has changed, so that the action is not properly opposing the disturbance. The feedforward controller, by sensing the state of the disturbance in advance of its effects on the CV, allows actions to be developed early enough to compensate for the delay in the effects of those actions on the CV, thus allowing for good control despite these delays.

Bruce

[wmansell]

[From Bruce Abbott (2014.02.01.1040 EST)]

[Martin Taylor 2014.01.31.23.53] –

···

[From Bruce Abbott (2014.01.31.1915 EST)]

The recent discussion of feedforward versus feedback control systems, together with Rick Marken’s prompting, led me to create a demonstration program that shows off the advantage of feedforward control. The program was modified from the TrackAnalyze demo that is included in the suite of programs that accompany LCS III, and works in exactly the same way with one small difference: There is now a pair of radio buttons that can be set for either feedback or feedforward control.

MT: Bruce, I’m afraid I don’t see the point of this demo. What it looks like to me is that in the “feedforward” phase you are tracking normally with no lag, because the output and disturbance are both being delayed equally. The fitted model may have a controlled variable that is delayed, but so far as I understand it, the human does not. I don’t see where feedforward or prediction comes into it. Maybe I’m not looking at it the way you see it.

MT: Am I missing something, that you and Rick both think the problem ought to arise when there’s no environmental feedback delay and the effects of both output and disturbance are equally delayed in perception? That would be a most unusual situation in the real world, where perception usually is much faster than the effect of output on the environmental variable. In the computer-simulated world, we often make the “cursor” respond instantaneously to the mouse, but that’s not the usual case in everyday life.

I hope the diagram below will clarify:

<image003.jpg>

In Ashby’s feedforward controller (top diagram), the potential disturbance to a variable, x, is sensed before it reaches x. A mechanism (left box) converts the disturbance waveform into an appropriately scaled action waveform. The disturbance impinges on x after some delay and the counteracting action is applied to x at the same time. The controlled variable is simply x plus any residual error.

This feedforward system will work even if there is essentially no delay, so long as the action can be applied approximately in phase with the disturbance (similar to the delays present in the output of a feedback control system relative to variations in the disturbance). For the demo, I used a 1 second delay to emphasize the advantage of having advance information about the disturbance when delays are present.

For the demo, I did not have a way to get inside the participant’s nervous system and create the function shown in the left box of the Ashby controller. So instead, I had the participant act as a negative feedback control system to keep the cursor tracking the target. It’s important to keep in mind that in this case the target is supposed to represent the disturbance that is being sensed by the participant in advance of its arrival at the variable x that is to be controlled. To the extent that the participant accurately follows the target, the participant’s cursor movements will provide output actions that can be subtracted from x just as in the Ashby feedforward controller. The primary variable to be controlled here is x; this is achieved by acting to control the cursor position relative to the disturbance. Controlling the latter is the only the means by which x is controlled. The negative feedback controller (left box) is here being treated as equivalent to the input-output device of the Ashby-style feedforward control system.

In retrospect, I should have made the variable x explicit in the demo, to make it clear that it is x that is the variable to be controlled; controlling the target-cursor relationship is just the means through which this control over x is achieved. I could have made x follow its own waveform, completely different from that of the disturbance, while the participant’s actions almost completely oppose the disturbance’s effect on x.

In the “feedback� mode provided by the demo, the actions produced by the control system are delayed to simulate a situation in which control is sluggish. By the time the participant’s actions begin to have an influence on the CV, the disturbance value has changed, so that the action is not properly opposing the disturbance. The feedforward controller, by sensing the state of the disturbance in advance of its effects on the CV, allows actions to be developed early enough to compensate for the delay in the effects of those actions on the CV, thus allowing for good control despite these delays.

Bruce

[rsmarken]

[From Rick Marken (2014.02.01.0945)]

WM: Hi folks, in fact, in real life, isn't it that we only get a perception of
the potential cause of the disturbance in advance of the disturbance itself.

RM: Right. I think that's what's going on in conditioning, which does
look a lot like feedforward or anticipatory control. But as you can
see from Bill's description of the PCT model of conditioning in the
2011 paper you have up at PCTweb this phenomenon can be explained
without any changes to the PCT model; it's still just feedback control
of perception. In classical conditioning, for example, the PCT
explanation of the apparent anticipation involved is that this is
simply reorganization driven development of of a new perceptual
function with the CS now becoming a disturbance to the perception that
was previously only disturbed by the US. So the dog that only
salivated to food in the mouth (food being a disturbance to the
perception of swallowability of the bolus) comes to salivate to the
sight of food, which have now come to be be a disturbance to the
perception of swallowability due to reorganization. It's the
perception of "swallowability" that has changed, such that the CS
(sight of food) is now as much of a disturbance to it as the US (food
in mouth) had been. It's all control of perception!

···

On Sat, Feb 1, 2014 at 8:42 AM, Warren Mansell <wmansell@gmail.com> wrote:

This perception is therefore subjectively perceived, probably within a
different sensory modality (e.g. Vision rather than touch), and not even
necessarily accurate (the upcoming lorry turns out to have no load and so
doesn't buffet the car), and it is not the actual disturbance - this can
only perceived at the exact time it occurs, not before. The asteroid might
miss earth...
I really like having to think round these things but it all comes round to a
feedback to me, but with some clever trick that does allow some 'perceived
correlational association' between information in our environment that is
predictive of a later disturbance and the feedback system that corrects that
disturbance as soon as it arrives. That would be a new little tweak to PCT,
much like Bill manages in his memory chapter of B:CP.
I wonder whether we can model that?
Any thoughts Bruce, Rick, Martin?
Warren

Sent from my iPhone

On 1 Feb 2014, at 15:40, Bruce Abbott <bbabbott@FRONTIER.COM> wrote:

[From Bruce Abbott (2014.02.01.1040 EST)]

[Martin Taylor 2014.01.31.23.53] --

[From Bruce Abbott (2014.01.31.1915 EST)]

The recent discussion of feedforward versus feedback control systems,
together with Rick Marken's prompting, led me to create a demonstration
program that shows off the advantage of feedforward control. The program
was modified from the TrackAnalyze demo that is included in the suite of
programs that accompany LCS III, and works in exactly the same way with one
small difference: There is now a pair of radio buttons that can be set for
either feedback or feedforward control.

MT: Bruce, I'm afraid I don't see the point of this demo. What it looks like
to me is that in the "feedforward" phase you are tracking normally with no
lag, because the output and disturbance are both being delayed equally. The
fitted model may have a controlled variable that is delayed, but so far as I
understand it, the human does not. I don't see where feedforward or
prediction comes into it. Maybe I'm not looking at it the way you see it.

MT: Am I missing something, that you and Rick both think the problem ought
to arise when there's no environmental feedback delay and the effects of
both output and disturbance are equally delayed in perception? That would be
a most unusual situation in the real world, where perception usually is much
faster than the effect of output on the environmental variable. In the
computer-simulated world, we often make the "cursor" respond instantaneously
to the mouse, but that's not the usual case in everyday life.

I hope the diagram below will clarify:

<image003.jpg>

In Ashby's feedforward controller (top diagram), the potential disturbance
to a variable, x, is sensed before it reaches x. A mechanism (left box)
converts the disturbance waveform into an appropriately scaled action
waveform. The disturbance impinges on x after some delay and the
counteracting action is applied to x at the same time. The controlled
variable is simply x plus any residual error.

This feedforward system will work even if there is essentially no delay, so
long as the action can be applied approximately in phase with the
disturbance (similar to the delays present in the output of a feedback
control system relative to variations in the disturbance). For the demo, I
used a 1 second delay to emphasize the advantage of having advance
information about the disturbance when delays are present.

For the demo, I did not have a way to get inside the participant's nervous
system and create the function shown in the left box of the Ashby
controller. So instead, I had the participant act as a negative feedback
control system to keep the cursor tracking the target. It's important to
keep in mind that in this case the target is supposed to represent the
disturbance that is being sensed by the participant in advance of its
arrival at the variable x that is to be controlled. To the extent that the
participant accurately follows the target, the participant's cursor
movements will provide output actions that can be subtracted from x just as
in the Ashby feedforward controller. The primary variable to be controlled
here is x; this is achieved by acting to control the cursor position
relative to the disturbance. Controlling the latter is the only the means
by which x is controlled. The negative feedback controller (left box) is
here being treated as equivalent to the input-output device of the
Ashby-style feedforward control system.

In retrospect, I should have made the variable x explicit in the demo, to
make it clear that it is x that is the variable to be controlled;
controlling the target-cursor relationship is just the means through which
this control over x is achieved. I could have made x follow its own
waveform, completely different from that of the disturbance, while the
participant's actions almost completely oppose the disturbance's effect on
x.

In the "feedback" mode provided by the demo, the actions produced by the
control system are delayed to simulate a situation in which control is
sluggish. By the time the participant's actions begin to have an influence
on the CV, the disturbance value has changed, so that the action is not
properly opposing the disturbance. The feedforward controller, by sensing
the state of the disturbance in advance of its effects on the CV, allows
actions to be developed early enough to compensate for the delay in the
effects of those actions on the CV, thus allowing for good control despite
these delays.

Bruce

--
Richard S. Marken PhD
www.mindreadings.com

The only thing that will redeem mankind is cooperation.
                                                   -- Bertrand Russell

[Abbott]

[From Bruce Abbott (2014.02.01.1300 EST)]

In my feedforward demo I used the participant’s control of the difference between target position (representing the current value of the disturbance) and cursor position as a way to convert the disturbance signal into an output suitable to be applied to the forward-controlled variable. Bill Powers once told me that the scanning tunneling microscope works in this way, using negative feedback control to generate an output that traces the positions of atoms in a metal surface. Here’s a description from the Free Dictionary (http://www.thefreedictionary.com/scanning+tunneling+microscope ):

scanning tunneling microscope

A microscope used to make images of individual atoms on the surface of a metal. The microscope has a probe with a small voltage applied to it ending in a tiny sharp tip (ideally consisting of one atom) that is moved close the material’s surface. Quantum tunneling of electrons between tip and the metal provides a small current, and that current is held constant by varying the distance between the tip and the material’s surface atoms. As the probe is moved across the surface, a three-dimension image of the surface is formed, based on the continual adjustments made to the height of the tip to keep the electron flow constant.

By adjusting the height of the needle above the surface, the system keeps the current flowing from the scanning needle constant. The image is constructed by reading out the height of the tip as the needle moves across the surface. That height is the output of the current-adjusting control system. So here we have a negative feedback control system being used to convert the distance of an atom from the needle (input) to the vertical position of the tip (output), acting much like a simple input-output function relating atom position to tip position.

In the demo, by adjusting the cursor position to keep it aligned with the moving target, the participant produces an output that tracks the disturbance, and thus creating what appears to be a simple input-output function relating disturbance value to output value.

Bruce

[Abbott]

[From Bruce Abbott (2014.02.01.1320 EST)]

···

From: Control Systems Group Network (CSGnet) [mailto:CSGNET@LISTSERV.ILLINOIS.EDU] On Behalf Of Warren Mansell
Sent: Saturday, February 01, 2014 11:24 AM
To: CSGNET@LISTSERV.ILLINOIS.EDU
Subject: Re: Feedforward Demo

WM: Hi Bruce, I think this makes sense to me. It does seem important for there to be this kind of anticipatory control and I think it is a little similar to how Bill, and Wayne Hershberger, tried to model the observation of conditioning Bill mentions it in the 2011 paper on pctweb and I think it also crops up in Henry Yin’s recent chapter.

For example, you see a lorry heading from behind in the outside lane and you start to turn away to anticipate the buffeting it will cause as it passes. There seems to be two aspects to feed forward. One is about tracking the perception of the disturbance and the other is about applying the appropriate action. It seems to be that both are feedback loops but there is some important connection between the two - the person seems to be treating the perception of the upcoming disturbance as a surrogate for the effects of the actual disturbance and this occurs earlier than if the actual disturbance was used. But it is still the same feedback loop that would have been used if the upcoming disturbance had not been perceived, and it would momentarily start as open loop until the effects of the actual disturbance on the controlled variable commence, then it’s negative feedback again…

Is that how everyone sees it, or not?

Warren

Bill and I played around with modeling classical conditioning some time ago and yes, the feedforward demo is suggestive of that process. In classical conditioning, a “stimulusâ€? event such as a tone precedes a second event, such as a puff of air delivered to the cornea of a participant’s eye. The puff of air disturbs a controlled variable – presumably the intensity of corneal sensation – and the control system then operates the eyelid to close it. Typically the puff is short so that the eyelid closes after the puff has been delivered, too late to protect the cornea from the disturbance, which by then has gone away. After many pairings of the tone with the puff, the participant develops a so-called conditional response in which the eyelid closes when the tone sounds. Typically the delay between tone onset and puff is around 200-500 ms, and at this delay, the eyelid gets at least partly closed by the time the puff occurs. The blink is thus timed to at least partially protect the cornea from the puff.

To create an adequate model of the conditioning process, one must of course add reorganization to the model. Perceptual input in the form of the tone must somehow get wired into the reference of the system controlling eyelid closure, and act with a proper delay so that eyelid closure is timed to coincide with the arrival of the puff of air. How the reorganizing system accomplishes this remains to be worked out.

Bruce

[wmansell]

[From Bruce Abbott (2014.02.01.1320 EST)]

···

From: Control Systems Group Network (CSGnet) [mailto:CSGNET@LISTSERV.ILLINOIS.EDU] On Behalf Of Warren Mansell
Sent: Saturday, February 01, 2014 11:24 AM
To: CSGNET@LISTSERV.ILLINOIS.EDU
Subject: Re: Feedforward Demo

WM: Hi Bruce, I think this makes sense to me. It does seem important for there to be this kind of anticipatory control and I think it is a little similar to how Bill, and Wayne Hershberger, tried to model the observation of conditioning Bill mentions it in the 2011 paper on pctweb and I think it also crops up in Henry Yin’s recent chapter.

For example, you see a lorry heading from behind in the outside lane and you start to turn away to anticipate the buffeting it will cause as it passes. There seems to be two aspects to feed forward. One is about tracking the perception of the disturbance and the other is about applying the appropriate action. It seems to be that both are feedback loops but there is some important connection between the two - the person seems to be treating the perception of the upcoming disturbance as a surrogate for the effects of the actual disturbance and this occurs earlier than if the actual disturbance was used. But it is still the same feedback loop that would have been used if the upcoming disturbance had not been perceived, and it would momentarily start as open loop until the effects of the actual disturbance on the controlled variable commence, then it’s negative feedback again…

Is that how everyone sees it, or not?

Warren

Bill and I played around with modeling classical conditioning some time ago and yes, the feedforward demo is suggestive of that process. In classical conditioning, a “stimulusâ€? event such as a tone precedes a second event, such as a puff of air delivered to the cornea of a participant’s eye. The puff of air disturbs a controlled variable – presumably the intensity of corneal sensation – and the control ssystem then operates the eyelid to close it. Typically the puff is short so that the eyelid closes after the puff has been delivered, too late to protect the cornea from the disturbance, which by then has gone away. After many pairings of the tone with the puff, the participant develops a so-called conditional response in which the eyelid closes when the tone sounds. Typically the delay between tone onset and puff is around 200-500 ms, and at this delay, the eyelid gets at least partly closed by the time the puff occurs. The blink is thus timed to at least partially protect the cornea from the puff.

To create an adequate model of the conditioning process, one must of course add reorganization to the model. Perceptual input in the form of the tone must somehow get wired into the reference of the system controlling eyelid closure, and act with a proper delay so that eyelid closure is timed to coincide with the arrival of the puff of air. How the reorganizing system accomplishes this remains to be worked out.

Bruce

[rsmarken]

[From Rick Marken (2014.02.01.1420)]

···

Bruce Abbott (2014.02.01.1320 EST)–

BA: To create an adequate model of the conditioning process, one must of course add reorganization to the model. Perceptual input in the form of the tone must somehow get wired into the reference of the system controlling eyelid closure, and act with a proper delay so that eyelid closure is timed to coincide with the arrival of the puff of air.

RM: Maybe this is where the modeling took the wrong turn. All you need for an adequate model of conditioning (at east according to my reading of the description in Bill’s 2011 paper at pctweb) is for the CS to become part of the controlled perception (CV) so that variations in the CV become a disturbance to the CV just like the US is a disturbance to the CV. So in the air puff situation conditioning results in the subject controlling a perception of tone+moist eye by blinking.

BA:How the reorganizing system accomplishes this remains to be worked out.

That’s not a problem if the conditioning is thought of as resulting in a new CV; we know how the reorganization system creates new CVs; Bill has demonstrated that process many times.

Best

Rick

–
Richard S. Marken PhD
www.mindreadings.com
The only thing that will redeem mankind is cooperation.
– Bertrand Russell

[rsmarken]

[From Rick Marken (2014.02.01.1430)]

···

Rick Marken (2014.02.01.1420)]

RM: Oops. If the presence of the tone is a disturbance to the CV then when the subject blinks to the tone (with or w/o the air puff) it can be seen as controlling for no tone + mouse eye.

I think this could be a very do-able model/demo.

Best

Rick

–
Richard S. Marken PhD
www.mindreadings.com
The only thing that will redeem mankind is cooperation.

                                               -- Bertrand Russell

RM: So in the air puff situation conditioning results in the subject controlling a perception of tone+moist eye by blinking.

–
Richard S. Marken PhD
www.mindreadings.com
The only thing that will redeem mankind is cooperation.
– Bertrand Russell

[rsmarken]

This is better; no "mouse" eyes.
RM: If the presence of the tone is a disturbance to the CV then when the subject blinks to the tone (with or w/o the air puff) it can be seen as controlling for _no tone_ + moist eye.

···

--
Richard S. Marken PhD
<http://www.mindreadings.com>www.mindreadings.com
The only thing that will redeem mankind is cooperation.
-- Bertrand Russell

[Abbott]

[From Bruce Abbott (2014.02.01.1900 EST)]

Rick Marken (2014.02.01.1420) –

Bruce Abbott (2014.02.01.1320 EST)

BA: To create an adequate model of the conditioning process, one must of course add reorganization to the model. Perceptual input in the form of the tone must somehow get wired into the reference of the system controlling eyelid closure, and act with a proper delay so that eyelid closure is timed to coincide with the arrival of the puff of air.

RM: Maybe this is where the modeling took the wrong turn. All you need for an adequate model of conditioning (at east according to my reading of the description in Bill’s 2011 paper at pctweb) is for the CS to become part of the controlled perception (CV) so that variations in the CV become a disturbance to the CV just like the US is a disturbance to the CV. So in the air puff situation conditioning results in the subject controlling a perception of tone+moist eye by blinking.

BA:How the reorganizing system accomplishes this remains to be worked out.

That’s not a problem if the conditioning is thought of as resulting in a new CV; we know how the reorganization system creates new CVs; Bill has demonstrated that process many times.

BA: I think you are being misled by the demonstrations into thinking that reorganization is simpler than it is. In the demos (e.g., ArmControlReo from LCS III), the control systems already exist and reorganization merely alters the weights connecting the outputs of the systems to the various joints. The direction and magnitude of change in the weights varies in small steps whose sizes change in proportion to the level of overall error in the control systems. So the demos start out with a good deal of structure already in place. Furthermore, these changes depend only on error within the respective control systems controlling the angles of various joints; they have nothing to do with “intrinsic” error that Bill hypothesized drives reorganization in real biological systems. Bill’s demonstrations of reorganization show that such a process can work, but they are only a start on the problem. We are a long way from modeling a realistic reorganizing system.

BA: The model you suggest for eyeblink conditioning might work for some cases of classical conditioning but not for all. The problem is that in some cases, the conditional response is actually the opposite of the unconditional response. Unconditional responses to certain drugs are in these cases opposed by the conditional responses that develop to stimuli that precede drug injection. This conditional response is thought to be responsible for at least a portion of the tolerance that develops when the drug is taken repeatedly.

BA: As for the physiological basis of the conditional response, in eyeblink conditioning, the CR and UR both involve blinking the eye. Studies by Richard F. Thompson and others using rabbits (in which the “eye-blink” actually involves the closing of the rabbit’s nictitating membrane, the so-called “third” eyelid) have shown that a structure in the cerebellum called the interpositus nucleus is crucial for the development and maintenance of the conditional response. Lesion the interpositus nucleus prior to conditioning and the CR cannot be developed. Do it after conditioning and the CR is abolished, whereas the UR remains unaffected. (See http://en.wikipedia.org/wiki/Eyeblink_conditioning . )

It is not sufficient to say that conditioning develops as a result of reorganization, because reorganization might produce any number of organizations. One needs to learn what changes within the nervous system actually occur during reorganization (no easy task!). At present, in rabbit eyeblink conditioning, the interpositus nucleus of the cerebellum appears to be an important component involved in the process, but beyond that much remains to be learned.

Bruce

[rsmarken]

[From Rick Marken (2014.02.02.0850)]

···

Bruce Abbott (2014.02.01.1900 EST)

RM: That’s not a problem if the conditioning is thought of as resulting in a new CV; we know how the reorganization system creates new CVs; Bill has demonstrated that process many times.

BA: I think you are being misled by the demonstrations into thinking that reorganization is simpler than it is…Bill’s demonstrations of reorganization show that such a process can work, but they are only a start on the problem. We are a long way from modeling a realistic reorganizing system.

RM: A journey of a 1000 miles begins with the first step, grasshopper. And this first step would be in the right direction since it would be based on an understanding that all conditioning is a control and not an S-R process. Wouldn’t it be lovely to show that classical and operant conditioning can be understood as control of perception.

BA: The model you suggest for eyeblink conditioning might work for some cases of classical conditioning but not for all.

RM: Maybe not the specific implementation of the model. But a control model should work in all cases.

BA: The problem is that in some cases, the conditional response is actually the opposite of the unconditional response.

RM: Why is that a problem? I’d have to see the specific situation but it suggests that when the CS is incorporated into the CV the nature of the new CV results in a reversal of the effect of the UR on this CV.

BA: As for the physiological basis of the conditional response, in eyeblink conditioning, the CR and UR both involve blinking the eye. Studies by Richard F. Thompson and others using rabbits (in which the “eye-blink” actually involves the closing of the rabbit’s nictitating membrane, the so-called “third” eyelid) have shown that a structure in the cerebellum called the interpositus nucleus is crucial for the development and maintenance of the conditional response. Lesion the interpositus nucleus prior to conditioning and the CR cannot be developed. Do it after conditioning and the CR is abolished, whereas the UR remains unaffected. (See http://en.wikipedia.org/wiki/Eyeblink_conditioning . )

RM: I think basing functional models of behavior on gross physiological studies like this is like basing one’s understanding of how a computer program works by looking at the effect of breaking various connection on the motherboard and seeing how program performance if affected. I think you’re finding that out in spade with the Zan/Zhu model of limb control, which is ostensibly based on what is known of the physiology underlying limb movement. Apparently this “physiologically accurate” equilibrium point model doesn’t even control. I think that’s why they tested it in a situation where they asked they subjects to try to not control their arm movement when it was disturbed.

BA: It is not sufficient to say that conditioning develops as a result of reorganization, because reorganization might produce any number of organizations. One needs to learn what changes within the nervous system actually occur during reorganization (no easy task!). At present, in rabbit eyeblink conditioning, the interpositus nucleus of the cerebellum appears to be an important component involved in the process, but beyond that much remains to be learned.

RM: I disagree with this (as usual). While whatever model we develop should not be grossly inconsistent with what we know about how the NS works, I think a control theory model of classical conditioning of the eyeblink, say, would make quite an impression in the S-R community. And if you don’t do it – and you are much more skilled at programming than I am – then I will. I think this would be a very important project because it would show how control theory can handle an area of psychology – conditioning studies – that we too often neglect.

Best

Rick

–
Richard S. Marken PhD
www.mindreadings.com
The only thing that will redeem mankind is cooperation.

                                               -- Bertrand Russell

[MartinT]

[Martin Taylor 2014.02.01.00.01]

Warren,

There are two ways the future state of a variable can be predicted,
but before dealing with that split, I think we have to consider what
is meant by “prediction”. Prediction never means certainty about a
future value. It means only that on average you will be closer to
the future value of a variable if you use prediction than if you use
only the current value of the variable.
The first way of prediction is to observe something that influences
the variation of the variable of interest, “x”. If you know, for
example, that an old-fashioned steam locomotive can go
“chuff-chuff-chuff” at intervals that only slowly increase or
decrease according to the acceleration and speed of the train, you
can control any variable that is disturbed by the “chuff” better
than if you never can guess when the next “chuff” will occur. For
example, some control system might interpose a shield whenever a
“chuff” is expected. Charles Babbage used exactly that procedure to examine safely a lava
lake at the bottom of a vent in the crater floor of Vesuvius during
an eruption (though his shoes were burned to a crisp!). The lake
sent up a geyser of lava that fell all around the small vent.
Babbage measured the timing and the range of lava bombs, and went to
the vent to look down during the non-explosive period, getting away
just in time before the next explosion.
This means of prediction always involves at least two control units.
The second way of prediction, if the variable is continuously
variable, is to use the variable’s own statistics, such as for
example, the velocity and acceleration. The left panel shows the distribution of a variable at time t if you
know its overall distribution but don’t know its present value. The
second panel assumes that you know the value of the variable at time
t-tau and the distribution of its velocity. If tau is short enough
compared to the expected rate of change of the variable, the
distribution is narrower than the global distribution. The right
panel assumes that you know the velocity at t-tau and the
distribution of acceleration. The uncertainty of the variable’s
value at time t+tau is much reduced. Ignoring this fact turned out
to be the reason why my analysis two or three years ago of the best
possible precision of control actually gave an answer worse than the
precision Bill achieved in a real-life tracking experiment.
Including the prediction available from the velocity resolved the
problem.
The requirement to know the distributions of the velocity in panel 2
and the acceleration in panel 3 applies only to the analyst, who
understands that those distributions depend on the bandwidth and
power of the variable’s waveform, among other things. The control
system simply uses the current values. Its control performance will
be affected by the velocity and acceleration distributions, even
though it knows nothing of them. When you shoot at a distant moving
target, you don’t aim where it is now, you aim where you think it
will be when the shot reaches it.
If the variable is discontinuous, the principle is the same, though
the technique may be different. If you see what appears to be an
English text, and you have recently seen space-T-h, you are much
more likely to guess correctly that the next character will be “e”
than you would be if you had seen only “-h”. For many variables,
continuous or discrete, knowledge of the past statistical history
makes a big difference to the ability to predict. Whether such
knowledge is used by any particular control system is another
matter. Bill’s Artificial Cerebellum learned the temporal statistics
of a variable in order to improve its control performance, but the
standard control loop does not.
These two approaches to prediction (model the disturbance, use the
history of a waveform) lead to different possibilities for
implementing “feedforward”. The first kind (Ashby) requires the
intervention of a second (“helping”) control system that does
something to reduce the effect of the disturbance on the controlled
variable. The second kind requires the development of something
within the single control loop, or (as Rick tirelessly points out)
the construction of another level of control, the two perhaps being
functionally equivalent.
There is a third situation that is a kind of amalgam of the other
two. It is made manifest in conditioning studies. Statistically,
some “cue event” has occurred just before or along with a
disturbance to some controlled variable. As Rick has said, a
reasonable PCT description of what happens could be the construction
of a new controlled perception that incorporates the cue event but
whose output opposes the disturbance to the original controlled
variable or interposes a shield so that the disturbance cause no
longer influences the controlled variable. But there is another
reasonable PCT description of this case.
Assume that Bill was correct in B:CP Figure 15.3, that the reference
value for a control unit is a stored perception in a
content-addressable associative memory. The address into this memory
has to be a vector of the contexts in which this reference
perception was appropriate. As the cue event is very often present
when the disturbance must be opposed, the cue event is likely to
become an important part of the context that defines a reference
value that results in the disturbance being opposed. In this
interpretation, no new controlled perception containing both the cue
event and the original controlled perception would be developed. Any
context containing the cue event would be likely to result in the
provision of the appropriate reference value to the control unit
that originally acted alone.
These two (or three?) ways of implementing feedforward all come down
to different ways of implementing prediction. Whether any particular
biological system uses any of them is a research question.
Martin

···

On 2014/02/1 11:42 AM, Warren Mansell
wrote:

    Hi folks, in fact, in real life, isn't it that we only get a

perception of the potential cause of the disturbance in advance
of the disturbance itself. This perception is therefore
subjectively perceived, probably within a different sensory
modality (e.g. Vision rather than touch), and not even
necessarily accurate (the upcoming lorry turns out to have no
load and so doesn’t buffet the car), and it is not the actual
disturbance - this can only perceived at the exact time it
occurs, not before. The asteroid might miss earth…

    I really like having to think round these things but it all

comes round to a feedback to me, but with some clever trick that
does allow some ‘perceived correlational association’ between
information in our environment that is predictive of a later
disturbance and the feedback system that corrects that
disturbance as soon as it arrives. That would be a new little
tweak to PCT, much like Bill manages in his memory chapter of
B:CP.

I wonder whether we can model that?

Any thoughts Bruce, Rick, Martin?

Warren

[wmansell]

Thank you Martin so much! Your multi-component definition has really dissected the idea of 'feed forward' for me in a way that actually makes a difference. I'm had thought about the memory cue idea before but then abandoned it. The example of firing ahead of a moving target is a good one to imagine and I can see how a new perceptual layer within a feedback control system can manage that too. I then see your third example as a helping control system and the idea of a 'shield' that can be placed ahead of the disruption is again a nice one to remember.
I guess what I am still left with is the fact that contemporary uses of feed forward involve pre-specifying the output. I was wondering whether even though the timing of the output would be adjusted on the fly, they actual output would merely be an existing control system, already developed through the reorganisation of perceptual feedback in the past. This would offset the limitations of trying to use inverse modeling or a response that is maybe evolutionarily prepared rather then learned. My hunch is that the rate of the moving legs of a newly hatched chick are driven by feedback?
The other important question is whether these categories of 'predictive' disturbances are published together, or are from your own inference? If not published, they need to be!
Warren

···

Sent from my iPhone

On 2 Feb 2014, at 23:01, Martin Taylor <mmt-csg@MMTAYLOR.NET> wrote:

[Martin Taylor 2014.02.01.00.01]

On 2014/02/1 11:42 AM, Warren Mansell wrote:
Hi folks, in fact, in real life, isn't it that we only get a perception of the potential cause of the disturbance in advance of the disturbance itself. This perception is therefore subjectively perceived, probably within a different sensory modality (e.g. Vision rather than touch), and not even necessarily accurate (the upcoming lorry turns out to have no load and so doesn't buffet the car), and it is not the actual disturbance - this can only perceived at the exact time it occurs, not before. The asteroid might miss earth...
I really like having to think round these things but it all comes round to a feedback to me, but with some clever trick that does allow some 'perceived correlational association' between information in our environment that is predictive of a later disturbance and the feedback system that corrects that disturbance as soon as it arrives. That would be a new little tweak to PCT, much like Bill manages in his memory chapter of B:CP.
I wonder whether we can model that?
Any thoughts Bruce, Rick, Martin?
Warren

Warren,

There are two ways the future state of a variable can be predicted, but before dealing with that split, I think we have to consider what is meant by "prediction". Prediction never means certainty about a future value. It means only that on average you will be closer to the future value of a variable if you use prediction than if you use only the current value of the variable.

The first way of prediction is to observe something that influences the variation of the variable of interest, "x". If you know, for example, that an old-fashioned steam locomotive can go "chuff-chuff-chuff" at intervals that only slowly increase or decrease according to the acceleration and speed of the train, you can control any variable that is disturbed by the "chuff" better than if you never can guess when the next "chuff" will occur. For example, some control system might interpose a shield whenever a "chuff" is expected.

Charles Babbage used exactly that procedure to examine safely a lava lake at the bottom of a vent in the crater floor of Vesuvius during an eruption (though his shoes were burned to a crisp!). The lake sent up a geyser of lava that fell all around the small vent. Babbage measured the timing and the range of lava bombs, and went to the vent to look down during the non-explosive period, getting away just in time before the next explosion.

This means of prediction always involves at least two control units.

The second way of prediction, if the variable is continuously variable, is to use the variable's own statistics, such as for example, the velocity and acceleration.

<HistoryInformation.jpg>

The left panel shows the distribution of a variable at time t if you know its overall distribution but don't know its present value. The second panel assumes that you know the value of the variable at time t-tau and the distribution of its velocity. If tau is short enough compared to the expected rate of change of the variable, the distribution is narrower than the global distribution. The right panel assumes that you know the velocity at t-tau and the distribution of acceleration. The uncertainty of the variable's value at time t+tau is much reduced. Ignoring this fact turned out to be the reason why my analysis two or three years ago of the best possible precision of control actually gave an answer worse than the precision Bill achieved in a real-life tracking experiment. Including the prediction available from the velocity resolved the problem.

The requirement to know the distributions of the velocity in panel 2 and the acceleration in panel 3 applies only to the analyst, who understands that those distributions depend on the bandwidth and power of the variable's waveform, among other things. The control system simply uses the current values. Its control performance will be affected by the velocity and acceleration distributions, even though it knows nothing of them. When you shoot at a distant moving target, you don't aim where it is now, you aim where you think it will be when the shot reaches it.

If the variable is discontinuous, the principle is the same, though the technique may be different. If you see what appears to be an English text, and you have recently seen space-T-h, you are much more likely to guess correctly that the next character will be "e" than you would be if you had seen only "-h". For many variables, continuous or discrete, knowledge of the past statistical history makes a big difference to the ability to predict. Whether such knowledge is used by any particular control system is another matter. Bill's Artificial Cerebellum learned the temporal statistics of a variable in order to improve its control performance, but the standard control loop does not.

These two approaches to prediction (model the disturbance, use the history of a waveform) lead to different possibilities for implementing "feedforward". The first kind (Ashby) requires the intervention of a second ("helping") control system that does something to reduce the effect of the disturbance on the controlled variable. The second kind requires the development of something within the single control loop, or (as Rick tirelessly points out) the construction of another level of control, the two perhaps being functionally equivalent.

There is a third situation that is a kind of amalgam of the other two. It is made manifest in conditioning studies. Statistically, some "cue event" has occurred just before or along with a disturbance to some controlled variable. As Rick has said, a reasonable PCT description of what happens could be the construction of a new controlled perception that incorporates the cue event but whose output opposes the disturbance to the original controlled variable or interposes a shield so that the disturbance cause no longer influences the controlled variable. But there is another reasonable PCT description of this case.

Assume that Bill was correct in B:CP Figure 15.3, that the reference value for a control unit is a stored perception in a content-addressable associative memory. The address into this memory has to be a vector of the contexts in which this reference perception was appropriate. As the cue event is very often present when the disturbance must be opposed, the cue event is likely to become an important part of the context that defines a reference value that results in the disturbance being opposed. In this interpretation, no new controlled perception containing both the cue event and the original controlled perception would be developed. Any context containing the cue event would be likely to result in the provision of the appropriate reference value to the control unit that originally acted alone.

These two (or three?) ways of implementing feedforward all come down to different ways of implementing prediction. Whether any particular biological system uses any of them is a research question.

Martin

[Abbott]

[From Bruce Abbott (2014.02.03.0905 EST)]

Rick Marken (2014.02.02.0850)–

Bruce Abbott (2014.02.01.1900 EST)

RM: That’s not a problem if the conditioning is thought of as resulting in a new CV; we know how the reorganization system creates new CVs; Bill has demonstrated that process many times.

BA: I think you are being misled by the demonstrations into thinking that reorganization is simpler than it is…Bill’s demonstrations of reorganization show that such a process can work, but they are only a start on the problem. We are a long way from modeling a realistic reorganizing system.

RM: A journey of a 1000 miles begins with the first step, grasshopper. And this first step would be in the right direction since it would be based on an understanding that all conditioning is a control and not an S-R process. Wouldn’t it be lovely to show that classical and operant conditioning can be understood as control of perception.

BA: No doubt, but the conditioning process is more complex than it first appears. An adequate model will have to address a number of known conditioning phenomena. I’d rather attempt to develop an adequate model rather than propose a different model for each case on an ad hoc basis.

BA: The model you suggest for eyeblink conditioning might work for some cases of classical conditioning but not for all.

RM: Maybe not the specific implementation of the model. But a control model should work in all cases.

BA: Yes, but the question is, what control model? Developing an adequate one will be no easy task.

BA: The problem is that in some cases, the conditional response is actually the opposite of the unconditional response.

RM: Why is that a problem? I’d have to see the specific situation but it suggests that when the CS is incorporated into the CV the nature of the new CV results in a reversal of the effect of the UR on this CV.

BA: It’s not a problem if you want to create a set of models that look like classical conditioning but had to be written on an ad hoc basis to reflect what happens in each specific case. That exercise sounds more like curve-fitting than science to me. The outcome of the conditioning process varies in surprising ways depending on the values of a number of parameters.

BA: As for the physiological basis of the conditional response, in eyeblink conditioning, the CR and UR both involve blinking the eye. Studies by Richard F. Thompson and others using rabbits (in which the “eye-blink” actually involves the closing of the rabbit’s nictitating membrane, the so-called “third” eyelid) have shown that a structure in the cerebellum called the interpositus nucleus is crucial for the development and maintenance of the conditional response. Lesion the interpositus nucleus prior to conditioning and the CR cannot be developed. Do it after conditioning and the CR is abolished, whereas the UR remains unaffected. (See http://en.wikipedia.org/wiki/Eyeblink_conditioning . )

RM: I think basing functional models of behavior on gross physiological studies like this is like basing one’s understanding of how a computer program works by looking at the effect of breaking various connection on the motherboard and seeing how program performance if affected. I think you’re finding that out in spade with the Zan/Zhu model of limb control, which is ostensibly based on what is known of the physiology underlying limb movement. Apparently this “physiologically accurate” equilibrium point model doesn’t even control. I think that’s why they tested it in a situation where they asked they subjects to try to not control their arm movement when it was disturbed.

BA: Bill used available physiological information when speculating about the first-order control system that operates the muscles. Without some guidance from physiological data, one is free to just make stuff up. The models we usually develop simply assume that changes in reference levels for joint angles somehow – and I emphasize that word somehow – result in joint angle changes. We model that with a simple control system, which is an adequate place-marker until we discover how the actual physiological systems are organized.

BA: It is not sufficient to say that conditioning develops as a result of reorganization, because reorganization might produce any number of organizations. One needs to learn what changes within the nervous system actually occur during reorganization (no easy task!). At present, in rabbit eyeblink conditioning, the interpositus nucleus of the cerebellum appears to be an important component involved in the process, but beyond that much remains to be learned.

RM: I disagree with this (as usual). While whatever model we develop should not be grossly inconsistent with what we know about how the NS works, I think a control theory model of classical conditioning of the eyeblink, say, would make quite an impression in the S-R community. And if you don’t do it – and you are much more skilled at programming than I am – then I will. I think this would be a very important project because it would show how control theory can handle an area of psychology – conditioning studies – that we too often neglect.

BA: Developing an adequate control theory model of classical conditioning is something I hope to do – it’s something Bill and I worked on together for a time – but for the present I have another major project on my hands – understanding the equilibrium point model of muscle control, critiquing it, and attempting a PCT model that reproduces known data patterns. That’s enough for now.

Bruce

[MartinT]

[Martin Taylor 2014.02.03.14.40]

···

On 2014/02/3 2:18 AM, Warren Mansell wrote:

Thank you Martin so much! Your multi-component definition has really dissected the idea of 'feed forward' for me in a way that actually makes a difference. ...
The other important question is whether these categories of 'predictive' disturbances are published together, or are from your own inference? If not published, they need to be!

No, they were just off the top of my head. If you like them, use them.

Martin

[rsmarken]

[From Rick Marken (2014.02.03.1630)]

Martin Taylor (2014.02.01.00.01) --

MT: The first way of prediction is to observe something that influences the variation of the variable of interest, "x". If you know, for example, that an old-fashioned steam locomotive can go "chuff-chuff-chuff" at intervals that only slowly increase or decrease according to the acceleration and speed of the train, you can control any variable that is disturbed by the "chuff" better than if you never can guess when the next "chuff" will occur. For example, some control system might interpose a shield whenever a "chuff" is expected.

RM: The deployment of the shield can be a commanded output artificial
control system, which can have S-R components, but in a living system
it's inputs, not output, that are commanded. So the analogous
situation in a living control system would involve controlling the
perception of the relationship between one sequence (shield
deployments) and another (chuffs). In living systems, apparent
feedforward control -- where outputs seem to anticipate disturbances
to inputs -- has to involve control of perception, not output.

MT: The second way of prediction, if the variable is continuously variable, is to use the variable's own statistics, such as for example, the velocity and acceleration.

RM: I believe this is the kind of feedforward is contained in your
model that adds the derivative of the input variable to the perception
of the input variable, as per your equatoin:

p(t) = s(t) + k*(ds(t)/dt)

where is the the sensory input variable. When I implemented this model
I found what I expected: the model controls the perception s(t) +
k*(ds(t)/dt), which resulted in increasingly poor control of s(t) as k
increased from 0; in other words, control of s(t) was best when no
derivative was added to s(t), which is not a controlled variable
anyway because the system cannot perceive it; it perceives s(t) +
k*(ds(t)/dt).

This result seemed to anger you since you said I didn't know how to
properly compute (ds(t)/dt). But you never told me the proper way to
compute the derivative so I never got a chance to test it again. I
don't really know why you got so upset about this but I suspect it's
because you think of the derivative, (ds(t)/dt) as feedforward that
should improve control of s(t). So if you tell me the correct way to
compute the derivative I'll check it out. But I don't believe that any
change in the computation of (ds(t)/dt) will help; what you showed me
were the equations for a feedback control model that controls s(t) +
k*(ds(t)/dt). There is no feedforward in your model.

MT: . The uncertainty of the variable's value at time t+tau is much reduced. Ignoring this fact turned out to be the reason why my analysis two or three years ago of the best possible precision of control actually gave an answer worse than the precision Bill achieved in a real-life tracking experiment. Including the prediction available from the velocity resolved the problem.

RM: You seem to be saying that a model that controlled s(t) +
k*(ds(t)/dt) accounted for actual tracking data better than a model
that controls just s(t) (the usual variable we put into our models).
That's quite a surprise; do you have the data and model results to
prove it? I'd really like to see that!

MT: Bill's Artificial Cerebellum learned the temporal statistics of a variable in order to improve its control performance, but the standard control loop does not.

RM: I don't think the Artificial Cerebellum did anything other than
use reorganization to find parameters of control systems that results
in best control (indicated by minimum error). The model knew nothing
about the temporal statistics of any of the variables involved in the
control process.

MT: These two (or three?) ways of implementing feedforward all come down to different ways of implementing prediction. Whether any particular biological system uses any of them is a research question.

RM: But you didn't "implement" the control systems we are interested
in understanding; living control systems. So you are dreaming up
feedforward schemes that simply can be implemented in devices you can
build; but there is no evidence that such systems have been
implemented in living control systems. The only thing we know that
living systems do that looks like "feedforward" is do things, like
running away, when they see other things; organisms seem to be able to
act as though they "anticipate" bad (or good) things happening in the
future and act accordingly. But this is an S-R view of what is going
on -- a very seductive one but an S-R view nevertheless.

Since we know that living systems, being closed loop control systems,
don't act in response to events, we have to revise our thinking about
what we are seeing when we see organisms apparently acting in
anticipation of something. Since we know that control systems only act
to protect controlled variables from disturbance, what we must be
seeing when we see apparent "predictive" or "feedforward" control must
be action aimed at protecting a controlled variable from disturbance.
Think about this the next time you see what appears to be predictive
control actions. For example, what is going on when I see dark clouds
on the horizon and get an umbrella from the closet. Is it a
feedforward response to a stimulus or control of perception?

Best

Rick

···

--
Richard S. Marken PhD
www.mindreadings.com

The only thing that will redeem mankind is cooperation.
                                                   -- Bertrand Russell

[MartinT]

[Martin Taylor 2014.02.03.23.24]

[From Rick Marken (2014.02.03.1630)]

Martin Taylor (2014.02.01.00.01) --

MT: The first way of prediction is to observe something that influences the variation of the variable of interest, "x". If you know, for example, that an old-fashioned steam locomotive can go "chuff-chuff-chuff" at intervals that only slowly increase or decrease according to the acceleration and speed of the train, you can control any variable that is disturbed by the "chuff" better than if you never can guess when the next "chuff" will occur. For example, some control system might interpose a shield whenever a "chuff" is expected.

  RM: The deployment of the shield can be a commanded output artificial
control system, which can have S-R components, but in a living system
it's inputs, not output, that are commanded.

As I proposed. The words "some control system" might have given you a clue.

MT: The second way of prediction, if the variable is continuously variable, is to use the variable's own statistics, such as for example, the velocity and acceleration.

  RM: I believe this is the kind of feedforward is contained in your
model that adds the derivative of the input variable to the perception
of the input variable, as per your equatoin:

p(t) = s(t) + k*(ds(t)/dt)

where is the the sensory input variable. When I implemented this model

You haven't mentioned on CSGnet or privately that you implemented this model. You did say that you implemented a quite different one, which you claimed to be the same. You told me that I had mis-specified the equations and substituted your own. I corrected you, but have not heard that you implemented the correct model.

I found what I expected:

Because you used a problem for which you already knew the answer would be what you want, rather than the problem to which that circuit was addressed.

This result seemed to anger you since you said I didn't know how to
properly compute (ds(t)/dt). But you never told me the proper way to
compute the derivative so I never got a chance to test it again.

Of course that wasn't what annoyed me, though it did depress me. What annoyed me was the way you changed the problem and then proclaimed from the rooftops that you had proved that i was wrong. As for computing the derivative, I told you in detail how to do it during the last go round of this particular bone of contention, and you refused. Why should I expect you to take any more notice this time?

There is no feedforward in your model.

There is, however, prediction. Previously you have equated the two. Now you don't. It makes discussion very confusing when the words in contention keep changing their meaning according to the phase of the moon.

MT: . The uncertainty of the variable's value at time t+tau is much reduced. Ignoring this fact turned out to be the reason why my analysis two or three years ago of the best possible precision of control actually gave an answer worse than the precision Bill achieved in a real-life tracking experiment. Including the prediction available from the velocity resolved the problem.

RM: You seem to be saying that a model that controlled s(t) +
k*(ds(t)/dt) accounted for actual tracking data better than a model
that controls just s(t) (the usual variable we put into our models).

That;'s not what I said. Read it again.

MT: Bill's Artificial Cerebellum learned the temporal statistics of a variable in order to improve its control performance, but the standard control loop does not.

RM: I don't think the Artificial Cerebellum did anything other than
use reorganization to find parameters of control systems that results
in best control (indicated by minimum error). The model knew nothing
about the temporal statistics of any of the variables involved in the
control process.

Oh, well. If you want to substitute your own version of something and call it the Artificial Cerebellum, I can't stop you. But that's another case of changing the meaning of words. I intended "Artificial Cerebellum" to mean only what Bill described. I double checked a few minutes ago to be sure I had remembered correctly.

MT: These two (or three?) ways of implementing feedforward all come down to different ways of implementing prediction. Whether any particular biological system uses any of them is a research question.

RM: But you didn't "implement" the control systems we are interested
in understanding; living control systems.

I have actually implemented two. I have a son and a daughter. I know of no other means to really implement a living control system.

  So you are dreaming up
feedforward schemes that simply can be implemented in devices you can
build; but there is no evidence that such systems have been
implemented in living control systems.

Maybe not, but that applies to almost anything in PCT. What we know for sure is that living systems control their perceptions, defined as internal variables subject to influence from incoming sensory data. We don't know that the perceptions are scalar, but we do know that nerves emit impulses, not neural currents. We don't know whether the continuous variation of potential within a neuron matters to the control of perceptions. There is no evidence that ANY of the constructs of HPCT exist, other than as functional models that quite well produce observable behaviour that matches some living system behaviour.

  The only thing we know that
living systems do that looks like "feedforward" is do things, like
running away, when they see other things; organisms seem to be able to
act as though they "anticipate" bad (or good) things happening in the
future and act accordingly. But this is an S-R view of what is going
on -- a very seductive one but an S-R view nevertheless.

Since everything that isn't described exactly the way you personally KNOW to be true is by definition an S-R view, I suppose feedforward (alias prediction) is S-R as well. But that's another example of changing the meanings of words to suit your argument of the moment.

Since we know that living systems, being closed loop control systems,
don't act in response to events,

Don't they really? If a control unit has a fixed reference value, it most definitely does act in response to events, and in a deterministic way. It does this ONLY because it controls its inputs.

  For example, what is going on when I see dark clouds
on the horizon and get an umbrella from the closet. Is it a
feedforward response to a stimulus or control of perception?

Yes.

Martin

[wmansell]

Hi Rick and Martin, I don't know whether this will fit with both of you or either of you, but this isthe way I hope to use the illuminations you are providing. I am taking Martin's descriptions of feed forward as observable data that the majority of (non-PCT) psychologist will take to be examples of feed forward and considering the various forms that they take, including the apparent predictive information that is contained in the data. I am then assuming that behaviour is the control of perception and assuming that Bill's description of PCT as it stands will explain the if we find the right component (e.g. hierachies, reorganisation, memory) in the right context. They generally seem to. But then I will consider if any tweaks seem necessary, given that behaviour is the control of perception....
Warren

···

Sent from my iPhone

On 4 Feb 2014, at 04:48, Martin Taylor <mmt-csg@MMTAYLOR.NET> wrote:

[Martin Taylor 2014.02.03.23.24]

[From Rick Marken (2014.02.03.1630)]

Martin Taylor (2014.02.01.00.01) --

MT: The first way of prediction is to observe something that influences the variation of the variable of interest, "x". If you know, for example, that an old-fashioned steam locomotive can go "chuff-chuff-chuff" at intervals that only slowly increase or decrease according to the acceleration and speed of the train, you can control any variable that is disturbed by the "chuff" better than if you never can guess when the next "chuff" will occur. For example, some control system might interpose a shield whenever a "chuff" is expected.

RM: The deployment of the shield can be a commanded output artificial
control system, which can have S-R components, but in a living system
it's inputs, not output, that are commanded.

As I proposed. The words "some control system" might have given you a clue.

MT: The second way of prediction, if the variable is continuously variable, is to use the variable's own statistics, such as for example, the velocity and acceleration.

RM: I believe this is the kind of feedforward is contained in your
model that adds the derivative of the input variable to the perception
of the input variable, as per your equatoin:

p(t) = s(t) + k*(ds(t)/dt)

where is the the sensory input variable. When I implemented this model

You haven't mentioned on CSGnet or privately that you implemented this model. You did say that you implemented a quite different one, which you claimed to be the same. You told me that I had mis-specified the equations and substituted your own. I corrected you, but have not heard that you implemented the correct model.

I found what I expected:

Because you used a problem for which you already knew the answer would be what you want, rather than the problem to which that circuit was addressed.

This result seemed to anger you since you said I didn't know how to
properly compute (ds(t)/dt). But you never told me the proper way to
compute the derivative so I never got a chance to test it again.

Of course that wasn't what annoyed me, though it did depress me. What annoyed me was the way you changed the problem and then proclaimed from the rooftops that you had proved that i was wrong. As for computing the derivative, I told you in detail how to do it during the last go round of this particular bone of contention, and you refused. Why should I expect you to take any more notice this time?

There is no feedforward in your model.

There is, however, prediction. Previously you have equated the two. Now you don't. It makes discussion very confusing when the words in contention keep changing their meaning according to the phase of the moon.

MT: . The uncertainty of the variable's value at time t+tau is much reduced. Ignoring this fact turned out to be the reason why my analysis two or three years ago of the best possible precision of control actually gave an answer worse than the precision Bill achieved in a real-life tracking experiment. Including the prediction available from the velocity resolved the problem.

RM: You seem to be saying that a model that controlled s(t) +
k*(ds(t)/dt) accounted for actual tracking data better than a model
that controls just s(t) (the usual variable we put into our models).

That;'s not what I said. Read it again.

MT: Bill's Artificial Cerebellum learned the temporal statistics of a variable in order to improve its control performance, but the standard control loop does not.

RM: I don't think the Artificial Cerebellum did anything other than
use reorganization to find parameters of control systems that results
in best control (indicated by minimum error). The model knew nothing
about the temporal statistics of any of the variables involved in the
control process.

Oh, well. If you want to substitute your own version of something and call it the Artificial Cerebellum, I can't stop you. But that's another case of changing the meaning of words. I intended "Artificial Cerebellum" to mean only what Bill described. I double checked a few minutes ago to be sure I had remembered correctly.

MT: These two (or three?) ways of implementing feedforward all come down to different ways of implementing prediction. Whether any particular biological system uses any of them is a research question.

RM: But you didn't "implement" the control systems we are interested
in understanding; living control systems.

I have actually implemented two. I have a son and a daughter. I know of no other means to really implement a living control system.

So you are dreaming up
feedforward schemes that simply can be implemented in devices you can
build; but there is no evidence that such systems have been
implemented in living control systems.

Maybe not, but that applies to almost anything in PCT. What we know for sure is that living systems control their perceptions, defined as internal variables subject to influence from incoming sensory data. We don't know that the perceptions are scalar, but we do know that nerves emit impulses, not neural currents. We don't know whether the continuous variation of potential within a neuron matters to the control of perceptions. There is no evidence that ANY of the constructs of HPCT exist, other than as functional models that quite well produce observable behaviour that matches some living system behaviour.

The only thing we know that
living systems do that looks like "feedforward" is do things, like
running away, when they see other things; organisms seem to be able to
act as though they "anticipate" bad (or good) things happening in the
future and act accordingly. But this is an S-R view of what is going
on -- a very seductive one but an S-R view nevertheless.

Since everything that isn't described exactly the way you personally KNOW to be true is by definition an S-R view, I suppose feedforward (alias prediction) is S-R as well. But that's another example of changing the meanings of words to suit your argument of the moment.

Since we know that living systems, being closed loop control systems,
don't act in response to events,

Don't they really? If a control unit has a fixed reference value, it most definitely does act in response to events, and in a deterministic way. It does this ONLY because it controls its inputs.

For example, what is going on when I see dark clouds
on the horizon and get an umbrella from the closet. Is it a
feedforward response to a stimulus or control of perception?

Yes.

Martin

[wmansell]

I think that the cloud and umbrella example really illustrates that we are talking about multi-componential phenomena here.
First, putting up an umbrella is a behaviour to control the perception of staying dry.
But, given it only takes a second to put up an umbrella, you wouldn't do it until the rain started anyway.
Maybe a better example is whether you would put up a canopy at you garden party if rain was predicted.
Now, the actual putting up of the canopy is all controlled perception and the question is rather whether one takes the prediction of the perception of rain as the perception of rain.
So, this one seems to be an imagination mode process.
However I am sure that other apparent feed forward examples use a different combination of PCT components....
Warren

···

Sent from my iPhone

On 4 Feb 2014, at 00:32, Richard Marken <rsmarken@GMAIL.COM> wrote:

[From Rick Marken (2014.02.03.1630)]

Martin Taylor (2014.02.01.00.01) --

MT: The first way of prediction is to observe something that influences the variation of the variable of interest, "x". If you know, for example, that an old-fashioned steam locomotive can go "chuff-chuff-chuff" at intervals that only slowly increase or decrease according to the acceleration and speed of the train, you can control any variable that is disturbed by the "chuff" better than if you never can guess when the next "chuff" will occur. For example, some control system might interpose a shield whenever a "chuff" is expected.

RM: The deployment of the shield can be a commanded output artificial
control system, which can have S-R components, but in a living system
it's inputs, not output, that are commanded. So the analogous
situation in a living control system would involve controlling the
perception of the relationship between one sequence (shield
deployments) and another (chuffs). In living systems, apparent
feedforward control -- where outputs seem to anticipate disturbances
to inputs -- has to involve control of perception, not output.

MT: The second way of prediction, if the variable is continuously variable, is to use the variable's own statistics, such as for example, the velocity and acceleration.

RM: I believe this is the kind of feedforward is contained in your
model that adds the derivative of the input variable to the perception
of the input variable, as per your equatoin:

p(t) = s(t) + k*(ds(t)/dt)

where is the the sensory input variable. When I implemented this model
I found what I expected: the model controls the perception s(t) +
k*(ds(t)/dt), which resulted in increasingly poor control of s(t) as k
increased from 0; in other words, control of s(t) was best when no
derivative was added to s(t), which is not a controlled variable
anyway because the system cannot perceive it; it perceives s(t) +
k*(ds(t)/dt).

This result seemed to anger you since you said I didn't know how to
properly compute (ds(t)/dt). But you never told me the proper way to
compute the derivative so I never got a chance to test it again. I
don't really know why you got so upset about this but I suspect it's
because you think of the derivative, (ds(t)/dt) as feedforward that
should improve control of s(t). So if you tell me the correct way to
compute the derivative I'll check it out. But I don't believe that any
change in the computation of (ds(t)/dt) will help; what you showed me
were the equations for a feedback control model that controls s(t) +
k*(ds(t)/dt). There is no feedforward in your model.

MT: . The uncertainty of the variable's value at time t+tau is much reduced. Ignoring this fact turned out to be the reason why my analysis two or three years ago of the best possible precision of control actually gave an answer worse than the precision Bill achieved in a real-life tracking experiment. Including the prediction available from the velocity resolved the problem.

RM: You seem to be saying that a model that controlled s(t) +
k*(ds(t)/dt) accounted for actual tracking data better than a model
that controls just s(t) (the usual variable we put into our models).
That's quite a surprise; do you have the data and model results to
prove it? I'd really like to see that!

MT: Bill's Artificial Cerebellum learned the temporal statistics of a variable in order to improve its control performance, but the standard control loop does not.

RM: I don't think the Artificial Cerebellum did anything other than
use reorganization to find parameters of control systems that results
in best control (indicated by minimum error). The model knew nothing
about the temporal statistics of any of the variables involved in the
control process.

MT: These two (or three?) ways of implementing feedforward all come down to different ways of implementing prediction. Whether any particular biological system uses any of them is a research question.

RM: But you didn't "implement" the control systems we are interested
in understanding; living control systems. So you are dreaming up
feedforward schemes that simply can be implemented in devices you can
build; but there is no evidence that such systems have been
implemented in living control systems. The only thing we know that
living systems do that looks like "feedforward" is do things, like
running away, when they see other things; organisms seem to be able to
act as though they "anticipate" bad (or good) things happening in the
future and act accordingly. But this is an S-R view of what is going
on -- a very seductive one but an S-R view nevertheless.

Since we know that living systems, being closed loop control systems,
don't act in response to events, we have to revise our thinking about
what we are seeing when we see organisms apparently acting in
anticipation of something. Since we know that control systems only act
to protect controlled variables from disturbance, what we must be
seeing when we see apparent "predictive" or "feedforward" control must
be action aimed at protecting a controlled variable from disturbance.
Think about this the next time you see what appears to be predictive
control actions. For example, what is going on when I see dark clouds
on the horizon and get an umbrella from the closet. Is it a
feedforward response to a stimulus or control of perception?

Best

Rick
--
Richard S. Marken PhD
www.mindreadings.com

The only thing that will redeem mankind is cooperation.
                                                  -- Bertrand Russell