Stabilizing control by adding rate to the input function

[Abbott]

[From Bruce Abbott (2016.11.10.1555 EST)

I thought that PCTers might appreciate a short tutorial on the effect of adding rate input to an integral control system.

In the June 1979 issue of Byte magazine, Bill Powers presented the first in a series of articles ostensibly on “The Nature of Robots” but actually introducing to Byte readers what later came to be known as PCT. This first installment described a scenario in which “Chip” is trying to keep his car in the center of its lane while a steady crosswind exerts a force tending to push the car left or right. Chip’s means of doing so is to change the angle of his steering wheel, which in turn changes the angle of the front wheels, which generates a force tending to make the car drift to the right or left.

Bill presented a short program written in BASIC that implemented Chip’s control system. Most of the details that would be included in a fully fleshed out simulation are omitted; for example, the wind speed and steering angles are treated as if these were forces acting on the car rather than inputs to an environmental function that would compute the forces being developed as a result of these inputs. However, this crude level of representation is sufficient to produce a working model illustrating the basic principles.

In the simulation, wind and steering angle values add to produce the net effect, which is to cause the car to drift left or right depending on whether the result is positive or negative. Drift is the change in position per iteration of the simulation loop and thus qualifies as a rate. Each time through the loop, the drift is added to the car’s previous position to yield the new position. Position thus arises from the integration of drift across iterations. The control system varies the drift rate (via the steering angle) to control the car’s position.

A problem arises due to the lag introduced by the drift integration. Lag produces a phase shift that can yield instability. To help compensate for this lag, Bill added a term to the perceptual input function that is proportional to drift, i.e., the rate of change in position. Thus, Chip’s perception of the car’s position in the lane is affected not only by the car’s actual position but also by the rate at which it is changing position.

In a way, this rate factor can be viewed as a prediction of the car’s future position (on the next iteration), based on its current position and the rate at which that position is changing. Using the “predicted” position rather than the actual one creates a small phase advance that helps to overcome the phase delay due to integration (or to loop transport delay, for that matter).

The figure below presents the results of the simulation, with Chip’s perception of the car’s position equal to its actual position plus 0.8 times the drift.

In the simulation, a constant 20 mph crosswind is pushing the car toward the right (positive drift). After overcorrecting a bit, Chip settles down to maintaining a wheel angle of -10 degrees and a deviation from lane center of 5 feet (Drift = 0.5*wind + steering angle). At this point drift becomes zero.

This performance was achieved with rate input added to position input. How would this system perform if rate compensation were eliminated, all else being equal? The figure below shows the result:

The control system now displays a marked degree of instability, with several under- and overshoots in evidence before the system finally settles down. (An envelope drawn across the peaks would form a decaying exponential function.) This demonstrates that the added rate input that was present in the first illustration does indeed help to offset the integration lag.

There is good evidence that the control of perceived muscle length similarly includes an added rate component, with both length and rate of change in length being provided as input signals to the alpha motor neurons that drive the muscles. The alpha motor neurons thus play the roles of both comparator (with reference signals being input from higher levels in the nervous system) and output function driving muscle contraction.

NOTE: The Byte article from which this simulation was taken is available on Dag Forssell’s Living Control Systems website at

http://www.livingcontrolsystems.com/intro_papers/index.html .

Bruce

[MartinT]

[From Bruce Abbott (2016.11.10.1555 EST)

      I thought that PCTers might appreciate a

short tutorial on the effect of adding rate input to an
integral control system.

In the June 1979 issue of Byte
magazine, Bill Powers presented the first in a series of
articles ostensibly on “The Nature of Robots” but actually
introducing to Byte readers what later came to be known as
PCT. This first installment described a scenario in which
“Chip” is trying to keep his car in the center of its lane
while a steady crosswind exerts a force tending to push the
car left or right. Chip’s means of doing so is to change the
angle of his steering wheel, which in turn changes the angle
of the front wheels, which generates a force tending to make
the car drift to the right or left.

      Bill presented a short program written in

BASIC that implemented Chip’s control system. Most of the
details that would be included in a fully fleshed out
simulation are omitted; for example, the wind speed and
steering angles are treated as if these were forces acting on
the car rather than inputs to an environmental function that
would compute the forces being developed as a result of these
inputs. However, this crude level of representation is
sufficient to produce a working model illustrating the basic
principles.

      In the simulation, wind and steering angle

values add to produce the net effect, which is to cause the
car to drift left or right depending on whether the result is
positive or negative. Drift is the change in position per
iteration of the simulation loop and thus qualifies as a
rate. Each time through the loop, the drift is added to the
car’s previous position to yield the new position. Position
thus arises from the integration of drift across iterations.
The control system varies the drift rate (via the steering
angle) to control the car’s position.

      A problem arises due to the lag introduced

by the drift integration. Lag produces a phase shift that can
yield instability. To help compensate for this lag, Bill
added a term to the perceptual input function that is
proportional to drift, i.e., the rate of change in position.
Thus, Chip’s perception of the car’s position in the lane is
affected not only by the car’s actual position but also by the
rate at which it is changing position.

      In a way, this rate factor can be viewed as

a prediction of the car’s future position (on the next
iteration), based on its current position and the rate at
which that position is changing. Using the “predicted”
position rather than the actual one creates a small phase
advance that helps to overcome the phase delay due to
integration (or to loop transport delay, for that matter).

      The figure below presents the results of

the simulation, with Chip’s perception of the car’s position
equal to its actual position plus 0.8 times the drift.

      In the simulation, a constant 20 mph

crosswind is pushing the car toward the right (positive
drift). After overcorrecting a bit, Chip settles down to
maintaining a wheel angle of -10 degrees and a deviation from
lane center of 5 feet (Drift = 0.5*wind + steering angle).
At this point drift becomes zero.

      This performance was achieved with rate

input added to position input. How would this system perform
if rate compensation were eliminated, all else being equal?
The figure below shows the result:

      The control system now displays a marked

degree of instability, with several under- and overshoots in
evidence before the system finally settles down. (An envelope
drawn across the peaks would form a decaying exponential
function.) This demonstrates that the added rate input that
was present in the first illustration does indeed help to
offset the integration lag.

      There is good evidence that the control of

perceived muscle length similarly includes an added rate
component, with both length and rate of change in length being
provided as input signals to the alpha motor neurons that
drive the muscles. The alpha motor neurons thus play the
roles of both comparator (with reference signals being input
from higher levels in the nervous system) and output function
driving muscle contraction.

NOTE: The Byte article from which
this simulation was taken is available on Dag Forssell’s * Living
Control Systems* website at

http://www.livingcontrolsystems.com/intro_papers/index.html
.

Bruce

[Abbott]

[From Bruce Abbott (3026.11.10.2030 EST)]

Martin Taylor 2016.11.10.17.27 –

MT: Good find, Bruce. I just read the Byte articles again (I have those Byte issues), and checked that it was in fact Bill, not you, who used the derivative in that program and described it as a predictor useful for mitigating the effects of transport lag and integrator lag. Your message was ambiguous on that point. It doesn’t matter to the science who included the derivative in the program, but the fact that it was Bill protects you now from being heralded (or excoriated) again as a deviant from PCT and a prime advocate of S-R behaviourism. In future, when I want to use exactly that form (a form I had thought I got from Rick, but which he disclaims), I need only refer back to the first Byte article.

MT :Incidentally, the Byte articles were from 1979, whereas B:CP was 1973, though I guess the actual PCT label may not have been used in B:CP. I’m not going to read all of B:CP to see whether the triple “P-C-T” is there to check

BA: It is my impression that the PCT label came later, after a couple of other labels were tried and discarded.

BA: I thought I had made it clear that I was describing Bill’s simulation from the Byte article, including the use of the derivative in the input function, but perhaps not. But I will take this opportunity to clarify some points relating to the BASIC code Bill presented to implement his simulation of “Chip’s” driving.

Bill was programming in Northstar BASIC, which limited variable names to one or two characters, all caps. Today we are thankfully able to use longer and more descriptive names. For those who wonder what Bill’s code looked like and how it functioned, I have included the code below, followed by commentary.

The program is deceptively simple, just 13 lines of code:

1 INPUT "WIND, MPH: ",W

2 PRINT “WHEEL ANGLE, DEGREES”,

3 PRINT TAB(25),“CAR DEVIATION, FEET”

4 FOR J=1 TO 10

5 PRINT %7F1," ",A*10,

6 PRINT %7F1,TAB(25),I

7 D=.05*W+A

8 I=I+D

9 A1= –2*(I+.8*D)

10 A=A+.200*(A1–A)

11 NEXT

12 GOTO 1

13 END

Line 1 asks the user to input the wind speed and stores the value in W. Lines 2 and 3 then print the initial wheel angle and car deviation (from lane center), both variables initialized implicitly to zero. Line 4 then starts the simulation loop, which in this case is set to repeat ten times. Lines 5 and 6 print the wheel angle A and position I. The code that performs the simulation calculations then begins on line 7 with the FOR loop.

The variable D is the deviation of the car from its initial position due to the effect of wind (W) and steering angle (A). Wind acts as a disturbance to car position. Note that the wind (W) has one twentieth the effect of steering angle (A): D = 0.05*W + A.

Line 8 computes the car’s position (lane center is position 0) by adding the deviation computed on line 7 to the current position (I). Each time through the loop the new deviation will be added to I to produce a new position; thus I is the integral (sum) of all the deviations up to this iteration of the loop. Note that D is the rate of change in position per iteration.

What follows on the next two lines is a little tricky to understand. Line 9 computes the value of the variable A1 by adding the current position I and the rate of change in position D after first multiplying D by 0.8. The result is then multiplied by -2. Line 10 then presents another integrator function. The new value of A on this iteration is computed by adding the current value of A to the difference between A1 and the current value of A after multiplying that difference by 0.2. The NEXT statement on line 11 then sends us back to the start of the loop with J = 2. This loop repeats 10 times, after which line 12 is executed, which sends us back to the beginning of the program where we are asked to input another initial wind velocity. (We can keep trying different wind directions and speeds; the program will end only if we Esc out of it.)

So what’s going on in lines 9 and 10? It may not be clear because Bill has collapsed some functions into these two statements that normally would be represented separately. In line 9, the portion in parentheses normally would be considered two separate inputs to the perceptual input function. The output of this function is a perception of vehicle position that is influenced by the vehicle’s rate of change in position as well as position itself. The “0.8*D is the derivative of position, weighted by the constant factor 0.8, thus giving the derivative a little less influence in determining the perception than position itself does.

This perception of position (relative to lane center) is then subtracted from the reference value of 0 deviation from lane center. You don’t see the reference signal? It’s not explicitly represented! If you subtract p from an r of 0, you get p again, with a sign change. The difference is the error, and this is multiplied by the output gain, which is 2.

Line 10 thus reduces to the standard PCT “leaky bucket” output function, i.e., output = output + slowing factor*(ke – output). In terms of the variables used in the simulation, this translates to A = A + 0.200*(A1 – A) where 0.200 is the slowing factor. Line 10 thus implements the standard leaky integrator used to prevent the introduction of a digital artifact that occurs if the output is allowed to change fully to its new value during a single iteration. (Instead, it approaches that value exponentially at a rate determined by the slowing factor.)

To represent the system more clearly, we might reorganize the code as follows:

7 D = 0.05W + A // deviation (rate of change in position) determined by wind disturbance and steering angle
8 I = I + D // integrates deviations to yield current position
9 P = I + 0.8D // perceptual input function; 0.8 is a weighting factor
10 E = R – P // R set to zero before entering the simulation loop; E thus equals -P.
11 A = A + 0.2*(2.0*E – A) // output function; 0.2 = slowing; 2 = output gain

You may have noticed the absence of dt, the time elapsing with each iteration of the loop. To keep the simulation simple, Bill in effect made dt equal to 1.0 and did not include it explicitly. If you used another dt it would have to be included in the rate computations, e.g., D multiplied by dt (say 0.01 seconds/iteration) to yield D as deviation per second rather than deviation per iteration. (The same goes for the deviation represented within the parentheses in the output function.)

Nature often combines functions similar to the way Bill did in his version of the program. As I noted last time, alpha motor neurons carry out the functions of perceptual input and comparator. The motor neuron’s output can be considered an error signal that drives the output (muscle contraction) via the neuromuscular junctions. Unfortunately, evolution is more of a tinkerer than a designer, always working with what it currently has and modifying it a bit to suit new purposes. The resulting structures and their functions can be difficult to disentangle, but as with my revision of Bill’s code, it is always possible to create an alternate structure that performs identically to the original but cleanly separates their functions to make them and their connections clear.

Bruce

[rsmarken]

[From Rick Marken (2016.11.10.1910)]

···

 Bruce Abbott (2016.11.10.1555 EST)–

Â

BA: I thought that PCTers might appreciate a short tutorial on the effect of adding rate input to an integral control system.

Â

BA: In the June 1979 issue of Byte magazine, Bill Powers presented the first in a series of articles ostensibly on “The Nature of Robotsâ€? but actually introducing to Byte readers what later came to be known as PCT.

RM: Thanks for the nice analysis and the trip down memory lane (I came across the Byte articles after I had already read Bill’s 1978 Psych Review paper and I was already programming control demos in Basic on an Apple II but the Byte articles were a real treasure). I think it would be interesting to see if adding the derivative to the input improves control when systems are controlling something other than position. Would it help if the controlled variable were more complex, like an area? Or “simpler”, like a rate change?Â

RM: By the way, Â I believe Kent McClelland is responsible for coming up with the name “Perceptual Control Theory” to distinguish Powers’ application of control theory to behavior from others. Kent would probably recall when he came up with the name better than I can but I suspect it was in the late 1980s or early 1990s.Â

BestÂ

Rick

Â

 This first installment described a scenario in which “Chipâ€? is trying to keep his car in the center of its lane while a steady crosswind exerts a force tending to push the car left or right. Chip’s means of doing so is to change the angle of his steering wheel, which in turn changes the angle of the front wheels, which generates a force tending to make the car drift to the right or left.

Â

Bill presented a short program written in BASIC that implemented Chip’s control system. Most of the details that would be included in a fully fleshed out simulation are omitted; for example, the wind speed and steering angles are treated as if these were forces acting on the car rather than inputs to an environmental function that would compute the forces being developed as a result of these inputs. However, this crude level of representation is sufficient to produce a working model illustrating the basic principles.

Â

In the simulation, wind and steering angle values add to produce the net effect, which is to cause the car to drift left or right depending on whether the result is positive or negative. Drift is the change in position per iteration of the simulation loop and thus qualifies as a rate. Each time through the loop, the drift is added to the car’s previous position to yield the new position. Position thus arises from the integration of drift across iterations. The control system varies the drift rate (via the steering angle) to control the car’s position.

Â

A problem arises due to the lag introduced by the drift integration. Lag produces a phase shift that can yield instability. To help compensate for this lag, Bill added a term to the perceptual input function that is proportional to drift, i.e., the rate of change in position. Thus, Chip’s perception of the car’s position in the lane is affected not only by the car’s actual position but also by the rate at which it is changing position.Â

Â

In a way, this rate factor can be viewed as a prediction of the car’s future position (on the next iteration), based on its current position and the rate at which that position is changing. Using the “predictedâ€? position rather than the actual one creates a small phase advance that helps to overcome the phase delay due to integration (or to loop transport delay, for that matter).

Â

The figure below presents the results of the simulation, with Chip’s perception of the car’s position equal to its actual position plus 0.8 times the drift.

Â

In the simulation, a constant 20 mph crosswind is pushing the car toward the right (positive drift). After overcorrecting a bit, Chip settles down to maintaining a wheel angle of -10 degrees and a deviation from lane center of 5 feet (Drift = 0.5*wind + steering angle). At this point drift becomes zero.

Â

This performance was achieved with rate input added to position input. How would this system perform if rate compensation were eliminated, all else being equal? The figure below shows the result:

Â

The control system now displays a marked degree of instability, with several under- and  overshoots in evidence before the system finally settles down. (An envelope drawn across the peaks would form a decaying exponential function.) This demonstrates that the added rate input that was present in the first illustration does indeed help to offset the integration lag.

Â

There is good evidence that the control of perceived muscle length similarly includes an added rate component, with both length and rate of change in length being provided as input signals to the alpha motor neurons that drive the muscles. The alpha motor neurons thus play the roles of both comparator (with reference signals being input from higher levels in the nervous system) and output function driving muscle contraction.

Â

NOTE: The Byte article from which this simulation was taken is available on Dag Forssell’s Living Control Systems website at

http://www.livingcontrolsystems.com/intro_papers/index.html .

Â

Bruce

–

Richard S. MarkenÂ

“The childhood of the human race is far from over. We
have a long way to go before most people will understand that what they do for
others is just as important to their well-being as what they do for
themselves.” – William T. Powers

[Abbott]

[From Bruce Abbott (2016.11.11.0855 EST)]

Rick Marken (2016.11.10.1910) –

Bruce Abbott (2016.11.10.1555 EST)–

BA: I thought that PCTers might appreciate a short tutorial on the effect of adding rate input to an integral control system.

BA: In the June 1979 issue of Byte magazine, Bill Powers presented the first in a series of articles ostensibly on “The Nature of Robots� but actually introducing to Byte readers what later came to be known as PCT.

RM: Thanks for the nice analysis and the trip down memory lane (I came across the Byte articles after I had already read Bill’s 1978 Psych Review paper and I was already programming control demos in Basic on an Apple II but the Byte articles were a real treasure). I think it would be interesting to see if adding the derivative to the input improves control when systems are controlling something other than position. Would it help if the controlled variable were more complex, like an area? Or “simpler”, like a rate change?

BA: In 1979 I read the Byte articles and ran a couple of the demos on the PDP-8F minicomputer (about the size of a microwave oven!) that I was given the use of shortly after being hired into the Psych Department at IPFW. I had purchased and read B:CP while still in grad school, around 1975 or so.

BA: Adding the derivative to the input function should improve control whenever significant lag is present in the loop, no matter what the controlled variable is or how complex. The derivative adds a bit of phase advance that reduces the lag.  Consider the case of a system controlling, say, the speed of an automobile by varying the car’s acceleration. You get speed by integrating acceleration, and the integration introduces integral lag. Adding the derivative of speed to the input will reduce this lag.

RM: By the way, I believe Kent McClelland is responsible for coming up with the name “Perceptual Control Theory” to distinguish Powers’ application of control theory to behavior from others. Kent would probably recall when he came up with the name better than I can but I suspect it was in the late 1980s or early 1990s.

BA: Ah, HA! Now that you mention it, I seem to remember Kent being credited with coming up with the name.

Bruce

[Kent]

[From Kent McClelland (2016.11.11.1530)]

RM: By the way, I believe Kent McClelland is responsible for coming up with the name “Perceptual Control Theory” to distinguish Powers’ application of control theory to behavior from
others. Kent would probably recall when he came up with the name better than I can but I suspect it was in the late 1980s or early 1990s.

BA: Ah, HA! Now that you mention it, I seem to remember Kent being credited with coming up with the name.

Bruce

KM: Yes, I was the first to use the name "Perceptual Control Theory,� as well as the acronym PCT, in a 1990 paper I wrote as I was attempting to understand the theory for myself and to speculate on how it might be related to sociology. I took
this paper to the 1991 CSG conference and eventually published a later draft of it under the title, “Perceptual Control and Social Power� in 1994.

KM: For the record, my recollection is that Gary Cziko was the one who started the discussion on CSGnet about what to call Bill’s theory sometime in 1991 or early 1992. Cziko liked PCT name and strongly advocated for adopting it, and he was also
the first person to publish a paper that used "Perceptual Control Theory� to describe the theory. His paper was published in an education journal in 1992.

KM: The CSGnet discussion about a name for the theory was somewhat contentious, as I recall it, and Bill’s preference was for “Hierarchical Control Theory� or HCT. As a sociologist, I didn’t particularly like that name, because it made it sound
like the theory might refer to social hierarchies, rather than to the structure of the brain. In retrospect today, I wouldn’t want to argue that "Perceptual Control Theory� was the perfect name to have chosen, but it’s the one that stuck.

KM: Thanks, Bruce, for all the information you’re providing about the models Bill presented in the 1979 Byte articles. Those articles helped me enormously in the process of trying to figure out what the theory was all about. I had already read
Behavior: The Control of Perception, but those articles explained how you could build a working model of the theory, and they made it all much clearer in my mind. I’d certainly recommend them to anyone who hasn’t read them.

Kent

···

On Nov 11, 2016, at 7:55 AM, Bruce Abbott bbabbott@frontier.com wrote:

[Abbott]

[From Bruce Abbott (2016.11.11.1825 EST)]

Kent McClelland (2016.11.11.1530)] –

RM: By the way, I believe Kent McClelland is responsible for coming up with the name “Perceptual Control Theory” to distinguish Powers’ application of control theory to behavior from others. Kent would probably recall when he came up with the name better than I can but I suspect it was in the late 1980s or early 1990s.

BA: Ah, HA! Now that you mention it, I seem to remember Kent being credited with coming up with the name.

Bruce

KM: Yes, I was the first to use the name "Perceptual Control Theory,â€? as well as the acronym PCT, in a 1990 paper I wrote as I was attempting to understand the theory for myself and to speculate on how it might be related to sociology. I took this paper to the 1991 CSG conference and eventually published a later draft of it under the title, “Perceptual Control and Social Powerâ€? in 1994.

KM: For the record, my recollection is that Gary Cziko was the one who started the discussion on CSGnet about what to call Bill’s theory sometime in 1991 or early 1992. Cziko liked PCT name and strongly advocated for adopting it, and he was also the first person to publish a paper that used "Perceptual Control Theoryâ€? to describe the theory. His paper was published in an education journal in 1992.

KM: The CSGnet discussion about a name for the theory was somewhat contentious, as I recall it, and Bill’s preference was for “Hierarchical Control Theoryâ€? or HCT. As a sociologist, I didn’t particularly like that name, because it made it sound like the theory might refer to social hierarchies, rather than to the structure of the brain. In retrospect today, I wouldn’t want to argue that "Perceptual Control Theoryâ€? was the perfect name to have chosen, but it’s the one that stuck.

KM: Thanks, Bruce, for all the information you’re providing about the models Bill presented in the 1979 Byte articles. Those articles helped me enormously in the process of trying to figure out what the theory was all about. I had already read Behavior: The Control of Perception, but those articles explained how you could build a working model of the theory, and they made it all much clearer in my mind. I’d certainly recommend them to anyone who hasn’t read them.

BA: And thanks, Kent, for posting those recollections about how PCT got its name. As for the Byte articles, I second your recommendation. Even if one does not program and run the demos, Bill’s lucid descriptions of them and the effects of varying system parameters are well worth the read.

BA: Thanks also to Dag Forssell, who has worked tirelessly to provide access to PCT-relevant articles and demos on his Living Control Systems website (http://www.livingcontrolsystems.com/index.html ), including the four-installment series from Byte and the extensive CSGnet archive, among many others.

Bruce

···

On Nov 11, 2016, at 7:55 AM, Bruce Abbott bbabbott@frontier.com wrote: