Hierarchical Control (was Radius of Curvature)

From Bruce Abbott (2017.11.15.1435 EST)]

[From Erling Jorgensen (2017.11.15 1212 EST)]

Erling Jorgensen (2017.11.15 0925 EST)

Martin Taylor 2017.11.1511.29

EJ: Just a clarification.

EJ: But in a hierarchical PCT arrangement for controlling Position, don’t all of these implementing layers below that relationship level of Position become pursuit tracking tasks in their own right, since references for changing each of those implementing perceptions are also being generated?

MT: Not pursuit tracking, which implies chasing an ever-changing target. Here, only the reference values for the perceptions are being varied.

EJ: As I am visualizing it, the ever-changing reference values are the target that the perceptions are chasing. That is why a tracking task is a paradigmatic model for examining control, regardless of the type of control involved. It may represent relational position, degree of loudness when listening to music, an acceptable rate of return on an investment, the extent to which one’s sense of self is being portrayed genuinely. All of those are matching tasks, where a current perception is chasing a preferred reference for that perception.

EJ: If the perception is being pushed away from an already acceptable preference state, we call that compensatory tracking. But if the reference signal is leading the way in a changing manner, isn’t that pursuit tracking? I realize the distinction is a little artificial, because control is never perfect, so adjustments are common with either method.

BA: Technically speaking, pursuit tracking involves “pursuing” a variable that is external to the control system, such as the position of a target on a screen. We can set a reference for the distance between the target and the cursor and then move the cursor in a way that reduces that distance. (As Bill Powers noted, the reference distance does not have to be zero, although it typically is. For example, you could try to keep the cursor 1 cm below the target.)

BA: On the other hand, when the reference is varying over time, the control system will attempt to make the perception match the reference, making the perception “pursue” the reference. Control systems designed to do this are called “servomechanisms,” or “servos” for short.

BA: As for hierarchical control, a nice example is provided by the inverted pendulum demo that was created for LCS III. It implements a three-level hierarchy: A deviation of the pendulum bob from its reference position changes the reference of a velocity control. Deviations from this reference determine the reference for an acceleration control. The output of the acceleration control is a force, which acts on the cart and accelerates it according to the cart’s mass. Acceleration of the cart acts through the pendulum’s rod to accelerate the bob.

BA: This arrangement produces some interesting and perhaps unexpected behavior. In the demo, the user can move the bob position reference along the X axis. If before the change the bob had been balanced in the inverted position, what would you expect to happen? If the new reference position is to the right of the bob’s current position, one would think that the system would begin by accelerating the cart in that direction. However, the vertical rod would fail to transfer this motion to the bob, so the cart would begin to move out from beneath the bob. This would cause the bob to begin falling toward the left, and this motion would then be increased by the horizontal component of the force acting on the bob through its connection to the cart. The pendulum would simply fall over.

BA: But this is not what happens! Instead, one observes the cart accelerating to the left. The bob is no longer directly over the cart; instead, the pendulum is starting to lean to the right. As the rightward lean reaches a certain angle, the cart reverses direction, moving rightward at just the right speed to keep the bob at about that angle. As the cart catches up the new reference position, it speeds up slightly. This allows the cart to get under the bob again and then pass it, so that the pendulum is now leaning slightly leftward. In this position a breaking force is applied to the cart, slowing the movements of both the bob and cart. As the bob comes to a stop at the new reference position, so does the cart, directly under the bob and thus in position to keep the inverted pendulum balanced. The system will now move the cart slightly left or right as needed to offset the effect of any disturbance to bob position.

BA: On the surface it looks as though a relatively complex set of “if-then” commands have been executed to produce the observed movements, (e.g., if the reference moves right, then initially move the cart to the left.) In fact it is all accomplished without any such programming, simply through the actions of the pendulum’s hierarchical control system.

BA: Bill’s Little Man demo implements a similar hierarchical system to move Little Man’s finger into the target triangle. The bell-shaped pattern of velocities that emerges when the arm moves from one target position to another is no doubt a byproduct of this hierarchical control. (For example, to prevent overshoot, one must begin to apply a breaking force to the appropriate parts of the arm well before the finger-tip arrives at the target. As the finger closes in on the target position, the positional error diminishes, reducing the velocity reference, reducing the acceleration reference, reducing the force exerted by the muscles that are driving the arm in the direction of the target.)

BA: It should be noted that this form of hierarchical control is not identical to the hierarchy of perceptions described by HPCT. If perception of a position is a Level 1 (intensity) perception, it cannot then be said that velocity belongs to Level 2, otherwise velocity would have to qualify as a sensation and be “composed” of two or more Level 1 intensities, and so on. However, the hierarchical control principle is enforced: The next level up sets the reference for the level below, and the outputs of the lower level act as the means by which the level above it controls its own perception.

Bruce

[From Erling Jorgensen (2017.11.16 0730 EST)]

Bruce Abbott (2017.11.15.1435 EST)

Hi Bruce,

BA: As for hierarchical control, a nice example is provided by the inverted pendulum demo that was created for LCS III. It implements a three-level hierarchy: A deviation of the pendulum bob from its reference position changes the reference of a velocity control. Deviations from this reference determine the reference for an acceleration control. The output of the acceleration control is a force, which acts on the cart and accelerates it according to the cart’s mass. Acceleration of the cart acts through the pendulum’s rod to accelerate the bob.

EJ: So I would describe the levels this way: Applied force controls cart acceleration, which controls bob velocity, which controls bob position (which I see as a relationship perception comprised of distance of current bob position from bob reference.) Do you agree that all three levels are being controlled, even though only the top one has a fixed reference value?

BA: This arrangement produces some interesting and perhaps unexpected behavior. In the demo, the user can move the bob position reference along the X axis. If before the change the bob had been balanced in the inverted position, what would you expect to happen? If the new reference position is to the right of the bob’s current position, one would think that the system would begin by accelerating the cart in that direction. …[snip]

BA: But this is not what happens! Instead, one observes the cart accelerating to the left. The bob is no longer directly over the cart; instead, the pendulum is starting to lean to the right. As the rightward lean reaches a certain angle, the cart reverses direction, moving rightward at just the right speed to keep the bob at about that angle.

EJ: I had forgotten about this aspect of the cart’s behavior! And even knowing that a PCT control system simply monitors its one perception and automatically alters its output as references change from above, I can still feel the pull of this (misplaced) question: How does the cart “know” to first accelerate to the left, when the bob reference shifts right? It reminds me of the counterintuitive results of control system operation, and the corresponding difficulty people can have in understanding them. It makes some sense of the leap others make to introduce those if-then programming statements into the process, instead of trusting that a simpler mechanism can make it come out okay!

EJ: As far as I know there is no indication of “memory” built into the Inverted Pendulum system, so the cart has not “learned” which way to go. I haven’t examined the numbers in the system for a long time. But I do suspect the velocity bob system is never quite at a dead stop, always having a bit of that residual error that PI controllers need to maintain control. After all, when the bob arrives at its proper reference, the lower level systems do not stop working, they keep working!

EJ: You also mention above a piece I was not quite aware of: “As the rightward lean reaches a certain angle, the cart reverses direction, moving rightward at just the right speed to keep the bob at about that angle.” Doesn’t this suggest an additional controlled variable? I.e., pendulum angle, with references that might range from 80 to 100 degrees from horizontal. I don’t remember seeing a control system for that perception, so is it just part of the dynamics of the bob velocity system? The key is that reversal of direction of the cart acceleration system. It seems there is some signal as to how much initiated velocity is too much velocity, (but here I am thinking digitally again, in terms of events signaling other events to happen.)

EJ: I have a further question. Once the cart has changed direction to now move right, and accelerated the cart enough to match the falling-velocity speed of the bob, it seems the distance to the reference would be getting progressively smaller. Wouldn’t this lead right away to the bob velocity needing to get progressively slower, at least by micro amounts, and thus the cart acceleration to speed up right away to overtake the bob in order to decrease its velocity by those small amounts? As I talk this out, I am realizing that any overtaking of the bob would make the pendulum more erect, and thus less susceptible to the simulated gravity, thus starting to adjust and reduce the bob velocity already, without having to yet surpass the bob position entirely to bring it to a full stop. Is this another situation where the parameters of the Integral portion of the outputs smooth out those dynamic changes with just the right timing?

EJ: Sorry to get into the weeds with all this. I do like the summary statement you make:

BA: On the surface it looks as though a relatively complex set of “if-then� commands have been executed to produce the observed movements, (e.g., if the reference moves right, then initially move the cart to the left.) In fact it is all accomplished without any such programming, simply through the actions of the pendulum’s hierarchical control system.

All the best,

Erling

···

Disclaimer: This message is intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, or the employer or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by telephone and delete the material from your computer. Thank you for your cooperation.

[From Bruce Abbott (2017.11.16.1150 EST)]

[From Erling Jorgensen (2017.11.16 0730 EST)]

Bruce Abbott (2017.11.15.1435 EST)

Hi Bruce,

BA: As for hierarchical control, a nice example is provided by the inverted pendulum demo that was created for LCS III. It implements a three-level hierarchy: A deviation of the pendulum bob from its reference position changes the reference of a velocity control. Deviations from this reference determine the reference for an acceleration control. The output of the acceleration control is a force, which acts on the cart and accelerates it according to the cart’s mass. Acceleration of the cart acts through the pendulum’s rod to accelerate the bob.

EJ: So I would describe the levels this way: Applied force controls cart acceleration, which controls bob velocity, which controls bob position (which I see as a relationship perception comprised of distance of current bob position from bob reference.) Do you agree that all three levels are being controlled, even though only the top one has a fixed reference value?

BA: Yes, all three levels are being controlled; however, the reference for the top one (Bob Position) is not necessarily fixed in the demo; the user can move it around dynamically during the simulation. As for there being only three control systems at work, I’m afraid I forgot about a couple (see below).

BA: This arrangement produces some interesting and perhaps unexpected behavior. In the demo, the user can move the bob position reference along the X axis. If before the change the bob had been balanced in the inverted position, what would you expect to happen? If the new reference position is to the right of the bob’s current position, one would think that the system would begin by accelerating the cart in that direction. …[snip]

BA: But this is not what happens! Instead, one observes the cart accelerating to the left. The bob is no longer directly over the cart; instead, the pendulum is starting to lean to the right. As the rightward lean reaches a certain angle, the cart reverses direction, moving rightward at just the right speed to keep the bob at about that angle.

EJ: I had forgotten about this aspect of the cart’s behavior! And even knowing that a PCT control system simply monitors its one perception and automatically alters its output as references change from above, I can still feel the pull of this (misplaced) question: How does the cart “know” to first accelerate to the left, when the bob reference shifts right? It reminds me of the counterintuitive results of control system operation, and the corresponding difficulty people can have in understanding them. It makes some sense of the leap others make to introduce those if-then programming statements into the process, instead of trusting that a simpler mechanism can make it come out okay!

EJ: As far as I know there is no indication of “memory” built into the Inverted Pendulum system, so the cart has not “learned” which way to go. I haven’t examined the numbers in the system for a long time. But I do suspect the velocity bob system is never quite at a dead stop, always having a bit of that residual error that PI controllers need to maintain control. After all, when the bob arrives at its proper reference, the lower level systems do not stop working, they keep working!

BA: Correct, there is no memory built into the Inverted Pendulum system. All the control systems are working all the time during the simulation, although it is possible for the system to come to a dead stop. This will happen when all the dynamic variables reach equilibrium values, and can happen because the simulation does not include small disturbances such as air currents that might act on the bob in a real inverted pendulum. Do not confuse “stop working� with “stop varying.� The system is still monitoring its inputs but producing no change in outputs when the system is in dynamic equilibrium.

EJ: You also mention above a piece I was not quite aware of: “As the rightward lean reaches a certain angle, the cart reverses direction, moving rightward at just the right speed to keep the bob at about that angle.” Doesn’t this suggest an additional controlled variable? I.e., pendulum angle, with references that might range from 80 to 100 degrees from horizontal. I don’t remember seeing a control system for that perception, so is it just part of the dynamics of the bob velocity system? The key is that reversal of direction of the cart acceleration system. It seems there is some signal as to how much initiated velocity is too much velocity, (but here I am thinking digitally again, in terms of events signaling other events to happen.)

BA: The system is not monitoring bob angle and controlling based on that angle. It’s just that when the various levels of control are properly tuned, the system is usually able to prevent the pendulum’s angle from exceeding a certain value. (If it didn’t, the bob would reach an angular velocity that exceeds the cart’s ability to accelerate enough to recover. This can happen under certain conditions.) Â

EJ: I have a further question. Once the cart has changed direction to now move right, and accelerated the cart enough to match the falling-velocity speed of the bob, it seems the distance to the reference would be getting progressively smaller. Wouldn’t this lead right away to the bob velocity needing to get progressively slower, at least by micro amounts, and thus the cart acceleration to speed up right away to overtake the bob in order to decrease its velocity by those small amounts? As I talk this out, I am realizing that any overtaking of the bob would make the pendulum more erect, and thus less susceptible to the simulated gravity, thus starting to adjust and reduce the bob velocity already, without having to yet surpass the bob position entirely to bring it to a full stop. Is this another situation where the parameters of the Integral portion of the outputs smooth out those dynamic changes with just the right timing?

BA: Unfortunately I forgot about the control systems that control perceptions of the cart’s position relative to the bob and the cart’s velocity. Rather than go into a verbal description of the whole system, I’ve elected to reproduce the Delphi code that implements the system’s dynamics below. This includes the environmental dynamics as well as the control systems. The code is well commented, so you should not have much trouble deciphering it even if you do not know the Delphi (pascal) programming language. (The system is also nicely described in the Inverted Pendulum chapter of LCS III and the mathematics behind it in Richard Kennaway’s appendix to the book.)

The first procedure (Dynamics) does require a little explaining. The pendulum pole is modeled as being a very stiff spring, so that it compressed very slightly by the acceleration of the cart relative to the bob ( and stretched slightly by the deceleration). This force is equal to the difference between the pole’s resting length L0 and it current length l, which is multiplied by the spring constant Ke. The forces acting on the bob that are directed into the pole as the cart changes velocity are found by computing the vertical and horizontal components of the forces acting on the bob, which include the effects of gravity, cart acceleration or deceleration, and an “impulse� force that the user can apply to the bob.

The simulation allows the user to observe what happens to the inverted pendulum when control is turned off, which explains why the procedure includes an “If Control� statement.

procedure Dynamics;Â // Environmental factors

begin

 with Bob do

 begin

   DeltaX := Bob.X - Cart.X;

   DeltaY := Bob.Y - Cart.Y;

   L := Sqrt(Sqr(DeltaX) + Sqr(DeltaY)); //Computed length of pole

   F := (L0 - l)*Ke;

   Fx := F*DeltaX/l + Impulse; //Horizontal force on bob

   Fy := FDeltaY/l - MassG; //Vertical force on bob

   Vx := Vx + Fx/Mass*dt; //Horizontal velocity

   X := X + (Vx + 0.5Fxdt/Mass)*dt; //Horizontal position

   Vy := Vy + Fy/Mass*dt; //Vertical velocity

   Y := Y + (Vy + 0.5Fydt/Mass)*dt; //Vertical position

 end;

 with Cart do

 begin

   if Control then Fx := -Bob.Fx + CartVel.O //Horizontal force on cart

   else Fx := -Bob.Fx;

   Vx := Vx + Fx/Mass*dt;

   X := X + (Vx + 0.5Fx/Massdt)*dt; //Horizontal position

 end;

end;

procedure CartVelCont;

begin

 with CartVel do

 begin

   P := Cart.Vx;

   R := CartPos.O; //Proportional to cart position error

   E := R - P;

   O := O + (100.0*E - O)/2.0; //Horizontal force on cart

 end;

end;

procedure CartPosCont;

begin

 with CartPos do

 begin

   P := Cart.X;

   R := BobAccel.o; //Proportional to error in cart pos relative to bob pos

   E := R - P;

   O := 50*E; //Connects to reference for cart velocity

 end

end;

procedure BobAccelCont;

begin

 with BobAccel do

 begin

   P := Bob.x - Cart.x;

   R := BobVel.O; //Proportional to error in bob horiz. velocity

   E := R - P;

   O := -50*E; //Connects to reference for cart position

 end

end;

procedure BobVelCont;

begin

 with BobVel do

 begin

   P := Bob.Vx;

   R := BobPos.O; //Proportional to error in bob position

   E := R - P;

   O := O + 10.0*(0.5*E - O)*dt; //Connects to reference for bob accel.

   if O > 0.4 then O := 0.4

   else if O < -0.4 then O := -0.4;

 end;

end;

procedure BobPosCont;

begin

 with BobPos do

 begin

   R := ScreenXtoX(MouseX); //Mouse position

   P := Bob.X;

   E := R - P;

   O := 2*E; //Connects to reference for bob velocity

 end;

end;

These procedures are contained in a loop that executes continually so long at the simulation is running. And that, my friend, is all there is to it!

Bruce