Self-Inverting Pendulum (was Reinforcement Learning)

Abbott · October 3, 2017, 4:53pm

[From Bruce Abbott (2017.10.03.1250 EDT)]

Bruce Abbott (2017.01.02.0945 EDT) –

Rupert Young (2017.10.02 12.15) –

(Bruce Abbott (2017.10.01.1645 EDT)]

BA: No learning is involved, neither reorganization nor RL. I simply altered the existing code from the original demo a very slight bit to make the pendulum self-inverting.

BA: If you download and run the program, you will see what self-inverting means. Try it!

BA: I’d be interested in hearing from anyone who has run this magical demo. Without looking at the source code, can you guess how self-inversion was achieved? The solution was so simple I was surprised when it actually worked!

RY: Awesome demo Bruce, I love it!

BA: Thanks!

RY: The trick is very neat, and interesting (though I had to look at the code). Is 0.8 the ground level?

BA: No, ground level is zero. The pendulum is one meter long, so when hanging straight down, the bob is at Y = -1.0, and +1.0 when standing straight up.

RY: Do you think it is valid, in terms of PCT, as it is introducing a perception by the back door?

BA: Not as currently implemented. But I don’t want to discuss this issue too much at this point, as I want to give others a chance at guessing the solution.

RY: I think that perception could be taken out into its own control system, which would then switch between two lower systems, in a similar way with the brake/throttle scenario?

BA: Yes, certainly, but at the cost of increased complexity, of course. Would we really need two nearly identical lower systems that differ only in you-know-what? Or could the higher system manipulate the lower system’s you-know-what directly? What perception would the higher system be controlling for?

BA: O.K., I’ve heard from Rupert, Rick, and Erling, but I’m still awaiting replies from Rick and Erling as to how they imagine that the self-inverting pendulum accomplishes its neat trick. After I’ve heard from everyone, I’ll reveal the secret.

Also, are these three the only subscribers to CSGnet who have tried the demo? It’s a quick download, even on my slow internet connection (a matter of a few seconds). Extract the files from the zip file and double-click the InvtPend2.exe file to start the demo. If you get a message that the program has been prevented from running because it is from an unknown publisher, tell it to go ahead anyway. If you have tried the demo, please let me know, even if you do not wish to hazard a guess about how it works.

Bruce

rsmarken · October 3, 2017, 5:10pm

[From Rick Marken (2017.10.03.1010)]

···

Â Bruce Abbott (2017.10.03.1250 EDT)

RM: I lost the pointer to the demo. Could you send it to me again please.

Thanks.

Rick

Â

Bruce Abbott (2017.01.02.0945 EDT) –

Rupert Young (2017.10.02 12.15) –

(Bruce Abbott (2017.10.01.1645 EDT)]

BA:Â No learning is involved, neither reorganization nor RL.Â I simply altered the existing code from the original demo a very slight bit to make the pendulum self-inverting.

BA:Â If you download and run the program, you will see what self-inverting means.Â Try it!

BA: Â Iâ€™d be interested in hearing from anyone who has run this magical demo.Â Without looking at the source code, can you guess how self-inversion was achieved?Â The solution was so simple I was surprised when it actually worked!

RY: Awesome demo Bruce, I love it!

BA: Thanks!

RY: The trick is very neat, and interesting (though I had to look at the code). Is 0.8 the ground level?

BA: No, ground level is zero.Â The pendulum is one meter long, so when hanging straight down, the bob is at Y = -1.0, and +1.0 when standing straight up.

RY: Do you think it is valid, in terms of PCT, as it is introducing a perception by the back door?

BA: Not as currently implemented.Â But I donâ€™t want to discuss this issue too much at this point, as I want to give others a chance at guessing the solution.

RY: I think that perception could be taken out into its own control system, which would then switch between two lower systems, in a similar way with the brake/throttle scenario?

BA:Â Yes, certainly, but at the cost of increased complexity, of course.Â Would we really need two nearly identical lower systems that differ only in you-know-what?Â Or could the higher system manipulate the lower systemâ€™s you-know-what directly?Â What perception would the higher system be controlling for?

BA: O.K., Iâ€™ve heard from Rupert, Rick, and Erling, but Iâ€™m still awaiting replies from Rick and Erling as to how they imagine that the self-inverting pendulum accomplishes its neat trick.Â After Iâ€™ve heard from everyone, Iâ€™ll reveal the secret.

Also, are these three the only subscribers to CSGnet who have tried the demo?Â Itâ€™s a quick download, even on my slow internet connection (a matter of a few seconds). Extract the files from the zip file and double-click the InvtPend2.exe file to start the demo.Â If you get a message that the program has been prevented from running because it is from an unknown publisher, tell it to go ahead anyway.Â If you have tried the demo, please let me know, even if you do not wish to hazard a guess about how it works.

Bruce

–

Richard S. MarkenÂ

"Perfection is achieved not when you have nothing more to add, but when you
have nothing left to take away.â€?
Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â --Antoine de Saint-Exupery

MartinT · October 3, 2017, 5:45pm

[From Bruce
Abbott (2017.10.03.1250 EDT)]

···

Bruce Abbott
(2017.01.02.0945 EDT) –

Rupert Young (2017.10.02 12.15) –

      (Bruce Abbott

(2017.10.01.1645 EDT)]

                  BA: 
No learning is involved, neither reorganization
nor RL. I simply altered the existing code from
the original demo a very slight bit to make the
pendulum self-inverting.

                  BA: 
If you download and run the program, you will see
what self-inverting means. Try it!

                  BA:
I’d be interested in hearing from anyone who has
run this magical demo. Without looking at the
source code, can you guess how self-inversion was
achieved? The solution was so simple I was
surprised when it actually worked!

      RY:           Awesome demo Bruce,

I love it!

BA: Thanks!

      RY:           The trick is very

neat, and interesting (though I had to look at the code). Is
0.8 the ground level?

        BA: No, ground level is zero.

The pendulum is one meter long, so when hanging straight
down, the bob is at Y = -1.0, and +1.0 when standing
straight up.

RY: Do you think it is
valid, in terms of PCT, as it is introducing a perception by
the back door?

        BA: Not as currently implemented.

But I don’t want to discuss this issue too much at this
point, as I want to give others a chance at guessing the
solution.

RY: I think that perception
could be taken out into its own control system, which would
then switch between two lower systems, in a similar way with
the brake/throttle scenario?

        BA:  Yes, certainly, but at the

cost of increased complexity, of course. Would we really
need two nearly identical lower systems that differ only in
you-know-what? Or could the higher system manipulate the
lower system’s you-know-what directly? What perception
would the higher system be controlling for?

        BA: O.K., I’ve heard from Rupert,

Rick, and Erling, but I’m still awaiting replies from Rick
and Erling as to how they imagine that the self-inverting
pendulum accomplishes its neat trick. After I’ve heard from
everyone, I’ll reveal the secret.

        Also, are these three the only

subscribers to CSGnet who have tried the demo? It’s a quick
download, even on my slow internet connection (a matter of a
few seconds). Extract the files from the zip file and
double-click the InvtPend2.exe file to start the demo. If
you get a message that the program has been prevented from
running because it is from an unknown publisher, tell it to
go ahead anyway. If you have tried the demo, please let me
know, even if you do not wish to hazard a guess about how it
works.

Abbott · October 3, 2017, 5:52pm

[From Bruce Abbott (2017.10.03.1355 EDT}

Rick Marken (2017.10.03.1010)]

Bruce Abbott (2017.10.03.1250 EDT)

RM: I lost the pointer to the demo. Could you send it to me again please.

[https://sites.google.com/site/perceptualcontroldemos/home/available-demo-files](https://urldefense
.proofpoint.com/v2/url?u=https-3A__sites.google.com_site_perceptualcontroldemos_home_available-2Ddemo-2Dfiles&d=DwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=-dJBNItYEMOLt6aj_KjGi2LMO_Q8QB-ZzxIZIF8DGyQ&m=5DIx7bRpef9Kp2zTYUj5FAfTgyRRcdvmXCnQGFdcwV4&s=xz2OhmOGmOWFSBraxXoFC35gNtjQkceR1zJYjT-7AdM&e=)

Bruce

Abbott · October 3, 2017, 6:08pm

[From Bruce Abbott (2017.10.03.1405 EDT)]

[Martin Taylor 2017.10.03.13.44]

[From Bruce Abbott (2017.10.03.1250 EDT)]

Bruce Abbott (2017.01.02.0945 EDT) –

Rupert Young (2017.10.02 12.15) –

(Bruce Abbott (2017.10.01.1645 EDT)]

BA: No learning is involved, neither reorganization nor RL. I simply altered the existing code from the original demo a very slight bit to make the pendulum self-inverting.

BA: If you download and run the program, you will see what self-inverting means. Try it!

BA: I’d be interested in hearing from anyone who has run this magical demo. Without looking at the source code, can you guess how self-inversion was achieved? The solution was so simple I was surprised when it actually worked!

RY: Awesome demo Bruce, I love it!

BA: Thanks!

RY: The trick is very neat, and interesting (though I had to look at the code). Is 0.8 the ground level?

BA: No, ground level is zero. The pendulum is one meter long, so when hanging straight down, the bob is at Y = -1.0, and +1.0 when standing straight up.

RY: Do you think it is valid, in terms of PCT, as it is introducing a perception by the back door?

BA: Not as currently implemented. But I don’t want to discuss this issue too much at this point, as I want to give others a chance at guessing the solution.

RY: I think that perception could be taken out into its own control system, which would then switch between two lower systems, in a similar way with the brake/throttle scenario?

BA: Yes, certainly, but at the cost of increased complexity, of course. Would we really need two nearly identical lower systems that differ only in you-know-what? Or could the higher system manipulate the lower system’s you-know-what directly? What perception would the higher system be controlling for?

BA: O.K., I’ve heard from Rupert, Rick, and Erling, but I’m still awaiting replies from Rick and Erling as to how they imagine that the self-inverting pendulum accomplishes its neat trick. After I’ve heard from everyone, I’ll reveal the secret.

Also, are these three the only subscribers to CSGnet who have tried the demo? It’s a quick download, even on my slow internet connection (a matter of a few seconds). Extract the files from the zip file and double-click the InvtPend2.exe file to start the demo. If you get a message that the program has been prevented from running because it is from an unknown publisher, tell it to go ahead anyway. If you have tried the demo, please let me know, even if you do not wish to hazard a guess about how it works.

MT: Can you compile it to run on a Mac?

BA: No, but it should run on a Mac using a PC emulator. No guarantees though, as I haven’t tried it.

Bruce

MartinT · October 3, 2017, 6:19pm

[From Bruce
Abbott (2017.10.03.1405 EDT)]

[Martin Taylor 2017.10.03.13.44]

···

          [From
Bruce Abbott (2017.10.03.1250 EDT)]

          Bruce
Abbott (2017.01.02.0945 EDT) –

Rupert Young (2017.10.02 12.15) –

        (Bruce
Abbott (2017.10.01.1645 EDT)]

                    BA: 
No learning is involved, neither reorganization
nor RL. I simply altered the existing code from
the original demo a very slight bit to make the
pendulum self-inverting.

                    BA: 
If you download and run the program, you will
see what self-inverting means. Try it!

                    BA:
I’d be interested in hearing from anyone who
has run this magical demo. Without looking at
the source code, can you guess how
self-inversion was achieved? The solution was
so simple I was surprised when it actually
worked!

        RY:             Awesome demo

Bruce, I love it!

BA: Thanks!
        RY:             The trick is very
neat, and interesting (though I had to look at the code). Is
0.8 the ground level?
                      BA: No, ground level is
zero. The pendulum is one meter long, so when hanging
straight down, the bob is at Y = -1.0, and +1.0 when
standing straight up.

RY: Do you think it is
valid, in terms of PCT, as it is introducing a perception by
the back door?

          BA: Not as currently
implemented. But I don’t want to discuss this issue too
much at this point, as I want to give others a chance at
guessing the solution.

RY: I think that
perception could be taken out into its own control system,
which would then switch between two lower systems, in a
similar way with the brake/throttle scenario?

          BA:  Yes, certainly, but at the
cost of increased complexity, of course. Would we really
need two nearly identical lower systems that differ only
in you-know-what? Or could the higher system manipulate
the lower system’s you-know-what directly? What
perception would the higher system be controlling for?

          BA: O.K., I’ve heard from Rupert,
Rick, and Erling, but I’m still awaiting replies from Rick
and Erling as to how they imagine that the self-inverting
pendulum accomplishes its neat trick. After I’ve heard
from everyone, I’ll reveal the secret.

          Also, are these three the only
subscribers to CSGnet who have tried the demo? It’s a
quick download, even on my slow internet connection (a
matter of a few seconds). Extract the files from the zip
file and double-click the InvtPend2.exe file to start the
demo. If you get a message that the program has been
prevented from running because it is from an unknown
publisher, tell it to go ahead anyway. If you have tried
the demo, please let me know, even if you do not wish to
hazard a guess about how it works.

      MT:           Can you compile it

to run on a Mac?

                  BA: No, but it should run on a

Mac using a PC emulator. No guarantees though, as I haven’t
tried it.

Bruce

bnhpct · October 3, 2017, 11:34pm

[From Bruce Nevin (2017.10.03.1624 PDT)]

Bruce Abbott (2017.10.03.1405 EDT) –

Looks like it was enough just to give it a starting position that it cannot attain by itself–the same start position for the cart as in the original demo, but the negative of the start position for the bob which serves as the start position for the original demo. Any slightest movement of the bob disturbs the equipoise. Any movement of the cart would have the same effect, but it is as though it has no error to determine which direction to move the cart in order to initiate the swing above ground. How could it have no error? Rather, the error is equal in each direction. It seems to be a cheat, since there’s no movement of the bob sufficient to swing it below the ground level, nor any cumulative rhythmic movement like that by which it brings the bob above ground level.

···

On Tue, Oct 3, 2017 at 2:08 PM, Bruce Abbott bbabbott@frontier.com wrote:

[From Bruce Abbott (2017.10.03.1405 EDT)]

[Martin Taylor 2017.10.03.13.44]

[From Bruce Abbott (2017.10.03.1250 EDT)]

Bruce Abbott (2017.01.02.0945 EDT) –

Rupert Young (2017.10.02 12.15) –

(Bruce Abbott (2017.10.01.1645 EDT)]

BA: No learning is involved, neither reorganization nor RL. I simply altered the existing code from the original demo a very slight bit to make the pendulum self-inverting.

BA: If you download and run the program, you will see what self-inverting means. Try it!

BA: I’d be interested in hearing from anyone who has run this magical demo. Without looking at the source code, can you guess how self-inversion was achieved? The solution was so simple I was surprised when it actually worked!

RY: Awesome demo Bruce, I love it!

BA: Thanks!

RY: The trick is very neat, and interesting (though I had to look at the code). Is 0.8 the ground level?

BA: No, ground level is zero. The pendulum is one meter long, so when hanging straight down, the bob is at Y = -1.0, and +1.0 when standing straight up.

RY: Do you think it is valid, in terms of PCT, as it is introducing a perception by the back door?

BA: Not as currently implemented. But I don’t want to discuss this issue too much at this point, as I want to give others a chance at guessing the solution.

RY: I think that perception could be taken out into its own control system, which would then switch between two lower systems, in a similar way with the brake/throttle scenario?

BA: Yes, certainly, but at the cost of increased complexity, of course. Would we really need two nearly identical lower systems that differ only in you-know-what? Or could the higher system manipulate the lower system’s you-know-what directly? What perception would the higher system be controlling for?

BA: O.K., I’ve heard from Rupert, Rick, and Erling, but I’m still awaiting replies from Rick and Erling as to how they imagine that the self-inverting pendulum accomplishes its neat trick. After I’ve heard from everyone, I’ll reveal the secret.

Also, are these three the only subscribers to CSGnet who have tried the demo? It’s a quick download, even on my slow internet connection (a matter of a few seconds). Extract the files from the zip file and double-click the InvtPend2.exe file to start the demo. If you get a message that the program has been prevented from running because it is from an unknown publisher, tell it to go ahead anyway. If you have tried the demo, please let me know, even if you do not wish to hazard a guess about how it works.

MT: Can you compile it to run on a Mac?

BA: No, but it should run on a Mac using a PC emulator. No guarantees though, as I haven’t tried it.

Bruce

Abbott · October 4, 2017, 1:29am

[From Bruce Abbott (2017.10.03.2130)]

Bruce Nevin (2017.10.03.1624 PDT) -

BN:Â Looks like it was enough just to give it a starting position that it cannot attain by itself–the same start position for the cart as in the original demo, but the negative of the start position for the bob which serves as the start position for the original demo. Any slightest movement of the bob disturbs the equipoise. Any movement of the cart would have the same effect, but it is as though it has no error to determine which direction to move the cart in order to initiate the swing above ground. How could it have no error? Rather, the error is equal in each direction. It seems to be a cheat, since there’s no movement of the bob sufficient to swing it below the ground level, nor any cumulative rhythmic movement like that by which it brings the bob above ground level.

BA:Â Iâ€™m sorry, but Iâ€™m not following your last sentence.Â What seems a cheat?Â What do you mean by â€œthereâ€™s no movement of the bob sufficient to swing it below the ground levelâ€??Â The simulation begins with the bob â€œbelow ground levelâ€? (or more accurately, below track level.Â Imagine the cart on an elevated track high enough to permit the bob to hang vertically below it.)

BA:Â As for your first sentence, it definitely is not enough just to give a starting position with the bob hanging straight down.Â The original inverted-pendulum demo cannot invert the pendulum once it has lost control and allowed the pendulum to hang down, no matter what.

BA:Â I made a small change in the control system that allowed the pendulum to become self-inverting.Â Iâ€™m still looking for a guess as to what that change might have been.Â What youâ€™ve given me thus far is mostly a description of its behavior rather than its mechanism.

BA:Â I should note, for those who try the demo, that you have to drag the reference mark (red line) slightly left or right in order to get the action going.

Bruce A.

_Erling_Jorgensen · October 4, 2017, 1:46am

[From Erling Jorgensen (2017.10.03 2133 EDT)]

Bruce Abbott (2017.10.03.2130)

BA: I made a small change in the control system that allowed the pendulum to become self-inverting. I’m still looking for a guess as to what that change might have been. What you’ve given me thus far is mostly a description of its behavior rather than its mechanism.

EJ: I haven’t looked at the source code because I’ve been trying to guess; (nor do I see a way to look at the source code.) I have tried slowing down some of the parameters, such as velocity, to try to see clearer what is going. Given your hint about “a small change,” I wonder about the following:

EJ: Increasing the Bob acceleration if it is below the track level, and decreasing the Bob acceleration if it is above the track level.

EJ: Whether I’m guessing right or not, it is still a marvelous demo! (And, yes, how do you get to that source code??)

All the best,

Erling

···

Disclaimer: This message is intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, or the employer or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by telephone and delete the material from your computer. Thank you for your cooperation.

Abbott · October 4, 2017, 12:10pm

[From Bruce Abbott (2017.10.04.0810 EDT)]

Erling Jorgensen (2017.10.03 2133 EDT) –

Bruce Abbott (2017.10.03.2130)

BA: I made a small change in the control system that allowed the pendulum to become self-inverting. Iâ€™m still looking for a guess as to what that change might have been. What youâ€™ve given me thus far is mostly a description of its behavior rather than its mechanism.

EJ: I haven’t looked at the source code because I’ve been trying to guess; (nor do I see a way to look at the source code.) I have tried slowing down some of the parameters, such as velocity, to try to see clearer what is going. Given your hint about “a small change,” I wonder about the following:

EJ: Increasing the Bob acceleration if it is below the track level, and decreasing the Bob acceleration if it is above the track level.

BA: Nice try, but sorry, no.

EJ: Whether I’m guessing right or not, it is still a marvelous demo! (And, yes, how do you get to that source code??)

BA: Thanks!Â The source code is contained in the file Main.pas.Â The â€œpasâ€? stands for Pascal, but it is a text file that you can open with Notepad or any word processor.

Bruce

···

Disclaimer: This message is intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, or the employer or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by telephone and delete the material from your computer. Thank you for your cooperation.

_Erling_Jorgensen · October 4, 2017, 12:53pm

[From Erling Jorgensen (2017.10.04 0838 EDT)]

Bruce Abbott (2017.10.04.0810 EDT)

Erling Jorgensen (2017.10.03 2133 EDT)
EJ: Increasing the Bob acceleration if it is below the track level, and decreasing the Bob acceleration if it is above the track level.

BA: Nice try, but sorry, no.

EJ: Continuing the guesses... It almost looks as though the cart seeks to keep the bob directly below the red reference line, even when it is below the track level. But then the physics of the environment, with a freely swinging pendulum, lead to runaway feedback making the error worse, until the bob gets above the track level. That might explain why there is enough momentum to actually swing above the track, whereas I (with somewhat limited physics training) would expect that a swing up to a horizontal level on one side would only have enough momentum to get an equal swing up to horizontal on the other. That it didn't is the first thing that surprised me about the demo.
EJ: In any event, my guess is that positive runaway feedback occurs below the track line, as the cart nonetheless seeks to keep the bob directly below the red reference line. But then when the bob actually gets above the track level, then it becomes negative feedback again, seemingly because of the different way gravity interacts (or the computer-simulated version of gravity) when balancing rather than suspending. Not sure if I have the terminology right, but is there a changeover point in loop gain from runaway to self-correcting feedback? Seems like a gain that is set too high might lead to that kind of oscillation below the track.
EJ: I await the professors 'feedback'.
All the best,
Erling

Disclaimer: This message is intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, or the employer or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by telephone and delete the material from your computer. Thank you for your cooperation.

bnhpct · October 4, 2017, 4:59pm

[From Bruce Nevin (2017.10.04.09:53 PT]

Bruce Abbott (2017.10.03.2130)

What youâ€™ve given me thus far is mostly a description of its behavior rather than its mechanism.Â

That is correct. The operative phrases are “looks like” and “seems to be”.

Given x-y coordinates, if the starting position of the plumb (the circle on the end of the pendulum) is yÂ for the original demo, then its starting position is -yÂ for the new one. I thought that was the “one simple change” that you made, and tried to imagine how that could have the observed effect.

So you made a second simple change affecting the control of the cart when the plumbÂ approaches the horizon or ‘ground level’ so that the feedback flips from negative feedback to positive feedback. My guess is that you reversed the sense (+ to - or vice versa) of the acceleration of the cart, or of some constant multiplier affecting the acceleration of the cart. When the cart is stationary and the plumb is vertical at the zenith (+y) acceleration remains zero; ditto when the plumb is vertical at the nadir (-y).

···

On Tue, Oct 3, 2017 at 9:29 PM, Bruce Abbott bbabbott@frontier.com wrote:

[From Bruce Abbott (2017.10.03.2130)]

Â

BruceÂ NevinÂ (2017.10.03.1624 PDT) -

Â

BN:Â Looks like it was enough just to give it a starting position that it cannot attain by itself–the same start position for the cart as in the original demo, but the negative of the start position for the bob which serves as the start position for the original demo. Any slightest movement of the bob disturbs the equipoise. Any movement of the cart would have the same effect, but it is as though it has no error to determine which direction to move the cart in order to initiate the swing above ground. How could it have no error? Rather, the error is equal in each direction. It seems to be a cheat, since there’s no movement of the bob sufficient to swing it below the ground level, nor any cumulative rhythmic movement like that by which it brings the bob above ground level.

Â

BA:Â Iâ€™m sorry, but Iâ€™m not following your last sentence.Â What seems a cheat?Â What do you mean by â€œthereâ€™s no movement of the bob sufficient to swing it below the ground levelâ€??Â The simulation begins with the bob â€œbelow ground levelâ€? (or more accurately, below track level.Â Imagine the cart on an elevated track high enough to permit the bob to hang vertically below it.)

Â

BA:Â As for your first sentence, it definitely is not enough just to give a starting position with the bob hanging straight down.Â The original inverted-pendulum demo cannot invert the pendulum once it has lost control and allowed the pendulum to hang down, no matter what.

Â

BA:Â I made a small change in the control system that allowed the pendulum to become self-inverting.Â Iâ€™m still looking for a guess as to what that change might have been.Â What youâ€™ve given me thus far is mostly a description of its behavior rather than its mechanism.

Â

BA:Â I should note, for those who try the demo, that you have to drag the reference mark (red line) slightly left or right in order to get the action going.

Â

Bruce A.

Abbott · October 5, 2017, 12:02am

[From Bruce Abbott (2017.10.04.1800 EDT)]

BA: I thought we might hear from Rick Marken by now, but apparently he does not wish to hazard a guess, so I’ll just go ahead with the “reveal.”

Erling Jorgensen (2017.10.04 0838 EDT) –

Bruce Abbott (2017.10.04.0810 EDT)

Erling Jorgensen (2017.10.03 2133 EDT)

EJ: Increasing the Bob acceleration if it is below the track level, and decreasing the Bob acceleration if it is above the track level.

BA: Nice try, but sorry, no.

EJ: Continuing the guesses… It almost looks as though the cart seeks to keep the bob directly below the red reference line, even when it is below the track level. But then the physics of the environment, with a freely swinging pendulum, lead to runaway feedback making the error worse, until the bob gets above the track level. That might explain why there is enough momentum to actually swing above the track, whereas I (with somewhat limited physics training) would expect that a swing up to a horizontal level on one side would only have enough momentum to get an equal swing up to horizontal on the other. That it didn’t is the first thing that surprised me about the demo.

EJ: In any event, my guess is that positive runaway feedback occurs below the track line, as the cart nonetheless seeks to keep the bob directly below the red reference line. But then when the bob actually gets above the track level, then it becomes negative feedback again, seemingly because of the different way gravity interacts (or the computer-simulated version of gravity) when balancing rather than suspending. Not sure if I have the terminology right, but is there a changeover point in loop gain from runaway to self-correcting feedback? Seems like a gain that is set too high might lead to that kind of oscillation below the track.

EJ: I await the professors ‘feedback’.

BA: Earling, you’ve hit the nail on the head! The small changes I made to the original demo are as follows. There is an “if-then” statement that is evaluated on each iteration. If the bob height above the track is less than 0.80 meters, the output gain of the acceleration control system is made negative, which sets up a positive feedback loop. Consequently, any slight nudge of the pendulum when it is hanging down vertically will induce a runaway condition in which the force acting on the bob accelerates it to a higher position on each swing. When bob reaches at least 0.8 meters above the track, the feedback is restored to negative and the system then works to stabilize the pendulum in the inverted position.

Bruce Nevin (2017.10.04.09:53 PT –

Bruce Abbott (2017.10.03.2130)

BN: Given x-y coordinates, if the starting position of the plumb (the circle on the end of the pendulum) is y for the original demo, then its starting position is -y for the new one. I thought that was the “one simple change” that you made, and tried to imagine how that could have the observed effect.

BA: Well, that change alone would not be sufficient to make the pendulum self-inverting, as can be seen in the original version of the demo if you first turn off control and allow the pendulum to fall, then turn control back on.

BN: So you made a second simple change affecting the control of the cart when the plumb approaches the horizon or ‘ground level’ so that the feedback flips from negative feedback to positive feedback. My guess is that you reversed the sense (+ to - or vice versa) of the acceleration of the cart, or of some constant multiplier affecting the acceleration of the cart. When the cart is stationary and the plumb is vertical at the zenith (+y) acceleration remains zero; ditto when the plumb is vertical at the nadir (-y).

BA: If “sense” refers to the sign of feedback, then you are correct – positive feedback when the bob is 0.80 meters below the vertical position of 1.0 meters, negative feedback if above. This corresponds to a certain angle of the inverted pendulum from the vertical and was arrived at empirically. Given the maximum speed of the cart, length of the pendulum rod, and influences of cart motion and gravity on the speed with which the pendulum falls, this corresponds to an angle beyond which the cart cannot accelerate rapidly enough to “get under” the falling inverted pendulum in time to stop the fall and raise it back to the inverted vertical position.

Congratulations to you both! Nice job . . .

Bruce

rsmarken · October 5, 2017, 12:21am

[From Rick Marken (2017.01.04.1720)

[From Bruce Abbott (2017.10.04.1800 EDT)]

Â

BA: I thought we might hear from Rick Marken by now, but apparently he does not wish to hazard a guess, so Iâll just go ahead with the âreveal.â?

No! Stop! I was about to write. I really haven't have time toi concentrate on it but I finally did get a moment and managed to figure out what the innovation was! Very nice improvement! I think it's that you developed a way for the system not to go into positive feedback when the bob goes below "ground level".Â I haven't figured out how you did it. And I don't have time to think much about it today. So I'll wait to read the rest of your post until I come up with a hypothesis. But you say it's simple so maybe it's just a matter of having the perception be something like the absolute value of the angle of the bob relative to ground? OK, I'll let that be my guess and then read your answer after I post this.
BestÂ

···

Â

Erling Jorgensen (2017.10.04 0838 EDT)

Â

>Bruce Abbott (2017.10.04.0810 EDT)

>>Erling Jorgensen (2017.10.03 2133 EDT)

Â

>>EJ: Increasing the Bob acceleration if it is below the track level, and decreasing the Bob acceleration if it is above the track level.

Â

>BA: Nice try, but sorry, no.

Â

EJ:Â Continuing the guesses...Â Â It almost looks as though the cart seeks to keep the bob directly below the red reference line, even when it is below the track level.Â But then the physics of the environment, with a freely swinging pendulum, lead to runaway feedback making the error worse, until the bob gets above the track level.Â That might explain why there is enough momentum to actually swing above the track, whereas I (with somewhat limited physics training) would expect that a swing up to a horizontal level on one side would only have enough momentum to get an equal swing up to horizontalÂ on the other.Â That it didn't is the first thing that surprised me about the demo.Â

EJ:Â In any event, my guess is that positive runaway feedback occurs below the track line, as the cart nonetheless seeks to keep the bob directly below the redÂ reference line.Â But then when the bob actually gets above the track level, then it becomes negative feedback again, seemingly because of the different way gravity interacts (or the computer-simulated version of gravity) when balancing rather than suspending.Â Not sure if I have the terminology right, but is there a changeover point in loop gain from runaway to self-correcting feedback?Â Seems like a gain that is set too high might lead to that kind of oscillation below the track.Â

EJ:Â I await the professors 'feedback'.Â

Â

BA:Â Earling, youâve hit the nail on the head!Â The small changes I made to the original demo are as follows.Â There is an âif-thenâ? statement that is evaluated on each iteration.Â If the bob height above the track is less than 0.80 meters, the output gain of the acceleration control system is made negative, which sets up a positive feedback loop.Â Consequently, any slight nudge of the pendulum when it is hanging down vertically will induce a runaway condition in which the force acting on the bob accelerates it to a higher position on each swing.Â When bob reaches at least 0.8 meters above the track, the feedback is restored to negative and the system then works to stabilize the pendulum in the inverted position.

Â

Bruce Nevin (2017.10.04.09:53 PT --

Â

Bruce Abbott (2017.10.03.2130)

Â

BN: Given x-y coordinates, if the starting position of the plumb (the circle on the end of the pendulum) is yÂ for the original demo, then its starting position is -yÂ for the new one. I thought that was the "one simple change" that you made, and tried to imagine how that could have the observed effect.

Â

BA:Â Well, that change alone would not be sufficient to make the pendulum self-inverting, as can be seen in the original version of the demo if you first turn off control and allow the pendulum to fall, then turn control back on.

Â

BN: So you made a second simple change affecting the control of the cart when the plumbÂ approaches the horizon or 'ground level' so that the feedback flips from negative feedback to positive feedback. My guess is that you reversed the sense (+ to - or vice versa) of the acceleration of the cart, or of some constant multiplier affecting the acceleration of the cart. When the cart is stationary and the plumb is vertical at the zenith (+y) acceleration remains zero; ditto when the plumb is vertical at the nadir (-y).

Â

BA: If âsenseâ? refers to the sign of feedback, then you are correct positive feedback when the bob is 0.80 meters below the vertical position of 1.0 meters, negative feedback if above.Â This corresponds to a certain angle of the inverted pendulum from the vertical and was arrived at empirically.Â Given the maximum speed of the cart, length of the pendulum rod, and influences of cart motion and gravity on the speed with which the pendulum falls, this corresponds to an angle beyond which the cart cannot accelerate rapidly enough to âget underâ? the falling inverted pendulum in time to stop the fall and raise it back to the inverted vertical position.

Â

Congratulations to you both!Â Nice job . . .

Â

Bruce

Â

--
Richard S. MarkenÂ
"Perfection is achieved not when you have nothing more to add, but when you
have nothing left to take away.â?
Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â --Antoine de Saint-Exupery

bnhpct · October 5, 2017, 12:34am

[From Bruce Nevin (2017.10.04.17:16 PT)]

*** SPOILER ALERT FOR RICK ***

Bruce Abbott (2017.10.04.1800 EDT) –

ToÂ Erling Jorgensen (2017.10.03 2133 EDT)Â you replied

If the bob height above the track is less than 0.80 meters, the output gain of the acceleration control system is made negative, which sets up a positive feedback loop.Â

I think you meant if the bob height above the track is less than 0.20 meters. You phrased it better when you replied to myÂ Â (2017.10.04.09:53 PT):

positive feedback when the bob is 0.80 meters below the vertical position of 1.0 meters, negative feedback if aboveÂ

Martin’s flip-flop might accomplish this given aÂ way of perceiving thatÂ errorÂ (departure of the plumb bob from the vertical, however sensed) is increasing despiteÂ Â increasedÂ output (acceleration).

···

On Wed, Oct 4, 2017 at 8:02 PM, Bruce Abbott bbabbott@frontier.com wrote:

[From Bruce Abbott (2017.10.04.1800 EDT)]

Â

BA: I thought we might hear from Rick Marken by now, but apparently he does not wish to hazard a guess, so Iâ€™ll just go ahead with the â€œreveal.â€?

Â

Erling Jorgensen (2017.10.04 0838 EDT) –<

Â

Bruce Abbott (2017.10.04.0810 EDT)

Erling Jorgensen (2017.10.03 2133 EDT)

Â

EJ: Increasing the Bob acceleration if it is below the track level, and decreasing the Bob acceleration if it is above the track level.

Â

BA: Nice try, but sorry, no.

Â

EJ:Â Continuing the guesses…Â Â It almost looks as though the cart seeks to keep the bob directly below the red reference line, even when it is below the track level.Â But then the physics of the environment, with a freely swinging pendulum, lead to runaway feedback making the error worse, until the bob gets above the track level.Â That might explain why there is enough momentum to actually swing above the track, whereas I (with somewhat limited physics training) would expect that a swing up to a horizontal level on one side would only have enough momentum to get an equal swing up to horizontalÂ on the other.Â That it didn’t is the first thing that surprised me about the demo.Â

EJ:Â In any event, my guess is that positive runaway feedback occurs below the track line, as the cart nonetheless seeks to keep the bob directly below the redÂ reference line.Â But then when the bob actually gets above the track level, then it becomes negative feedback again, seemingly because of the different way gravity interacts (or the computer-simulated version of gravity) when balancing rather than suspending.Â Not sure if I have the terminology right, but is there a changeover point in loop gain from runaway to self-correcting feedback?Â Seems like a gain that is set too high might lead to that kind of oscillation below the track.Â

EJ:Â I await the professors ‘feedback’.Â

Â

BA:Â Earling, youâ€™ve hit the nail on the head!Â The small changes I made to the original demo are as follows.Â There is an â€œif-thenâ€? statement that is evaluated on each iteration.Â If the bob height above the track is less than 0.80 meters, the output gain of the acceleration control system is made negative, which sets up a positive feedback loop.Â Consequently, any slight nudge of the pendulum when it is hanging down vertically will induce a runaway condition in which the force acting on the bob accelerates it to a higher position on each swing.Â When bob reaches at least 0.8 meters above the track, the feedback is restored to negative and the system then works to stabilize the pendulum in the inverted position.

Â

Bruce Nevin (2017.10.04.09:53 PT –

Â

Bruce Abbott (2017.10.03.2130)

Â

BN: Given x-y coordinates, if the starting position of the plumb (the circle on the end of the pendulum) is yÂ for the original demo, then its starting position is -yÂ for the new one. I thought that was the “one simple change” that you made, and tried to imagine how that could have the observed effect.

Â

BA:Â Well, that change alone would not be sufficient to make the pendulum self-inverting, as can be seen in the original version of the demo if you first turn off control and allow the pendulum to fall, then turn control back on.

Â

BN: So you made a second simple change affecting the control of the cart when the plumbÂ approaches the horizon or ‘ground level’ so that the feedback flips from negative feedback to positive feedback. My guess is that you reversed the sense (+ to - or vice versa) of the acceleration of the cart, or of some constant multiplier affecting the acceleration of the cart. When the cart is stationary and the plumb is vertical at the zenith (+y) acceleration remains zero; ditto when the plumb is vertical at the nadir (-y).

Â

BA: If â€œsenseâ€? refers to the sign of feedback, then you are correct – positive feedbacck when the bob is 0.80 meters below the vertical position of 1.0 meters, negative feedback if above.Â This corresponds to a certain angle of the inverted pendulum from the vertical and was arrived at empirically.Â Given the maximum speed of the cart, length of the pendulum rod, and influences of cart motion and gravity on the speed with which the pendulum falls, this corresponds to an angle beyond which the cart cannot accelerate rapidly enough to â€œget underâ€? the falling inverted pendulum in time to stop the fall and raise it back to the inverted vertical position.

Â

Congratulations to you both!Â Nice job . . .

Â

Bruce

Â

_Erling_Jorgensen · October 5, 2017, 12:35pm

[From Erling Jorgensen (2017.10.05.0828 EDT)]

Rick Marken (2017.01.04.1720)

Bruce Abbott (2017.10.04.1800 EDT)

BA: I thought we might hear from Rick Marken by now, but apparently he does not wish to hazard a guess, so Iâ€™ll just go ahead with the â€œreveal.â€?

No! Stop! I was about to write. I really haven’t have time toi concentrate on it but I finally did get a moment and managed to figure out what the innovation was!

EJ: You know, Rick, there’s a show on NPR that I’m sure you’d love – “Wait! Wait! Don’t Tell Me!!” I know the feeling. I wouldn’t let myself look at the source code, because who wants to look at a Sudoku puzzle with all the numbers already filled in?

EJ: No Spoiler Alert here. Just be careful what posts you click on for a while.

All the best,

Erling

Abbott · October 5, 2017, 1:40pm

[From Bruce Abbott (2017.10.05.0940 EDT)]

Bruce Nevin (2017.10.04.17:16 PT) –

*** SPOILER ALERT FOR RICK ***

Bruce Abbott (2017.10.04.1800 EDT) –

BN: To Erling Jorgensen (2017.10.03 2133 EDT) you replied

If the bob height above the track is less than 0.80 meters, the output gain of the acceleration control system is made negative, which sets up a positive feedback loop.

BN: I think you meant if the bob height above the track is less than 0.20 meters. You phrased it better when you replied to my (2017.10.04.09:53 PT):

positive feedback when the bob is 0.80 meters below the vertical position of 1.0 meters, negative feedback if above

BA: Well, I misstated the case on one of those occasions, didn’t I? The first is correct: feedback becomes positive when the bob height falls below 0.8 meters above the pendulum’s pivot. This corresponds to an angle of roughly 10 degrees or so off the vertical, in either direction. Empirically, this is the angle beyond which the cart cannot move fast enough to stem the fall.

BN: Martin’s flip-flop might accomplish this given a way of perceiving that error (departure of the plumb bob from the vertical, however sensed) is increasing despite increased output (acceleration).

BA: Yes, there are several alternatives that would yield the same result.

BA: The pendulum with positive acceleration feedback resembles the back-and-forth behavior of a swing as the occupant attempts to get the swing to go as high as possible. The resemblance is superficial, I think. Not only is the method of accelerating the swing different, it is also the case that the swinger remains in control of the maximum swing-height (within the limits defined by physics). In contrast, the pendulum with positive feedback experiences a runaway condition that will always go to the maximum. I wonder what a PCT model of the swinger would look like . . . (:- >

Bruce

MartinT · October 5, 2017, 2:37pm

[From Bruce Abbott (2017.10.05.0940 EDT)]

…

···

BA: The pendulum with positive
acceleration feedback resembles the back-and-forth
behavior of a swing as the occupant attempts to get
the swing to go as high as possible. The resemblance
is superficial, I think. Not only is the method of
accelerating the swing different, it is also the case
that the swinger remains in control of the maximum
swing-height (within the limits defined by physics).
In contrast, the pendulum with positive feedback
experiences a runaway condition that will always go to
the maximum. I wonder what a PCT model of the swinger
would look like . . . (:- >

Bruce

rsmarken · October 6, 2017, 10:31pm

[From Rick Marken (2017.10.06.1530)]

···

RM: Well I was right about that!

RM: And wrong about that.Â But I was wondering whether there might be a way to do this by having the highest level system control a different perception than the one being controlled in the simulation (the vertical position of the bob).Â It would have to be a variable that takes the variations in the height of the bob relative to the track into account. Perhaps something like the angle of the bob relative to the horizontal track, which would go negative when the bob falls below the track.Â

RM: This led me to think that this inverted pendulum balancing task might be a good one to do using people as subjects. I think it would be easy to set it up this way. Then you could collect data to see how real live living control systems bring the pendulum from hanging below the track to balancing. It could be a tough task but once it’s learned you could start comparing the subjects’ behavior to that of different control models and see which fits best.Â

RM: Right off the bat I think there are three possible models you could test using your little “kluge” by putting the kluge at the three different levels of your model (you currentlyÂ have it at the lowest level) and see if there is any difference in the fit of these models to the human data. Another possibility is to see if you can think of a different high level variable that can work and make the if statement approach unnecessary. This would be like doing a test for the controlled variable (a la my 2013 paperÂ “Testing for Controlled Variables” paper, published inÂ Attention, Perception and PsychophysicsÂ and reprinted inÂ Doing Research on Purpose).Â

RM: Anyway, it would be great if you could adapt this nice inverted pendulum balancing program to do some PCT research using actual, living perceptual control systems.Â

BestÂ

Rick

Bruce Abbott (2017.10.04.1800 EDT)

Â

BA: I thought we might hear from Rick Marken by now, but apparently he does not wish to hazard a guess, so Iâll just go ahead with the âreveal.â?

RM: No! Stop! I was about to write. …I think it’s that you developed a way for the system not to go into positive feedback when the bob goes below “ground level”.Â

RM: I haven’t figured out how you did it. . But you say it’s simple so maybe it’s just a matter of having the perception be something like the absolute value of the angle of the bob relative to ground?

–

Richard S. MarkenÂ

"Perfection is achieved not when you have nothing more to add, but when you
have nothing left to take away.â?
Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â --Antoine de Saint-Exupery

bnhpct · October 10, 2017, 2:18pm

[From Bruce Nevin (2017.10.10.07:15 PT]

Rick Marken (2017.10.06.1530) –

RM: This led me to think that this inverted pendulum balancing task might be a good one to do using people as subjects. I think it would be easy to set it up this way. Then you could collect data to see how real live living control systems bring the pendulum from hanging below the track to balancing. It could be a tough task but once it’s learned you could start comparing the subjects’ behavior to that of different control models and see which fits best.Â

Unclear to me how to make it easy. There’s a bit of an engineering challenge to craft the apparatus so that the pendulum could swing freely past the track and the cart itself, and then the model would have to include the physics of the cart (inertia, friction).

···

On Fri, Oct 6, 2017 at 3:31 PM, Richard Marken rsmarken@gmail.com wrote:

[From Rick Marken (2017.10.06.1530)]

RM: Well I was right about that!

RM: And wrong about that.Â But I was wondering whether there might be a way to do this by having the highest level system control a different perception than the one being controlled in the simulation (the vertical position of the bob).Â It would have to be a variable that takes the variations in the height of the bob relative to the track into account. Perhaps something like the angle of the bob relative to the horizontal track, which would go negative when the bob falls below the track.Â

RM: This led me to think that this inverted pendulum balancing task might be a good one to do using people as subjects. I think it would be easy to set it up this way. Then you could collect data to see how real live living control systems bring the pendulum from hanging below the track to balancing. It could be a tough task but once it’s learned you could start comparing the subjects’ behavior to that of different control models and see which fits best.Â

RM: Right off the bat I think there are three possible models you could test using your little “kluge” by putting the kluge at the three different levels of your model (you currentlyÂ have it at the lowest level) and see if there is any difference in the fit of these models to the human data. Another possibility is to see if you can think of a different high level variable that can work and make the if statement approach unnecessary. This would be like doing a test for the controlled variable (a la my 2013 paperÂ “Testing for Controlled Variables” paper, published inÂ Attention, Perception and PsychophysicsÂ and reprinted inÂ Doing Research on Purpose).Â

RM: Anyway, it would be great if you could adapt this nice inverted pendulum balancing program to do some PCT research using actual, living perceptual control systems.Â

BestÂ

Rick

Bruce Abbott (2017.10.04.1800 EDT)

Â

BA: I thought we might hear from Rick Marken by now, but apparently he does not wish to hazard a guess, so Iâll just go ahead with the âreveal.â?

RM: No! Stop! I was about to write. …I think it’s that you developed a way for the system not to go into positive feedback when the bob goes below “ground level”.Â

RM: I haven’t figured out how you did it. . But you say it’s simple so maybe it’s just a matter of having the perception be something like the absolute value of the angle of the bob relative to ground?

–
Richard S. MarkenÂ

"Perfection is achieved not when you have nothing more to add, but when you
have nothing left to take away.â?
Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â --Antoine de Saint-Exupery