Bomb in the Hierarchy Simulation

[Subject was: Back to Control]

[From Erling Jorgensen (2005.02.22 0100 EST)]

I'm coming to the discussion late, but I can maybe provide one

way to sort out some of the ideas about positive feedback by way

of a simulation. The recent discussion of Martin's notion of

"a bomb in the hierarchy" reminded me of some modeling I did

some years ago (with "toy models", to use Bruce G.'s term).

I launched off of some remarks Martin T. included in his

"Principles for Intelligent Human-Computer Interaction: A

Tutorial on Layered Protocol Theory" (1993, North York, Ontario:

DCIEM). I then implemented them in a simple spreadsheet

simulation, based on Rick M.'s work & adapted to MS Works

spreadsheets (rather than Lotus).

There have been some recent calls for models of positive feedback,

rather than just verbal proposals, so maybe this can advance the

discussion some.

For instance, Martin & Rick had this recent exchange:

Martin Taylor (2005.02.18.09.26)

Rick Marken (2005.02.18.0830)

The real explosion happens when that reversal causes a

higher-level control system to go into positive feedback,

and so on up the chain.

This is what I would want to see demonstrated in a model.

I can imagine runaway error (as per the runaway after sign

reversal) increasing error at higher levels. But I don't see

how the reversal-caused runaway could cause a higher-level

control system to go into positive feedback.

I'm defining positive feedback basically the way Bill did --

Bill Powers (2005.02.19.1945 MST)

If what you do increases your error and leads to your

doing it even more, that's positive feedback, isn't it?

This is what I think I saw in my simulations, & it did seem

to spread to higher levels. I don't have ready access to the

Works files I was using, but I did find some of the results.

I'm also not sure how to make a simulation available to others

on CSGNet, but I thought maybe I could provide the basic

equations (as Rick requested in one of his posts), & maybe

Rick can readapt it to his spreadsheets & explore its properties.

The basic phenomenon is as Martin described it --

Martin Taylor 2005.02.18.01.21

The Bomb is something that exists in a complex environment

with many feedback paths connecting output and perception.

The strength of the feedback paths can vary, and on some

occasions the sign of the overall feedback can change. That

control system then goes into a positive feedback mode, and

if it is in the feedback path of a higher-level control system,

the higher one might also go into a positive feedback mode.

I set up the simulation as a set of eight elementary control

units, distributed in three hierarchical levels -- two in the

highest level, two in the middle level, & four in the lowest

level. The equations were fairly arbitrary & were not meant

to "represent" any perceptions in particular. They were just

linear functions of other cells in the spreadsheet. They were

actually based on the way Martin had set up a theoretical

scenario on p. 30 of the Layered Protocol tutorial (cited above).

Before listing the equations, let me give a general sense of

the set-up. Each control unit was implemented as a set of three

cells, labeled Ref, Per, & Out, (for Reference, Perception, &

Output, obviously). Each level used the same Delay & Gain

factors for the ECU's on that level, with higher levels having

slower Delay & greater Gain than the levels below. The

"environment" was very simplified, consisting of four arbitrary

disturbances (Dis), one for each of the four lowest level ECUs.

I think the way the equations were set up did involve potential

conflict, but in the first simulation involving stable control

at all three levels, there were alternate ways of compensating

for the conflict. The second situation leading to positive

feedback involved a change of sign for one reference in one of

the lowest ECUs. Only that one cell changed, & it seemed to

lead to a "bomb" of positive feedback, that affected all the

ECUs in the upper two levels. The four lowest level ECUs, by

contrast, were able to maintain adequate control, but with

steadily increasing values for the Ref, Per, & Out of each one.

In other words, the stable control at the lowest level was at a

high price of "wasted energy."

The equations below are not listed as spreadsheet formulas,

per se, since it's hard to follow the allusions to different

cells, (& each application uses slightly different conventions

at any rate). Since the simulation was also leading to

runaway feedback values in many of the cells, I also incorporated

what I call a "reset button" into the equation for every cell,

so I could modify & restart a run without much problem. The

reset button consisted of multiplying each equation by a certain

cell (anchored as $B$1), which could be manually set at 0 or 1.

The runaway feedback was basically monitored in terms of three

cells calculating the average error for each level of the

hierarchy. In my spreadsheet application, I did not have a

way to import & retain those values, so basically I would stop

after 50 or 100 iterations & print the values in the whole

spreadsheet, so I could compare them over time. I manually

inserted the number of iterations in a cell before printing.

I also manually recorded each average error at the start of

a run, which was after stable control had initially been

established, so I would have it as a point of reference.

With that as preface & context, these were the basic form

of the equations that I used. Each variable has one or two

numbers which should be read as subscripts (i,j), the first

of which designates the hierarchical level, the second of

which designates which ECU at that level.

Level 3 --

Delay3 = 0.00001

Gain3 = 1000

Ref31 = -1 [fixed throughout the run]

Per31 = 0.9*(Per21 - Per22) [constant arbitrarily chosen]

Out31 = Out31 + Delay3*(Gain3*(Ref31 - Per31) - Out31)

[this is the basic form of every Output function below]

Ref32 = 1 [fixed]

Per32 = 0.2*(Per21 + Per22) [arbitrary constant]

Out32 = ... [same form as Out31, but integrating Out32]

AvgErr3 = ((Ref31 - Per31) + (Ref32 - Per32))/2

Level 2 --

Delay2 = 0.0001

Gain2 = 500

Ref21 = (Out31 + Out32)

Per21 = 0.8*(Per11 - (2*Per12) + Per13) [arbitrary constant,

& arbitrary function, but with Per11 as an independently

changing degree of freedom]

Out21 = Out21 + Delay2*(Gain2*(Ref21 - Per21) - Out21)

Ref22 = (-Out31 + Out32)

Per22 = 0.5*(Per12 - Per13 + Per14) [some overlap with Per21,

but with Per14 as an independently changing df]

Out22 = ... [same form as Out21, but integrating Out22]

AvgErr2 = ((Ref21 - Per21) + (Ref22 - Per22))/2

Level 1 --

Delay1 = 0.01

Gain1 = 50

Ref11 = Out21

Per11 = Out11 + Dis1

Out11 = Out11 + Delay1*(Gain1*(Ref11 - Per11) - Out11)

Ref12 = (-Out21 + Out22)

Per12 = Out12 + Dis2

Out12 = ... [same form as Out11, but integrating Out12]

Ref13 = (Out21 - Out22)

Per13 = Out13 + Dis3

Out13 = ... [same form as Out11, but integrating Out13]

Ref14 = Out22 [on 1st run, leading to stable control]

= -Out22 [on 2nd run, leading to runaway feedback]

Per14 = Out14 + Dis4

Out14 = ... [same form as Out11, but integrating Out14]

AvgErr1 = ((Ref11 - Per11) + (Ref12 - Per12) + (Ref13 - Per13) +

(Ref14 - Per14))/4

Environment --

[disturbances were arbitrarily chosen & held constant]

e.g. --

Dis1 = 10

Dis2 = 22

Dis3 = 25

Dis4 = 15

As I said, Ref14 was the only cell that changed from run 1 to

run 2, with simply a change of sign. That seemed to be enough,

however, to have a positive feedback situation propagate upward,

not just to Level 2, but also to Level 3. And as the ECUs at

those levels tried unsuccessfully to bring their perceptions

under control, it led to greater & greater outputs propagating

back down to Level 1.

At any rate, I hope this provides a way free from the verbal

speculations to start to look at this matter of positive

feedback & a propagating "bomb in the hierarchy." The runaway

condition here was implemented entirely within the hierachy,

seemingly via conflicts among ECUs. It did not use overwhelming

disturbances to bring it about. Nor did it use limiting

conditions on a negative feedback path to unmask a hidden

positive feedback path. I don't know the answer to Bill's

question about whether qualitatively different invariants

at higher levels would tend to hold the runaway condition

in check. Perhaps the upwardly spreading runaway of this

simulation is simply an artifact of the linear nature of

all the functions. Obviously, there is no reorganization

simulated in this model to correct the positive feedback

once it is underway.

I'm hoping, Rick, that you can take this & pursue it further,

by putting it into one of your spreadsheet models, & then

experiment with permutations or uncover the limitations of

what is happening. I'm afraid I won't have much more time

than to look on from the sidelines at this point. I do think

it is perhaps an advantage at this stage in the modeling that

the perceptual functions are quite arbitrary, because then

we're not pre-deciding what the phenomenon consists of. We

can study it in elementary form, & then see if it maps onto

any real world examples.

Thanks, everyone, for the discussions of these matters, Hope

this is helpful to those who are interested.

All the best,

Erling