Bomb in the Hierarchy Simulation
[Subject was: Back to Control]
[From Erling Jorgensen (2005.02.22 0100 EST)]
I'm coming to the discussion late, but I can maybe provide one
way to sort out some of the ideas about positive feedback by way
of a simulation. The recent discussion of Martin's notion of
"a bomb in the hierarchy" reminded me of some modeling I did
some years ago (with "toy models", to use Bruce G.'s term).
I launched off of some remarks Martin T. included in his
"Principles for Intelligent Human-Computer Interaction: A
Tutorial on Layered Protocol Theory" (1993, North York, Ontario:
DCIEM). I then implemented them in a simple spreadsheet
simulation, based on Rick M.'s work & adapted to MS Works
spreadsheets (rather than Lotus).
There have been some recent calls for models of positive feedback,
rather than just verbal proposals, so maybe this can advance the
discussion some.
For instance, Martin & Rick had this recent exchange:
Martin Taylor (2005.02.18.09.26)
Rick Marken (2005.02.18.0830)
The real explosion happens when that reversal causes a
higher-level control system to go into positive feedback,
and so on up the chain.
This is what I would want to see demonstrated in a model.
I can imagine runaway error (as per the runaway after sign
reversal) increasing error at higher levels. But I don't see
how the reversal-caused runaway could cause a higher-level
control system to go into positive feedback.
I'm defining positive feedback basically the way Bill did --
Bill Powers (2005.02.19.1945 MST)
If what you do increases your error and leads to your
doing it even more, that's positive feedback, isn't it?
This is what I think I saw in my simulations, & it did seem
to spread to higher levels. I don't have ready access to the
Works files I was using, but I did find some of the results.
I'm also not sure how to make a simulation available to others
on CSGNet, but I thought maybe I could provide the basic
equations (as Rick requested in one of his posts), & maybe
Rick can readapt it to his spreadsheets & explore its properties.
The basic phenomenon is as Martin described it --
Martin Taylor 2005.02.18.01.21
The Bomb is something that exists in a complex environment
with many feedback paths connecting output and perception.
The strength of the feedback paths can vary, and on some
occasions the sign of the overall feedback can change. That
control system then goes into a positive feedback mode, and
if it is in the feedback path of a higher-level control system,
the higher one might also go into a positive feedback mode.
I set up the simulation as a set of eight elementary control
units, distributed in three hierarchical levels -- two in the
highest level, two in the middle level, & four in the lowest
level. The equations were fairly arbitrary & were not meant
to "represent" any perceptions in particular. They were just
linear functions of other cells in the spreadsheet. They were
actually based on the way Martin had set up a theoretical
scenario on p. 30 of the Layered Protocol tutorial (cited above).
Before listing the equations, let me give a general sense of
the set-up. Each control unit was implemented as a set of three
cells, labeled Ref, Per, & Out, (for Reference, Perception, &
Output, obviously). Each level used the same Delay & Gain
factors for the ECU's on that level, with higher levels having
slower Delay & greater Gain than the levels below. The
"environment" was very simplified, consisting of four arbitrary
disturbances (Dis), one for each of the four lowest level ECUs.
I think the way the equations were set up did involve potential
conflict, but in the first simulation involving stable control
at all three levels, there were alternate ways of compensating
for the conflict. The second situation leading to positive
feedback involved a change of sign for one reference in one of
the lowest ECUs. Only that one cell changed, & it seemed to
lead to a "bomb" of positive feedback, that affected all the
ECUs in the upper two levels. The four lowest level ECUs, by
contrast, were able to maintain adequate control, but with
steadily increasing values for the Ref, Per, & Out of each one.
In other words, the stable control at the lowest level was at a
high price of "wasted energy."
The equations below are not listed as spreadsheet formulas,
per se, since it's hard to follow the allusions to different
cells, (& each application uses slightly different conventions
at any rate). Since the simulation was also leading to
runaway feedback values in many of the cells, I also incorporated
what I call a "reset button" into the equation for every cell,
so I could modify & restart a run without much problem. The
reset button consisted of multiplying each equation by a certain
cell (anchored as $B$1), which could be manually set at 0 or 1.
The runaway feedback was basically monitored in terms of three
cells calculating the average error for each level of the
hierarchy. In my spreadsheet application, I did not have a
way to import & retain those values, so basically I would stop
after 50 or 100 iterations & print the values in the whole
spreadsheet, so I could compare them over time. I manually
inserted the number of iterations in a cell before printing.
I also manually recorded each average error at the start of
a run, which was after stable control had initially been
established, so I would have it as a point of reference.
With that as preface & context, these were the basic form
of the equations that I used. Each variable has one or two
numbers which should be read as subscripts (i,j), the first
of which designates the hierarchical level, the second of
which designates which ECU at that level.
Level 3 --
Delay3 = 0.00001
Gain3 = 1000
Ref31 = -1 [fixed throughout the run]
Per31 = 0.9*(Per21 - Per22) [constant arbitrarily chosen]
Out31 = Out31 + Delay3*(Gain3*(Ref31 - Per31) - Out31)
[this is the basic form of every Output function below]
Ref32 = 1 [fixed]
Per32 = 0.2*(Per21 + Per22) [arbitrary constant]
Out32 = ... [same form as Out31, but integrating Out32]
AvgErr3 = ((Ref31 - Per31) + (Ref32 - Per32))/2
Level 2 --
Delay2 = 0.0001
Gain2 = 500
Ref21 = (Out31 + Out32)
Per21 = 0.8*(Per11 - (2*Per12) + Per13) [arbitrary constant,
& arbitrary function, but with Per11 as an independently
changing degree of freedom]
Out21 = Out21 + Delay2*(Gain2*(Ref21 - Per21) - Out21)
Ref22 = (-Out31 + Out32)
Per22 = 0.5*(Per12 - Per13 + Per14) [some overlap with Per21,
but with Per14 as an independently changing df]
Out22 = ... [same form as Out21, but integrating Out22]
AvgErr2 = ((Ref21 - Per21) + (Ref22 - Per22))/2
Level 1 --
Delay1 = 0.01
Gain1 = 50
Ref11 = Out21
Per11 = Out11 + Dis1
Out11 = Out11 + Delay1*(Gain1*(Ref11 - Per11) - Out11)
Ref12 = (-Out21 + Out22)
Per12 = Out12 + Dis2
Out12 = ... [same form as Out11, but integrating Out12]
Ref13 = (Out21 - Out22)
Per13 = Out13 + Dis3
Out13 = ... [same form as Out11, but integrating Out13]
Ref14 = Out22 [on 1st run, leading to stable control]
= -Out22 [on 2nd run, leading to runaway feedback]
Per14 = Out14 + Dis4
Out14 = ... [same form as Out11, but integrating Out14]
AvgErr1 = ((Ref11 - Per11) + (Ref12 - Per12) + (Ref13 - Per13) +
(Ref14 - Per14))/4
Environment --
[disturbances were arbitrarily chosen & held constant]
e.g. --
Dis1 = 10
Dis2 = 22
Dis3 = 25
Dis4 = 15
As I said, Ref14 was the only cell that changed from run 1 to
run 2, with simply a change of sign. That seemed to be enough,
however, to have a positive feedback situation propagate upward,
not just to Level 2, but also to Level 3. And as the ECUs at
those levels tried unsuccessfully to bring their perceptions
under control, it led to greater & greater outputs propagating
back down to Level 1.
At any rate, I hope this provides a way free from the verbal
speculations to start to look at this matter of positive
feedback & a propagating "bomb in the hierarchy." The runaway
condition here was implemented entirely within the hierachy,
seemingly via conflicts among ECUs. It did not use overwhelming
disturbances to bring it about. Nor did it use limiting
conditions on a negative feedback path to unmask a hidden
positive feedback path. I don't know the answer to Bill's
question about whether qualitatively different invariants
at higher levels would tend to hold the runaway condition
in check. Perhaps the upwardly spreading runaway of this
simulation is simply an artifact of the linear nature of
all the functions. Obviously, there is no reorganization
simulated in this model to correct the positive feedback
once it is underway.
I'm hoping, Rick, that you can take this & pursue it further,
by putting it into one of your spreadsheet models, & then
experiment with permutations or uncover the limitations of
what is happening. I'm afraid I won't have much more time
than to look on from the sidelines at this point. I do think
it is perhaps an advantage at this stage in the modeling that
the perceptual functions are quite arbitrary, because then
we're not pre-deciding what the phenomenon consists of. We
can study it in elementary form, & then see if it maps onto
any real world examples.
Thanks, everyone, for the discussions of these matters, Hope
this is helpful to those who are interested.
All the best,
Erling