Bomb in the Hierarchy Simulation

Bomb in the Hierarchy Simulation
[Subject was: Back to Control]

[From Erling Jorgensen (2005.02.22 0100 EST)]

I'm coming to the discussion late, but I can maybe provide one
way to sort out some of the ideas about positive feedback by way
of a simulation. The recent discussion of Martin's notion of
"a bomb in the hierarchy" reminded me of some modeling I did
some years ago (with "toy models", to use Bruce G.'s term).

I launched off of some remarks Martin T. included in his
"Principles for Intelligent Human-Computer Interaction: A
Tutorial on Layered Protocol Theory" (1993, North York, Ontario:
DCIEM). I then implemented them in a simple spreadsheet
simulation, based on Rick M.'s work & adapted to MS Works
spreadsheets (rather than Lotus).

There have been some recent calls for models of positive feedback,
rather than just verbal proposals, so maybe this can advance the
discussion some.

For instance, Martin & Rick had this recent exchange:

Martin Taylor (2005.02.18.09.26)

Rick Marken (2005.02.18.0830)

The real explosion happens when that reversal causes a
higher-level control system to go into positive feedback,
and so on up the chain.

This is what I would want to see demonstrated in a model.
I can imagine runaway error (as per the runaway after sign
reversal) increasing error at higher levels. But I don't see
how the reversal-caused runaway could cause a higher-level
control system to go into positive feedback.

I'm defining positive feedback basically the way Bill did --

Bill Powers (2005.02.19.1945 MST)

If what you do increases your error and leads to your
doing it even more, that's positive feedback, isn't it?

This is what I think I saw in my simulations, & it did seem
to spread to higher levels. I don't have ready access to the
Works files I was using, but I did find some of the results.
I'm also not sure how to make a simulation available to others
on CSGNet, but I thought maybe I could provide the basic
equations (as Rick requested in one of his posts), & maybe
Rick can readapt it to his spreadsheets & explore its properties.

The basic phenomenon is as Martin described it --

Martin Taylor 2005.02.18.01.21

The Bomb is something that exists in a complex environment
with many feedback paths connecting output and perception.
The strength of the feedback paths can vary, and on some
occasions the sign of the overall feedback can change. That
control system then goes into a positive feedback mode, and
if it is in the feedback path of a higher-level control system,
the higher one might also go into a positive feedback mode.

I set up the simulation as a set of eight elementary control
units, distributed in three hierarchical levels -- two in the
highest level, two in the middle level, & four in the lowest
level. The equations were fairly arbitrary & were not meant
to "represent" any perceptions in particular. They were just
linear functions of other cells in the spreadsheet. They were
actually based on the way Martin had set up a theoretical
scenario on p. 30 of the Layered Protocol tutorial (cited above).

Before listing the equations, let me give a general sense of
the set-up. Each control unit was implemented as a set of three
cells, labeled Ref, Per, & Out, (for Reference, Perception, &
Output, obviously). Each level used the same Delay & Gain
factors for the ECU's on that level, with higher levels having
slower Delay & greater Gain than the levels below. The
"environment" was very simplified, consisting of four arbitrary
disturbances (Dis), one for each of the four lowest level ECUs.

I think the way the equations were set up did involve potential
conflict, but in the first simulation involving stable control
at all three levels, there were alternate ways of compensating
for the conflict. The second situation leading to positive
feedback involved a change of sign for one reference in one of
the lowest ECUs. Only that one cell changed, & it seemed to
lead to a "bomb" of positive feedback, that affected all the
ECUs in the upper two levels. The four lowest level ECUs, by
contrast, were able to maintain adequate control, but with
steadily increasing values for the Ref, Per, & Out of each one.
In other words, the stable control at the lowest level was at a
high price of "wasted energy."

The equations below are not listed as spreadsheet formulas,
per se, since it's hard to follow the allusions to different
cells, (& each application uses slightly different conventions
at any rate). Since the simulation was also leading to
runaway feedback values in many of the cells, I also incorporated
what I call a "reset button" into the equation for every cell,
so I could modify & restart a run without much problem. The
reset button consisted of multiplying each equation by a certain
cell (anchored as $B$1), which could be manually set at 0 or 1.

The runaway feedback was basically monitored in terms of three
cells calculating the average error for each level of the
hierarchy. In my spreadsheet application, I did not have a
way to import & retain those values, so basically I would stop
after 50 or 100 iterations & print the values in the whole
spreadsheet, so I could compare them over time. I manually
inserted the number of iterations in a cell before printing.
I also manually recorded each average error at the start of
a run, which was after stable control had initially been
established, so I would have it as a point of reference.

With that as preface & context, these were the basic form
of the equations that I used. Each variable has one or two
numbers which should be read as subscripts (i,j), the first
of which designates the hierarchical level, the second of
which designates which ECU at that level.

Level 3 --

Delay3 = 0.00001
Gain3 = 1000

Ref31 = -1 [fixed throughout the run]
Per31 = 0.9*(Per21 - Per22) [constant arbitrarily chosen]
Out31 = Out31 + Delay3*(Gain3*(Ref31 - Per31) - Out31)
  [this is the basic form of every Output function below]

Ref32 = 1 [fixed]
Per32 = 0.2*(Per21 + Per22) [arbitrary constant]
Out32 = ... [same form as Out31, but integrating Out32]

AvgErr3 = ((Ref31 - Per31) + (Ref32 - Per32))/2

Level 2 --

Delay2 = 0.0001
Gain2 = 500

Ref21 = (Out31 + Out32)
Per21 = 0.8*(Per11 - (2*Per12) + Per13) [arbitrary constant,
  & arbitrary function, but with Per11 as an independently
  changing degree of freedom]
Out21 = Out21 + Delay2*(Gain2*(Ref21 - Per21) - Out21)

Ref22 = (-Out31 + Out32)
Per22 = 0.5*(Per12 - Per13 + Per14) [some overlap with Per21,
  but with Per14 as an independently changing df]
Out22 = ... [same form as Out21, but integrating Out22]

AvgErr2 = ((Ref21 - Per21) + (Ref22 - Per22))/2

Level 1 --

Delay1 = 0.01
Gain1 = 50

Ref11 = Out21
Per11 = Out11 + Dis1
Out11 = Out11 + Delay1*(Gain1*(Ref11 - Per11) - Out11)

Ref12 = (-Out21 + Out22)
Per12 = Out12 + Dis2
Out12 = ... [same form as Out11, but integrating Out12]

Ref13 = (Out21 - Out22)
Per13 = Out13 + Dis3
Out13 = ... [same form as Out11, but integrating Out13]

Ref14 = Out22 [on 1st run, leading to stable control]
         = -Out22 [on 2nd run, leading to runaway feedback]
Per14 = Out14 + Dis4
Out14 = ... [same form as Out11, but integrating Out14]

AvgErr1 = ((Ref11 - Per11) + (Ref12 - Per12) + (Ref13 - Per13) +
                  (Ref14 - Per14))/4

Environment --
  [disturbances were arbitrarily chosen & held constant]
e.g. --
Dis1 = 10
Dis2 = 22
Dis3 = 25
Dis4 = 15

As I said, Ref14 was the only cell that changed from run 1 to
run 2, with simply a change of sign. That seemed to be enough,
however, to have a positive feedback situation propagate upward,
not just to Level 2, but also to Level 3. And as the ECUs at
those levels tried unsuccessfully to bring their perceptions
under control, it led to greater & greater outputs propagating
back down to Level 1.

At any rate, I hope this provides a way free from the verbal
speculations to start to look at this matter of positive
feedback & a propagating "bomb in the hierarchy." The runaway
condition here was implemented entirely within the hierachy,
seemingly via conflicts among ECUs. It did not use overwhelming
disturbances to bring it about. Nor did it use limiting
conditions on a negative feedback path to unmask a hidden
positive feedback path. I don't know the answer to Bill's
question about whether qualitatively different invariants
at higher levels would tend to hold the runaway condition
in check. Perhaps the upwardly spreading runaway of this
simulation is simply an artifact of the linear nature of
all the functions. Obviously, there is no reorganization
simulated in this model to correct the positive feedback
once it is underway.

I'm hoping, Rick, that you can take this & pursue it further,
by putting it into one of your spreadsheet models, & then
experiment with permutations or uncover the limitations of
what is happening. I'm afraid I won't have much more time
than to look on from the sidelines at this point. I do think
it is perhaps an advantage at this stage in the modeling that
the perceptual functions are quite arbitrary, because then
we're not pre-deciding what the phenomenon consists of. We
can study it in elementary form, & then see if it maps onto
any real world examples.

Thanks, everyone, for the discussions of these matters, Hope
this is helpful to those who are interested.

All the best,
Erling

[From Rick Marken (2005.02.22.0900)]

Erling Jorgensen (2005.02.22 0100 EST) --

There have been some recent calls for models of positive feedback,
rather than just verbal proposals, so maybe this can advance the
discussion some.

For instance, Martin & Rick had this recent exchange:

Martin Taylor (2005.02.18.09.26)

Rick Marken (2005.02.18.0830)

The real explosion happens when that reversal causes a
higher-level control system to go into positive feedback,
and so on up the chain.

This is what I would want to see demonstrated in a model.
I can imagine runaway error (as per the runaway after sign
reversal) increasing error at higher levels. But I don't see
how the reversal-caused runaway could cause a higher-level
control system to go into positive feedback.

I'm defining positive feedback basically the way Bill did --

Bill Powers (2005.02.19.1945 MST)

If what you do increases your error and leads to your
doing it even more, that's positive feedback, isn't it?

This is what I think I saw in my simulations, & it did seem
to spread to higher levels. I don't have ready access to the
Works files I was using, but I did find some of the results.
I'm also not sure how to make a simulation available to others
on CSGNet, but I thought maybe I could provide the basic
equations (as Rick requested in one of his posts), & maybe
Rick can readapt it to his spreadsheets & explore its properties.

Thanks, Erling, for this nice, substantive post.

It turns out to be rather easy to test the Bomb idea using my spreadsheet
hierarchy. Just download a copy from the net (I'm sure it will work in MS
works) and follow along with me.

I tested the bomb by forcing one level 2 system into positive feedback mode.
I did this for system 5 level 2 (cells H7-H9). Positive feedback is created
by changing the output equation in cell H9 from what it is:

=IF(H6="*",H9,H9+$A$9*($B$9*(H7-H8)-H9))

to

=IF(H6="*",H9,H9+$A$9*(1*(H8-H7)-H9))

H7 is the reference and H8 is the perceptual input to the system, so by
subtracting H7 from H8 rather than H8 from H7 the sign of the feedback is
changed from negative to positive. $B$9 is the gain and I changed that to 1
so the positive feedback would not be too "explosive".

If you make these changes and run the model (by hitting Function key 9) you
will see that the perceptual signal for this system (the value in H8) keeps
moving _away_ from the reference (in H7). This is positive feedback; the
system is acting to correct error (the difference between H7 and H8) by
acting in a way that actually increases error.

Note that this positive feedback situation has no effect on other control
systems at the same or at lower levels. In fact it only has an effect on
_one_ higher level system -- the one that is controlling a relationship
between the perception in H8 (the one driven by positive feedback) and the
perception in I8 (the one controlled by the system next to the one in
positive feedback).

This system -- which is system 5 at level three -- does seem to be in
positive feedback mode. It is acting (by changing the reference sent to
system 5 , level 2) in a way that actually makes the controlled perception
(H8>I8) move _away_ from the reference value (which is that H8>I8 = TRUE; it
is acting to make H8<I8).

But I'm not sure this is really positive feedback because when I reverse the
polarity of the higher level system (by changing the equation in cell H5
from

=IF(H2="*",H5,H5+$A$9*($B$5*(H3-H4)-H5))

to

=IF(H2="*",H5,H5+$A$9*($B$5*(H4-H3)-H5))

it does not fix things up. The system still acts to move the relative values
of H8 and I8 _away_ from the reference.

So my conclusion is that a positive feedback Bomb in the hierarchy leads to
loss of control of the perception that was controlled by the now positive
feedback system and loss of control by higher level systems that are trying
to control perceptions that are a function of the now uncontrolled
perception.

So Martin is right that a positive feedback loop in a control hierarchy will
have effects that are reflected up through higher levels of the hierarchy.
These effects are only on systems controlling perceptions that are a
function of the lower level perception that is uncontrolled. And I suppose
you could say that positive feedback is propagated up to these higher level
systems. But is seems that what is propagated to these higher level systems
is inability to control because, regardless of the polarity of the
connection between these systems and their controlled perceptions, because
what these systems do (in terms of sending reference inputs to lower level
systems) just makes things worse due to the lower level positive feedback
situation.

Best

Rick

···

--
Richard S. Marken
MindReadings.com
Home: 310 474 0313
Cell: 310 729 1400

--------------------

This email message is for the sole use of the intended recipient(s) and
may contain privileged information. Any unauthorized review, use,
disclosure or distribution is prohibited. If you are not the intended
recipient, please contact the sender by reply email and destroy all copies
of the original message.

[From Rick Marken (2005.02.22.1200)]

Rick Marken (2005.02.22.0900)]

So Martin is right that a positive feedback loop in a control hierarchy will
have effects that are reflected up through higher levels of the hierarchy.
These effects are only on systems controlling perceptions that are a
function of the lower level perception that is uncontrolled. And I suppose
you could say that positive feedback is propagated up to these higher level
systems. But is seems that what is propagated to these higher level systems
is inability to control because, regardless of the polarity of the
connection between these systems and their controlled perceptions, because
what these systems do (in terms of sending reference inputs to lower level
systems) just makes things worse due to the lower level positive feedback
situation.

I doubt that this was comprehensible at all. I think the best way to state
the conclusion I come to based on my model excursions with positive feedback
is that Martin is _exactly_ right: a lower level positive feedback system
can creates a virtual positive feedback situation for a higher level system.
I think this is a fascinating observation regarding the nature of
hierarchical control systems. Nice going Martin!

Best

Rick

···

--
Richard S. Marken
MindReadings.com
Home: 310 474 0313
Cell: 310 729 1400

--------------------

This email message is for the sole use of the intended recipient(s) and
may contain privileged information. Any unauthorized review, use,
disclosure or distribution is prohibited. If you are not the intended
recipient, please contact the sender by reply email and destroy all copies
of the original message.

[From Bill Powers (2005.02.22.1435 MST)]

Rick Marken (2005.02.22.1200) --

I think the best way to state the conclusion I come to based on my model excursions with positive feedback is that Martin is _exactly_ right: a lower level positive feedback system can creates a virtual positive feedback situation for a higher level system. I think this is a fascinating observation regarding the nature of hierarchical control systems. Nice going Martin!

Well said. I'm still focussed on the implications of the combined negative and positive loops, however, where negative feedback is stuff like holding talks for negotiating and declaring cease-fires, while positive feedback results from one dumb radical breaking the cease-fire by taking it on himself to blow up a bus, which leads to retaliation by a dumb army officer, which leads to blowing up two buses and a restaurant, which leads to blowing up a block of buildings, which effectively destroys the peace effort until it can be reassembled.

Martin, I hope you will reflect on all this and grace the 2005 CSG meeting with a talk on this subject. Not at the expense of other things you want to talk about, but in addition.

Best,

Bill P.

Re: Bomb in the Hierarchy Simulation
[From Erling Jorgensen (2005.02.22 1630 EST)]

Rick Marken (2005.02.22.0900)

Thanks, Rick, for a look at your spreadsheet.

I used HierarchicalControl.xls file at the ftp site linked to
your Web site. The formulas did not seem to be exactly the
same as the ones referred to in your post, but I could follow
along.

I tested the bomb by forcing one level 2 system into positive
feedback mode. I did this for system 5 level 2 (cells H7-H9).
Positive feedback is created by changing the output equation in
cell H9 from what it is:

=IF(H6="*",H9,H9+$A$9*($B$9*(H7-H8)-H9))

to

=IF(H6="*",H9,H9+$A$9*(1*(H8-H7)-H9))

H7 is the reference and H8 is the perceptual input to the system,
so by subtracting H7 from H8 rather than H8 from H7 the sign of
the feedback is changed from negative to positive.

Yes. This is comparable to the way I did it in my simulation,
by changing the sign on the reference (Ref14) of one control
system in the next lower level, which was directly dependent on
the output (Out22) of a control system at level 2.

I think the effect of your change would be that any P(2,5)
perceptions below R(2,5) would keep getting lower (as it did
when I ran your demo), and any P(2,5) perceptions above R(2,5)
would keep getting greater. In either case, P(2,5) would be
moving away from R(2,5) rather than closer, which is how we
define positive feedback.

It was a little hard to tell how those P(2,i) perceptions were
being derived, (as well as the level one R(1,i) references),
because I didn't understand the matrix way of calculating them.

But there definitely was positive feedback occurring for the
control system at cells H7 to H9, and it also showed up as a
steadily increasing average error for that whole level.

Note that this positive feedback situation has no effect on
other control systems at the same or at lower levels. In fact
it only has an effect on _one_ higher level system -- the one
that is controlling a relationship between the perception in H8
(the one driven by positive feedback) and the perception in I8

Yes, other systems not directly affected seemed to maintain
adequate control. In my simulation, the positive feedback
spread further. It did not actually affect the system where
I reversed the sign (at Ref14 in level 1), but it constrained
the degrees of freedom for a system above it (Per22 in level 2)
that depended on the perceptual results of that lower control
system, leading to positive feedback for ECU22.

I am not sure (in my simulation) how control of perception Per21
in the adjoining control system at level 2 also came to show
positive feedback, because it was not directly linked to ECU22 or
ECU14. Because of the way the equations were written, I believe
the positive feedback spread upward to the two level 3 ECUs, which
then affected ECU21 (with a positive feedback error a little less
than that in ECU22.) On the other hand, all the level 1 systems
in my simulation (ECU11 thru ECU14) had steadily increasing
reference values and output values, although they were able to
maintain adequate control of their respective perceptions. It
would be good to find out whether those steadily increasing
(controlled) level 1 perceptions (Per11 thru Per13) were making
it too difficult for ECU21 to maintain negative feedback control.
In other words, I don't know whether the effect on ECU21 was
positive feedback from above, or inadequate degrees of freedom
from below.

Back to your post & demo --

This system -- which is system 5 at level three -- does seem
to be in positive feedback mode. It is acting (by changing the
reference sent to system 5 , level 2) in a way that actually
makes the controlled perception (H8>I8) move _away_ from the
reference value

I agree that that system appears to lose control of its perception.
It's hard to tell if it is positive feedback, because with the
reference R(3,5) & perception P(3.5) returning TRUE or FALSE
values (+1 or -1), it's a little hard to tell whether the
perception consistently stays opposite to the reference when
the multi-iterations occur.

But I'm not sure this is really positive feedback because when
I reverse the polarity of the higher level system (by changing
the equation in cell H5 from

=IF(H2="*",H5,H5+$A$9*($B$5*(H3-H4)-H5))

to

=IF(H2="*",H5,H5+$A$9*($B$5*(H4-H3)-H5))

it does not fix things up. The system still acts to move the
relative values of H8 and I8 _away_ from the reference.

I did not find this to be the case. When I made the polarity
adjustment you suggested, it did seem to restore consistent
negative feedback to that Level 3 system P(3,5).

Because when I did it, the sign change _did_ lead to negative
feedback, that is evidence to me that the _previous_ mode was
indeed positive feedback, despite the masking due to the
TRUE\FALSE logical conditions of the equations.

So my conclusion is that a positive feedback Bomb in the
hierarchy leads to loss of control of the perception that
was controlled by the now positive feedback system ...

Agreed.

... and loss of control by higher level systems that are trying
to control perceptions that are a function of the now uncontrolled
perception.

And based on running my simulations & your demo, I would still
call the form of that loss of control positive feedback.

So Martin is right that a positive feedback loop in a control
hierarchy will have effects that are reflected up through higher
levels of the hierarchy. These effects are only on systems
controlling perceptions that are a function of the lower level
perception that is uncontrolled. And I suppose you could say
that positive feedback is propagated up to these higher level
systems.

These are the working conclusions I am operating with for now,
until more tinkering & adjustment of parameters establish the
limits of this phenomenon.

I see by your subsequent post [Rick Marken (2005.02.22.1200)]
that you concur --

Martin is _exactly_ right: a lower level positive feedback system
can create a virtual positive feedback situation for a higher
level system. I think this is a fascinating observation regarding
the nature of hierarchical control systems. Nice going Martin!

Agreed on both counts. Now, is there any way you could play
more with the equations I provided, to tease out in that case
whether it was a top-down or bottom-up effect that destabilized
ECU21? What if I say "pretty please"?...

Thanks for your efforts, Rick. (You too, Martin.)

All the best,
Erling

[Martin Taylor 2005.02.22.17.40]

[From Bill Powers (2005.02.22.1435 MST)]

Martin, I hope you will reflect on all this and grace the 2005 CSG meeting with a talk on this subject. Not at the expense of other things you want to talk about, but in addition.

["this" meaning the Bomb in the hierarchy]. I'd be happy to.

I'm going to try not to think about it until after mid-May. Then, with luck, I'll have time to try some experiments. If you can remind me, around the beginning of June (my mind having flown elsewhere in the meantime), I'll see if I can try something.

Do you know a program called "AgentSheets"?. It's a kind of spreadsheet for cellular automata. I was thinking of setting it up like a hierarchy, and running visual Bombs in it. But not now, and not soon. You can find out about it at <http://agentsheets.com/>. It costs $120.

Martin

[From Rick Marken (2005.02.22.1615)]

Erling Jorgensen (2005.02.22 1630 EST) --

Rick Marken (2005.02.22.0900) --

Martin is _exactly_ right: a lower level positive feedback system
can create a virtual positive feedback situation for a higher
level system. I think this is a fascinating observation regarding
the nature of hierarchical control systems. Nice going Martin!

Agreed on both counts. Now, is there any way you could play
more with the equations I provided, to tease out in that case
whether it was a top-down or bottom-up effect that destabilized
ECU21? What if I say "pretty please"?...

I'm sorry, I only do spreadsheets;-)

I will say that the only effects of positive feedback that I observed are
bottom up. I believe, however, that they can also be top down if there are
limits on the value to which the system can bring it's perceptions, due to
physical constraints or output limitations. So a positive feedback system
could result in the setting of impossible references for lower level
perceptions, a top down problem. This can't happen in my spreadsheet because
the control hierarchy has no realistic physical or output constraints. So
the lower level systems are a able to do whatever it takes to get their
perceptions into a match with whatever references are set by the higher
level systems.

Best

Rick

···

--
Richard S. Marken
MindReadings.com
Home: 310 474 0313
Cell: 310 729 1400

--------------------

This email message is for the sole use of the intended recipient(s) and
may contain privileged information. Any unauthorized review, use,
disclosure or distribution is prohibited. If you are not the intended
recipient, please contact the sender by reply email and destroy all copies
of the original message.

[From Rick Marken (2005.02.22.1615)]

Erling Jorgensen (2005.02.22 1630 EST) --

Rick Marken (2005.02.22.0900) --

Martin is _exactly_ right: a lower level positive feedback system
can create a virtual positive feedback situation for a higher
level system. I think this is a fascinating observation regarding
the nature of hierarchical control systems. Nice going Martin!

Agreed on both counts. Now, is there any way you could play
more with the equations I provided, to tease out in that case
whether it was a top-down or bottom-up effect that destabilized
ECU21? What if I say "pretty please"?...

I'm sorry, I only do spreadsheets;-)

I will say that the only effects of positive feedback that I observed are
bottom up. I believe, however, that they can also be top down if there are
limits on the value to which the system can bring it's perceptions, due to
physical constraints or output limitations.

If there are sufficient side effects, an exponentially increasing output from one control unit will impose an exponentially increasing disturbance on some other perception. If its control unit can counter that, there's no problem. But if it can't, then to a higher-level system that uses its perceptual signal it will look like a unit that's gone into positive feedback. In that way, the Bomb blast can propagaget downward (less easily than upward, but that's true of real explosions, too:-)

It doesn't depend on anything reaching a limit.

Martin

[From Bjorn Simonsen (2005.03.09,13:25 EST)]

From Rick Marken (2005.02.22.0900)
I tested the bomb by forcing one level 2 system into positive feedback

mode.

I did this for system 5 level 2 (cells H7-H9). Positive feedback is created
by changing the output equation in cell H9 from what it is:

   =IF(H6="*",H9,H9+$A$9*($B$9*(H7-H8)-H9))

to

   =IF(H6="*",H9,H9+$A$9*(1*(H8-H7)-H9))

I think it is the gain that may be responsible for a positive feedback.

You don't need to change the output in cell H9 more than putting the gain to
1.

But it is interesting to see what happens if increase the repetitions to
1000 and play with F9 5-6 times.

[From Bjorn Simonsen (2005.03.09,13:25 EST)]

I have been away from CSG net the last 2-3 weeks and after coming home I
have spent some time on Erling Jorgensen's thread "Bomb in the Hierarchy
Simulation" and the following mails.
When good threads are introduced, I generally ask myself more question than
I can answer. The same happened this time.
Maybe I should refrain from sending this mail because you (all of you) has
become more reserved to positive feedback in natural organisms in your last
mails. But I send it.

Martin Taylor (2005.02.18.09.26)
I'm defining positive feedback basically the way Bill did --

Bill Powers (2005.02.19.1945 MST)

If what you do increases your error and leads to your
doing it even more, that's positive feedback, isn't it?

You wrote in 1978 a wonderful Quantitative analysis of Purposive Systems:
.... in Psychological Review, Bill (1978. Vol 85, No. 5, 417-435. There you
considered a behaving system in relationship to an environment and made a
Qusi-static Analysis. You evolved two completely general equations;
(7) g(qo) = qi*+(UV/(1-UV)*h(qd) and (8) qi =qi* + h(qd)/(1-UV). qi* is
defined as the value of qi when there is no net disturbance. And UV is a
dimensionless (and variable) number customarily called the loop gain.

I am not so fond of your definition of positive feedback above because I
can't find "what you do" in the equations. Of course g(qo) may be perceived
as "what you do". It is the effect of the output on the input quantity. But
I think we in CSG shall be careful and not express that it is our actions
that cause behavior (doing even more).
I don't think it is "what we do" that increases error, I think it is the
value of UV. You put it so nice into words in your section "Type P: Positive
Loop Gain". And if we talk about natural systems I agree (with your "Type P
section) that they have muscles that fatigue and that we may expect natural
P systems to be rare (I would say very rare, but I might be wrong).

From Bill Powers (2005.02.26,1201 MST)

The same thing would happen when we look at different levels of
discreteness. What would positive feedback look like at the level
where we name categories?

I don't know if I understand your question correct. Maybe somebody would say
that Huntington chorea and Parkinson's disease are examples of positive
feedback at the event level. I know too little about what is wrong and
where, but I think the illness is a result of not functioning nerve cells.
And then the loop gain in other loops result in the symptoms.

I also point out that there can be positive feedback at level N without

there

being positive feedback at any lower level. All it takes is that the sign

of the

relationship between output and input reverse at the higher level. This

does

not alter the sign of any lower loop. So a Bomb can exist at a higher level

if

there are many paths through lower systems, but some entail positive
feedback for the higher loop. A change of circumstances can set off that
bomb without causing any lower system to run away.

Are you sure that a Bomb can exist at a higher level if there are many paths
through lower systems? Of course the qi at that level can be greater with
many paths, but there must also be no or very few Renshaw cells close to the
comparator. If there are Renshaw cells, the error will be negative. And a
negative error doesn't form a Bomb.

I am not so fond of your definition of positive feedback above because I
can't find "what you do" in the equations. Of course g(qo) may be perceived
as "what you do". It is the effect of the output on the input quantity. But
I think we in CSG shall be careful and not express that it is our actions
that cause behavior (doing even more).

I don't understand that. Our actions ARE the behavior that other people see.

I don't think it is "what we do" that increases error, I think it is the
value of UV.

If e = r - p, then error is increased when r increases or p decreases. Is that what you mean? Or are you talking about the runaway condition, in which error increases continually because of positive feedback? When there is positive feedback, action increases error and error increases the action, which is what causes the runaway condition. It's not the value of UV (whether it's 10 or 100) that causes positive feedback, but its sign: whether at any moment the number is positive or negative. If the product of U times V is positive, there is positive feedback, so U = 10 and V = 10 gives positive feedback, and the same is true if U = -10 and V = -10. You get negative feedback only when U and V have opposite signs.

>The same thing would happen when we look at different levels of

>discreteness. What would positive feedback look like at the level
>where we name categories?

I don't know if I understand your question correct. Maybe somebody would say
that Huntington chorea and Parkinson's disease are examples of positive
feedback at the event level. I know too little about what is wrong and
where, but I think the illness is a result of not functioning nerve cells.
And then the loop gain in other loops result in the symptoms.

But when the variables are defined so they have only two values, there can't be any runaway condition. A flip-flop has only two states, and once it has flipped or flopped into a state, it can't keep going and be even more in the same state. It just stays the way it is. So categories can't run away, and logical truth can't run away, and principles and system concepts can't run away. This is why I doubt that a "bomb" can propagate very far up the levels.

Are you sure that a Bomb can exist at a higher level if there are many paths
through lower systems? Of course the qi at that level can be greater with
many paths, but there must also be no or very few Renshaw cells close to the
comparator. If there are Renshaw cells, the error will be negative. And a
negative error doesn't form a Bomb.

Positive feedback doesn't require that perceptual signals be positive. A comparator can work equally well if the reference signal is inhibitory (-) and the perceptual signal is excitatory (+) at the comparator.

It isn't the sign of the error that creates positive feedback, but the sign of the effect of a signal on itself via the closed loop. In a two-way system, negative feedback can be present whether the error signal is positive or negative (greater than or less than zero). If a negative error leads to actions that cause the error to become less negative, there is still negative feedback, just as when a positive error leads to actions that causes the error to become less positive. Positive feedback exists only when a positive error leads to actions that cause the error to become more positive, or when a negative error leads to actions that cause the error to become more negative.

Best,

Bill P.

[Martin Taylor 2005.03.10.10.10]
to Bill Powers 2005.03.10.06.50]

It's not the value of UV (whether it's 10 or 100) that causes positive feedback, but its sign: whether at any moment the number is positive or negative. If the product of U times V is positive, there is positive feedback, so U = 10 and V = 10 gives positive feedback, and the same is true if U = -10 and V = -10. You get negative feedback only when U and V have opposite signs.

I think Bjorn was referring to the fact that if the magnitude of the positive feedback is less than unity, the effect of a transient eventually dies away.

Martin

[From Bill Powers (2005.03.10.1158 MST)]

Martin Taylor 2005.03.10.10.10 --

I think Bjorn was referring to the fact that if the magnitude of the positive feedback is less than unity, the effect of a transient eventually dies away.

However, the effect of any disturbance, transient or continued, is amplified by positive feedback, the opposite of control.

Have you had any thoughts about how postive feedback can exist in a system that controls logical variables?

Best,

Bill P.

[From Bill Powers (2005.03.10.1158 MST)]

Martin Taylor 2005.03.10.10.10 --

I think Bjorn was referring to the fact that if the magnitude of the positive feedback is less than unity, the effect of a transient eventually dies away.

However, the effect of any disturbance, transient or continued, is amplified by positive feedback, the opposite of control.

True, but if the gain is less than unity, the integral levels off as time goes to infinity. Visualize the disturbance as an impulse, and the loop as having a frequency-independent time delay. With a gain less than unity, the successive returns of the fed-back impulse diminish to zero. If the disturbance were a step (the integral of an impulse), the result is a series of incrementing steps that diminish in amplitude toward zero. In the practical case, its an exponential rise to some asymptote, not an explosion to infinity, isn't it?

Have you had any thoughts about how postive feedback can exist in a system that controls logical variables?

I've had hardly any thoughts on anything except all the end-of-March deadlines that are coming up, and a possible April 15 presentation.

My question wouldn't be so much how positive feedback would exist, so much a how it would be manifest in behaviour. Consider a flip-flop. It sustains its "on" or "off" value against disturbance because it's equivalent of its perceptual signal is constrained by some limiting process. It's output is maximized, which could be the limitation on the perceptual value. It takes a big contrary disturbance to switch the output. The systyem as a whole isn't in the kind of low-energy stable state that one associates with an analogue loop in good control. It's a high-energy state, which, when there are potentially many degrees of freedom for output and for the inputs to the perceptual function, offers the opportunity for something drastic to happen (as with a column or a knwee joint that buckles under longitudinal force).

I know that's only a sloppy analogy, but it's the line on which my thinking goes when I have a moment to think of such things.

Martin

[From Bjorn Simonsen (2005.03.10,21:35 EST)]
Martin Taylor 2005.03.10.10.10

I think Bjorn was referring to the fact that if the magnitude of the
positive feedback is less than unity, the effect of a transient
eventually dies away.

If you by the magnitude of the feedback think upon the gain, I think upon
0<UV<1.
I don't quite understand your last sentence after comma. When I try with a
gain =0.98 in one cell in Ricks hier.exl, it doesn't die away.

bjorn

[From Bjorn Simonsen (2005.03.10,21:32EST)]
Bill's mail received 2005.03.10,15:25 EST

I am not so fond of your definition of positive feedback above because I
can't find "what you do" in the equations.

I don't understand that. Our actions ARE the behavior that other people

see.

Yes I know that actions are the behavior that other people might see. But
when we are able to express a behaving system to an environment as below,
then we should define concepts using symbols from the equations.
g(qo) = qi*+(UV/(1-UV)*h(qd) and (8) qi =qi* + h(qd)/(1-UV).
Besides I refer to your own words: If UV is positive and not zero, there is
a type P, or positive feedback, relationship between system and
environment".

I don't think it is "what we do" that increases error, I think it is the
value of UV.

Or are you talking about the runaway condition, in
which error increases continually because of positive feedback? When there
is positive feedback, action increases error and error increases the
action, which is what causes the runaway condition. It's not the value of
UV (whether it's 10 or 100) that causes positive feedback, but its sign:
whether at any moment the number is positive or negative. If the product of
U times V is positive, there is positive feedback, so U = 10 and V = 10
gives positive feedback, and the same is true if U = -10 and V = -10. You
get negative feedback only when U and V have opposite signs.

Maybe I am far away, but we talk about Positive feedback. As you say we have
positive feedback when UV>0. I guess we talk about stable systems. Then
0<UV<1.

It isn't the sign of the error that creates positive feedback, but the sign
of the effect of a signal on itself via the closed loop.

Agree.

In a two-way system, negative feedback can be present whether the error
signal is positive or negative (greater than or less than zero). If a

negative error

leads to actions that cause the error to become less negative, there is
still negative feedback, just as when a positive error leads to actions
that causes the error to become less positive.

I have not thought that an error may be negative. How shall I understand
B:CP page 84: "... but since the other input to the neuron is inhibitory,
there can be no effect on the motor neuron unless the reference signal is
assigned a net positive value, greater than or equal to zero".

bjorn

[From Bill Powers (2005.03.10.1409 MST)]

Bjorn Simonsen (2005.03.10,21:32EST) --

Maybe I am far away, but we talk about Positive feedback. As you say we have
positive feedback when UV>0. I guess we talk about stable systems. Then
0<UV<1.

>It isn't the sign of the error that creates positive feedback, but the sign
>of the effect of a signal on itself via the closed loop.

Agree.

>In a two-way system, negative feedback can be present whether the error
>signal is positive or negative (greater than or less than zero). If a
>negative error
>leads to actions that cause the error to become less negative, there is
>still negative feedback, just as when a positive error leads to actions
>that causes the error to become less positive.

I have not thought that an error may be negative. How shall I understand
B:CP page 84: "... but since the other input to the neuron is inhibitory,
there can be no effect on the motor neuron unless the reference signal is
assigned a net positive value, greater than or equal to zero".

This is for a one-way system. A two-way control system can be constructed of two one-way loops, one handling positive errors and the other handling negative errors. Control systems for positioning limbs, maintaining balance, aiming guns, and so on are two-way systems: that is, they can be represented by a single system in which variables can be negative as well as positive.

Note that a "negative error" simply means an error signal that indicates that r - p is negative (in other words, p is greater than r). That requires that p be excitatory and r be inhibitory. So we have

                      r
                        >
                       / \
                      / \
                     > \
                     > - | +
            ---------|--->[Comp] -----> Actuator for positive errors
          / |
p------/ |
        \ |
          \ + |-
            ------>[Comp]-----------> actuator for negative errors

Both p and r are positive signals, and the system will work as a two-way system for all values of p and r between 0 and the maximum possible values. If p is greater than r, the lower error signal will be active and it will drive the actuator for negative errors. The upper error signal will drive the actuator for positive errors (p < r). Error is defined as r - p. Think of the actuators as opposing muscles.

If this system is part of a negative feedback loop, positive feedback will happen if all the plus and minus signs in the diagram are interchanged. Then runaway can happen in either direction: toward more negative or more positive errors.

In the "Quantitative Analysis" paper I assumed that U and V are bidirectional functions, so they can be positive or negative. Positive and negative signs are also determined by the physical direction in which output effects act. You pull up on a rope to raise a bucket out of the well -- but you have to pull down on the rope to raise the bucket if the rope passes over a pulley.

If feedback is positive and 0 < UV < 1, the effects of disturbances on qi will be amplified: the effects will be greater than they would be if the active system were not there. If UV > 1, the variables will spontaneously increase (positively or negatively) until stopped by physical limits.

Negative feedback occurs when UV < 0, with significant control occurring when UV < -5 or so. Then the effects of disturbances will be much less than they would be if the system were not there.

Best,

Bill P.

[From Bill Powers (2005.03.10.1443 MST)]

Martin Taylor 2005.03.10.10.10 --

However, the effect of any disturbance, transient or continued, is amplified by positive feedback, the opposite of control.

True, but if the gain is less than unity, the integral levels off as time goes to infinity.

Leveling off will occur only if this is a leaky integrator and the slowing factor is such that the gain in each trip around the loop is less than 1. If the integrator is not leaky, the integral will increase without limit. Here is the transient case for a gain of 0.5 and a non-leaky integrator:

Disturbance input output delta
                (input + disturbance + output) (gain = 0.5)
1 0 0.5
0 0.5 0.25 -0.25
0 0.75 0.375 +0.125
0 1.125 0.5625 +0.1875
0 1.6875 0.84375 +0.28125
0 2.53125 1.2655625 +0.4218125

The system runs away (delta increases on each step) for any gain greater than zero. With a steady disturbance, of course, it runs away even faster.

Visualize the disturbance as an impulse, and the loop as having a frequency-independent time delay. With a gain less than unity, the successive returns of the fed-back impulse diminish to zero.

That would be true if the integrator is leaky. With a non-leaky integrator the oscillations would continue to grow.

If the disturbance were a step (the integral of an impulse), the result is a series of incrementing steps that diminish in amplitude toward zero.

That's true if the gain is less than 1 and the integrator is leaky enough. In any case, the amplitude of the final effect on the input quantity will be greater than it would be if the control system were not there. If the integrator is not leaky, the system will run away unconditionally.

In the practical case, its an exponential rise to some asymptote, not an explosion to infinity, isn't it?

For gains greater than 1, it's an explosion to a hard physical limit, which isn't the same thing as approaching an asyptote. The output will increase along a positive exponential curve which abruptly becomes horizontal at the limit

My question wouldn't be so much how positive feedback would exist, so much a how it would be manifest in behaviour. Consider a flip-flop. It sustains its "on" or "off" value against disturbance because it's equivalent of its perceptual signal is constrained by some limiting process. It's output is maximized, which could be the limitation on the perceptual value. It takes a big contrary disturbance to switch the output. The system as a whole isn't in the kind of low-energy stable state that one associates with an analogue loop in good control. It's a high-energy state, which, when there are potentially many degrees of freedom for output and for the inputs to the perceptual function, offers the opportunity for something drastic to happen (as with a column or a knee joint that buckles under longitudinal force).

Here's a logical control system with a reference level of 1, and qi = disturbance OR qo:

disturbance qi error qo next qi
     0 0 1 1 1
     0 1 1 0 0
     0 0 1 1 1 (oscillating)
....
     1 1 0 0 0 (step disturbance appears)
     1 1 0 0 0
     1 1 0 0 0 (oscillations stopped)
....
     0 0 1 1 1 (disturbance stops)
     0 1 1 0 0 (oscillations start again)

So a purely logical control system, naively conceived, does not behave in the way we would expect a control system to behave. Note that Rick has a three-level system in which the third level is logical. Why doesn't it oscillate between true and false?

Best,

Bill P.

[Martin Taylor 2005.03.11.00.30]

[From Bill Powers (2005.03.10.1443 MST)]

Martin Taylor 2005.03.10.10.10 --

However, the effect of any disturbance, transient or continued, is amplified by positive feedback, the opposite of control.

True, but if the gain is less than unity, the integral levels off as time goes to infinity.

Leveling off will occur only if this is a leaky integrator and the slowing factor is such that the gain in each trip around the loop is less than 1. If the integrator is not leaky, the integral will increase without limit. Here is the transient case for a gain of 0.5 and a non-leaky integrator:

Disturbance input output delta
               (input + disturbance + output) (gain = 0.5)
1 0 0.5
0 0.5 0.25 -0.25
0 0.75 0.375 +0.125
0 1.125 0.5625 +0.1875
0 1.6875 0.84375 +0.28125
0 2.53125 1.2655625 +0.4218125

The system runs away (delta increases on each step) for any gain greater than zero. With a steady disturbance, of course, it runs away even faster.

I guess I shouldn't have made the comment about Gain > 1, given that we usually talke about control systems with the output being an integrator. Integrators don't deal in a flat gain number. They deal in gain in units of 1/t (gain per second). With a flat gain of less than 1, in your example you would have the recurring impulse decaying toward zero, as I said.

But you are right in that it was an irrelevant example. I still have a suspicion it was what Bjorn was talking about when he mentioned the number 1.0 as being critical in dealing with positive feedback.

Martin

[From Bjorn Simonsen (2005.03.11,11:00 EST)]

From Bill Powers (2005.03.10.1409 MST)

I have not thought that an error may be negative. How shall I understand
B:CP page 84: "... but since the other input to the neuron is inhibitory,
there can be no effect on the motor neuron unless the reference signal is
assigned a net positive value, greater than or equal to zero".

This is for a one-way system. A two-way control system can be constructed
of two one-way loops, one handling positive errors and the other handling
negative errors. Control systems for positioning limbs, maintaining
balance, aiming guns, and so on are two-way systems: that is, they can be
represented by a single system in which variables can be negative as well
as positive.

Pardon me for my ignorance. A new world opened for me when Norbert Wiener
a.o. opened my eyes for purposive behavior and I discovered a galaxy when I
in 97-98 joined CSG. Since then I, with my simple-mindedness, have thought
that organisms just controlled one-way perceptual systems.

Of course I have known that communication may be two-ways, but I have never
thought that living systems also controlled their perceptions in a two-way
system.

At present I work hard to understand the new two-way system (new for me) of
controlling perceptions.

Let me make the most of this opportunity and talk about Renshaw cells. I
have tried to understand how a Renshaw cell worked beyond (Wooldridge
1963)'s ""Renshaw cells" are apparently specialized to emit inhibitory
substance at the end of the outgoing impulse-conducting fiber", but I have
not been more wise till last year (if it is correct what I think today).
It is different transmitters in the synapses that bias how the target cell
eventually responds to an incoming message. This biasing of neuronal
signaling is known as neuromodulation and the biasing will i.a. make the
actual signal enhanced or blunted.

When we talk about negative perceptual signals or reference signals we don't
talk about the modality of the signal, we talk about the process the signal
leads to because of different transmitters.

Is this reasonable?

Let me express your words with mine.

Note that a "negative error" simply means an error signal that indicates
that r - p is negative (in other words, p is greater than r). That requires
that p be excitatory and r be inhibitory.

A negative error simply means a blunted effect on the on the recipient cell.

Let me again express your word with mine.

Both p and r are positive signals, and the system will work as a two-way
system for all values of p and r between 0 and the maximum possible values.
If p is greater than r, the lower error signal will be active and it will
drive the actuator for negative errors. The upper error signal will drive
the actuator for positive errors (p < r). Error is defined as r - p. Think
of the actuators as opposing muscles.

When we say that both p and r are positive signals, we think of the
frequency of action potentials. This must be a positive number (or zero). If

r, the error signal is negative and it will have a blunted effect on the

Output function. And when the Output function has a blunted effect, the
muscles will relax according to the r which ask for a lower p.

Is this reasonable?

I have problems with your next section (I am sorry),

If this system is part of a negative feedback loop, positive feedback will
happen if all the plus and minus signs in the diagram are interchanged.
Then runaway can happen in either direction: toward more negative or more
positive errors.

With my words.
If we have a negative feedback loop _in a simulation_ , the p will approach
r. If we in the same feedback loop change the effect of p from minus to plus
and change the effect of r from plus to minus, then the effect on the output
function will be positive (|p|>|r|). And this positive error will have an
enhanced effect on the output function. The muscles will be still more
tightened. Now the p will be even greater. and we have a positive feedback.
This was in a simulation.
How can this happen in a living organism? The transmitters (proteins) don't
change, do they? How can the plus and minus signs in a living organism
change?

I can imagine illness and or injury in the brain will result in positive
feedback in a functioning control system. But there are limits for how much
muscles and glands can tighten or secrete. Near these limits I guess
reorganization will eliminate the positive feedback loops. (?)

bjorn