Ashby's law of requisite variety

[From Bruce Abbott (2004.05.09.1720 EST)]

Bill Powers (2004.05.08)

Unfortunately, statistical measures like variety don't specify that the
action must vary as a specific function of the disturbance -- only that a
measure similar to variability must have the same size. So the question of
how the _right kind_ of variability can be achieved is left unanswered by
the law of requisite variety (thanks for bringing that term back to me --
another neuron shot to hell). There are many ways to create variety of
output that will match the variety of the disturbance, yet not stabilize
the controlled variable at all.

True, but the law does call attention to an important requirement that must
be satisfied by whatever mechanism is used, if perfect control is to be
achieved. It also makes clear the relationship between control and
information. Control destroys information about variation in the
disturbance that otherwise would be present in the controlled variable if
control were absent. The better the control, the less information "passes
through." If your home heating/air-conditioning system controlled your
home's temperature perfectly, you would not be able to tell from the
interior temperature alone how the temperature outside was changing.

Bruce A.

[From Bill Powers (2004.05.09.1729 MST)]

Bruce Abbott (2004.05.09.1720 EST)--

True, but the law does call attention to an important requirement that must
be satisfied by whatever mechanism is used, if perfect control is to be
achieved.

No, it doesn't. You can meet the requirement of the law of requisite
variety and still have no control at all. The requirement for control is
that the output of the system have effects on the controlled variable that
are as nearly equal and opposite to the effects of disturbances as
possible. If that criterion is satisfied, the law of requisite variety will
prove to be satisfied. But the opposite is not true: if the law of
requisite variety is satisfied, it does not follow that the action is equal
and opposite to the disturbance in terms of effects on the controlled
variable. If the output varied equally and in the same direction as the
disturbance, the law of requisite variety would be perfectly satisfied, but
obviously there would be no control.

The law of requisite variety, like many other abstract generalizations
about control, is one of those ideas that follows from, but does not
explain control, or help us understand it better.

I know that there are many people who have a different idea about this. I
intend to be just as stubborn as they are.

But if you can drop the subject, I can, too. I have had enough
disagreements that I think I can make it through the rest of the year
without needing any more.

Best,

Bill P.

···

It also makes clear the relationship between control and
information. Control destroys information about variation in the
disturbance that otherwise would be present in the controlled variable if
control were absent. The better the control, the less information "passes
through." If your home heating/air-conditioning system controlled your
home's temperature perfectly, you would not be able to tell from the
interior temperature alone how the temperature outside was changing.

Bruce A.

[From Bill Powers (2004.04.05.11.0250 MST)]

Bruce Abbott (2004.05.09.1720 EST) --

Boning up on the Law of Requisite Variety, I find that it's not even as
general as I had thought. Ashby defined variety this way, in _An
Introduction to Cybernetics_,

"The word _variety_, in relation to a set of distinguishable elements, will
be used to mean either (i) the number of distinguishable elements, or (ii)
the logarithm to the base 2 of the number, the context indicating the sense
used."

Later, he expresses the Law of Requisite Variety this way:

"Only variety can destroy variety." [p. 207]

I'm afraid that advanced age has spoiled my original admiration of Ashby's
offerings. His writings now appear to me as those of a man obsessed with
orderliness, but lacking any real insights. What the LRV really says is
only that it takes a system with enough moving parts to control an
environment with multiple moving parts. You can't keep multiple
disturbances from altering a system unless you oppose them all. This sounds
an awful lot like an amateur standing around watching people do marvellous
things and wanting to contribute something even if it's only to emit
impressive-sounding generalizations. I'm sure you can recall doing the same
thing as an adolescent -- I certainly can. You watch the work crew digging
a new foundation with a big bulldozer, and you remark to your companions,
"Sometimes it take a lot of horsepower to get the job done." This pompous
generalization (the young tend to be pompous) makes it seem as if you know
a lot about what you're watching, but in reality it's just an attempt to
impress.

If you really think about most of Ashby's pronouncements of Laws, the
initial impression of rightness falls apart. He's just fishing around for
statements that sound general, with no real idea of whether they are
actually general. Is it really true that only variety can destroy variety?
That only one distinguishable number of elements can destroy another
distinguishable number of elements? When you plug his definition of variety
into the general law like that, you get gibberish. Everything that might
actually result in one form of "variety" "destroying" another form is left
out, one suspects because of a shortage of knowledge about how things work.

I had retained an impression that somehow the LRV was related to entropy
and information, and Ashby does make an attempt to create a connection. But
his mathematics is extremely sketchy and the reasoning is mostly verbal.
His generalizations are seasoned only very lightly with facts, and those
are mostly in the form of generalizations themselves. I think Ashby was
trying to create a discipline in the manner of the ancient Greeks: out of
pure reason. That didn't work any better for Ashby than it did for the
Greeks. This sort of thing, I think, is why Cybernetics just slowly
withered away.

I sort of wish I hadn't opened up those old books again. It was something
like seeing a movie that one remembers from childhood as wonderful, and
having to face the reality of how much the world has grown up. The sudden
appearance on the screen of Step-'n'Fechit rolling his eyes and saying
"Yowsah, Boss, yuck-yuck-yuck", which was so funny then, is jarring and
embarrassing now. In the 50 years since I first read Ashby's "Design", a
good deal of the impressiveness of the book has somehow evaporated.

Best,

Bill P.

[From Bill Powers (2004.05.11.0349 MST)]

Since I'm still awake, a little more on Ashby:

Bruce Abbott (2004.05.09.1720 EST)--

[The Law of Requisite Variety] also makes clear the relationship between
control and information. Control destroys information about variation in the
disturbance that otherwise would be present in the controlled variable if
control were absent. The better the control, the less information "passes
through." If your home heating/air-conditioning system controlled your
home's temperature perfectly, you would not be able to tell from the
interior temperature alone how the temperature outside was changing.

Now think about this. Does the lack of information explain how control
works, or does the way control works explain the (apparent) lack of
information? While it may seem that there is little information in the
controlled variable about the disturbance, this is because the controlled
variable is kept from changing very much by variations in the output of the
control system. If you transferred your gaze to the output, you would find
a lot of information there about at least the _net_ disturbance
(disturbances are usually multiple, so there's no way of telling from their
joint effect how much is due to each disturbance).

In fact, in the idealized control system no information is really lost;
that only happens if there is a noise level in the controlling system that
drowns out small fluctuations in the perceived input quantity. Without
noise, the effect of the disturbance is simply scaled down in amplitude by
a factor of the loop gain, with all the details of variations still fully
recoverable given knowledge of the controller's output. So the idea that
mere "variety" (as a measure of information) can destroy variety is false:
it is noise and only noise that destroys information. In the absence of
noise, scaling the size of fluctuations up and down has no effect on the
information they represent.

An interesting application of this idea:

We find that in tracking experiments, the better the subject's control
gets, the lower the correlation is between fluctuations in the controlled
variable at the input and changes in the controller's output. But if we
calculate the same input-output correlations on the variables in a model of
the control system, the correlation is much higher, especially if we take
the lags into account -- that makes the correlation essentially perfect.

How come? The difference comes from the presence of noise in the operation
of the real control system, noise that is missing from the model unless we
add some in. The model controls even better than the real person does, yet
the model does not "destroy variety" at all. It just miniaturizes the
variations, and by using a magnifying glass we can see them as well as
before. All the information that is there with no control is still there.

I can't think of a better proof that control has nothing to do with
changing entropy, information, or variety. Too bad I didn't think of this
about 10 years ago or so when there was a big CSGnet hassle on this subject.

Best,

Bill P.

[From Boss Man (2004.05.11)]

Bill Powers (2004.04.05.11.0250 MST)

I'm afraid that advanced age has spoiled my original admiration of Ashby's
offerings. His writings now appear to me as those of a man obsessed with
orderliness, but lacking any real insights. What the LRV really says is
only that it takes a system with enough moving parts to control an
environment with multiple moving parts. You can't keep multiple
disturbances from altering a system unless you oppose them all. This sounds
an awful lot like an amateur standing around watching people do marvellous
things and wanting to contribute something even if it's only to emit
impressive-sounding generalizations. I'm sure you can recall doing the same
thing as an adolescent -- I certainly can. You watch the work crew digging
a new foundation with a big bulldozer, and you remark to your companions,
"Sometimes it take a lot of horsepower to get the job done." This pompous
generalization (the young tend to be pompous) makes it seem as if you know
a lot about what you're watching, but in reality it's just an attempt to
impress.

You can imagine just how hard I had to struggle not to post, "Thank goodness this sort of thing
never happens on CSGnet." But I called my buddy "Jim Bearsley" and he talked me through the 12
steps needed to overcome Irony.

By the way Mary, I'd like to reveal my identity, but as part of the prgram I am bound by the
_omerta_ If I told you, I'd have to kill everyone on CSGnet. If I come to meeting, I will wear a black
hood over my head. Please do not ask me to remove it...

Somewhere in the rugged mountains of Pakistan, hopefully in the hands of a Higher Power...

From [Marc Abrams (2004.05.11.1005)]

[From Boss Man (2004.05.11)]

By the way Mary, I'd like to reveal my identity, but as part
of the prgram I am bound by the _omerta_ If I told you, I'd
have to kill everyone on CSGnet. If I come to meeting, I will
wear a black hood over my head. Please do not ask me to remove it...

Regardless of Whomever and whatever you are, you are one sick
individual.

Marc

Considering how often throughout history even intelligent people have
been proved to be wrong, it is amazing that there are still people who
are convinced that the only reason anyone could possibly say something
different from what they believe is stupidity or dishonesty.

Being smart is what keeps some people from being intelligent.

Thomas Sowell

[From Boss Man (2004.05.11)]

From [Marc Abrams (2004.05.11.1005)]

Regardless of Whomever and whatever you are, you are one sick
individual.

Your greatest strength, Marc, is your sense of humor. I hope you never lose it.

Ooops. I'm afraid I fell off the wagon, Jim.

From[Bill Williams 11 May 2004 11:30 AM CST]

[From Bill Powers (2004.05.11.0349 MST)]

I find your comments on Ashby quite interesting. And, I resent the time I
wasted wondering about whether what Ashby was saying made any sense.

But isn't what Ludwig von Bertalanffy did with his _General Systems Theory_
far worse?

Bill Williams

[From Bill Powers (2004.05.11.1431 MST)]

Bill Williams 11 May 2004 11:30 AM CST--\

I find your comments on Ashby quite interesting. And, I resent the time I
wasted wondering about whether what Ashby was saying made any sense.

But isn't what Ludwig von Bertalanffy did with his _General Systems Theory_
far worse?

It may have been, but I put in far less time reading his stuff than reading
Ashby's. My dilemma with Ashby was always that grasping what he was saying
required a lot of mental effort and I never felt justified in complaining
until I felt sure I had understood what he was saying. He was Somebody and
I was Nobody, and Nobodys are supposed to be quiet and listen to the Somebodys.

It actually took a bit of courage to say what I said in that last post,
knowing that many people will be outraged and claim that I just didn't
understand. Maybe I didn't, but maybe it's also possible that there was
less to understand than I had assumed there was.

Best,

Bill P.

[From Bruce Abbott (2004.05.11.1940 EST)]

Bill Powers (2004.05.11.0349
MST)

Since I’m still awake, a little more on Ashby:

Bruce Abbott (2004.05.09.1720 EST)–

[The Law of Requisite Variety] also
makes clear the relationship between

control and information. Control destroys information about
variation in the

disturbance that otherwise would be present in the controlled variable
if

control were absent. The better the control, the less information
"passes

through." If your home heating/air-conditioning system
controlled your

home’s temperature perfectly, you would not be able to tell from
the

interior temperature alone how the temperature outside was
changing.

Now think about this. Does the lack of information explain how
control

works, or does the way control works explain the (apparent) lack of

information? While it may seem that there is little information in
the

controlled variable about the disturbance, this is because the
controlled

variable is kept from changing very much by variations in the output of
the

control system. If you transferred your gaze to the output, you would
find

a lot of information there about at least the net disturbance

(disturbances are usually multiple, so there’s no way of telling from
their

joint effect how much is due to each disturbance).

Sorry Bill, but I must respectfully disagree with your position on
Ashby. You have misunderstood what Ashby was trying to do, and from
false premises come false conclusions.
In Introduction to Cybernetics, Ashby presents an extended
tutorial on cybernetics, which Ashby saw as a field concerned with
developing a general theory of machines. It is not about the
mechanics of particular machines, but about machines in the abstract, as
organizations of interacting variables. In this sense, a hydraulic
amplifier and an electronic amplifier are the same machine if both reduce
to the same diagram. An elementary control unit could be
constructed from neural circuits, electronic circuits, or circulating
hormones and still be the same elementary control unit, the same
machine, even though implemented in different hardware, with
different variables, if all reduce to the same block diagram.
Set a machine into motion, and many of the variables that compose it
begin to change over time, i.e., they vary. If variable A
influences variable B, then variation in A gets transmitted to B.
In information theory, this variation is called information.
Variation in outside temperature, if left unopposed, produces variation
in inside temperature.

You presented the following:

The alternative to negative
feedback that Ashby proposed looks like this (drawn from page 210 in the
1963 (Wiley) paperback reprint of “An introduction to
cybernetics”):

128b345.jpg

E is the “essential variable,” the state of which is to be
regulated. D is a disturbing variable which can act on E through the
transmission channel T. And R is a regulator designed to prevent
disturbances of E.

This is not Ashby’s proposed alternative to a PCT-style controller.
It is a diagram intended to show how an ideal control system would
“destroy” the information (variation) present in the
disturbance. Because D simultaneously transmits its variation to
both R and T, and because R and T are connected inversely, none of the
variation in D reappears as variation in E. Whether this particular
form of regulation is practical would depend on the circumstances under
which the regulator had to operate.

I don’t find Ashby championing this design as a practical one for all
situations, although he does suggest that versions of it may be found in
some biological systems. Later in the book, Ashby devotes an entire
chapter (12) to “The Error-Controlled Regulator,” i.e., the
PCT-type control system. There is another block diagram showing
that, for such as system, the variation in D cannot be eliminated in E,
for the simple reason that the variation in D must pass through E to get
to R.

What you say below begins by actually agreeing with Asby’s analysis,
although what you are calling the “idealized control system” is
of course Ashby’s error-controlled regulator:

In fact, in the idealized control
system no information is really lost;

that only happens if there is a noise level in the controlling system
that

drowns out small fluctuations in the perceived input quantity.
Without

noise, the effect of the disturbance is simply scaled down in amplitude
by

a factor of the loop gain, with all the details of variations still
fully

recoverable given knowledge of the controller’s output. So the idea
that

mere “variety” (as a measure of information) can destroy
variety is false:

it is noise and only noise that destroys information. In the absence
of

noise, scaling the size of fluctuations up and down has no effect on
the

information they represent.

See your block diagram above: Variety can destroy variety,
but the standard PCT control system can, as you say, only reduce it to
the point where it becomes buried in statistical noise.

An interesting application of this
idea:

We find that in tracking experiments, the better the subject’s
control

gets, the lower the correlation is between fluctuations in the
controlled

variable at the input and changes in the controller’s output. But if
we

calculate the same input-output correlations on the variables in a model
of

the control system, the correlation is much higher, especially if we
take

the lags into account – that makes the correlation essentially
perfect.

How come? The difference comes from the presence of noise in the
operation

of the real control system, noise that is missing from the model unless
we

add some in. The model controls even better than the real person does,
yet

the model does not “destroy variety” at all. It just
miniaturizes the

variations, and by using a magnifying glass we can see them as well
as

before. All the information that is there with no control is still
there.

I can’t think of a better proof that control has nothing to do with

changing entropy, information, or variety. Too bad I didn’t think of
this

about 10 years ago or so when there was a big CSGnet hassle on this
subject.

Your conclusion does not follow. Variety is what is being controlled by a
control system. Less variation in the CV for a given variation in
disturbance is less information (in the information-theoretic sense)
being transmitted from disturbance to CV. This isn’t an alternative
to control theory, it is simply a different way of describing what
happens to the variables of the system as the system runs. It’s
like thermodynamics: it can highlight some general principles that apply
to a given system, but you have to design (or specify) the system to be
analyzed.

Bruce A.

[From Bill Powers (2004.05.11.1939 MST)]

Bruce Abbott (2004.05.11.1940 EST)–

Sorry Bill, but I must respectfully
disagree with your position on Ashby. You have misunderstood what
Ashby was trying to do, and from false premises come false
conclusions.
In Introduction to Cybernetics, Ashby presents an extended
tutorial on cybernetics, which Ashby saw as a field concerned with
developing a general theory of machines. It is not about the
mechanics of particular machines, but about machines in the abstract, as
organizations of interacting variables. In this sense, a hydraulic
amplifier and an electronic amplifier are the same machine if both reduce
to the same diagram.

But they aren’t the same machine – they only appear so if you ignore how
they actually work and look only at the describing equations. A negative
feedback control system with a leaky integrator for an output function,
controlling a proportional variable, is described by a second-order
differential equation. So is the behavior of a mass on a spring with
damping. So is the behavior of a pendulum swinging in a viscous fluid.
But these three types of device are certainly not “the same
machine.” The differences make a difference.

I simply don’t believe that Ashby knew enough about machines to come up
with a general theory of machines, and aside from that, I don’t think
that a general theory of machines would even be useful. But this,
clearly, is a matter of opinion, and when our opinions are so different
there’s no point in going to the mat over them. I think I can see that
your opinion of Ashby’s work is not likely to change as a result of
anything I can say.

We can, however, talk about specific points.

This is not Ashby’s proposed
alternative to a PCT-style controller. It is a diagram intended to
show how an ideal control system would “destroy” the
information (variation) present in the disturbance. Because D
simultaneously transmits its variation to both R and T, and because R and
T are connected inversely, none of the variation in D reappears as
variation in E. Whether this particular form of regulation is
practical would depend on the circumstances under which the regulator had
to operate.

This is true ONLY if you think of the variables involved as being
measured in small integers. However, if you use the real number scale,
it’s clear that the cancellation of D by R can never be exact to the last
decimal place, and in any real system if probably can’t be exact to more
than two or three decimal places – say, 0.1%.

Let’s say that R varies in exact proportion to D as the designer intends,
say R = k*D,

E = D - k*D

E = D*(1-K).

The designer intends for K to be 1.000000000…, but of course that is
unachievable in a real system. Even if proportionality is achieved, the
exact value of the coefficient depends on how closely the system can be
adjusted. There will always some small delta by which K fails to be
exactly 1. The actual value of K will be the ideal value K’ plus
(or minus) delta. K = K’ + delta.

This gives us

E = D - D*(K + delta) where K is exactly 1, or

E = D*delta.

So in the absence of noise in the transmission channel T, variations in E
will be proportional to the disturbance, which is the point I am trying
to make here. True, delta may be very small, but these equations, like
Ashby’s, contain no noise terms: they are exact. If there is information
in D, it will be perfectly preserved in E, because simply scaling a
signal up or down in magnitude has no effect on the amount of information
it carries. It doesn’t matter that the effects of D on E are very small
after R has mostly cancelled out the effects of D. The critical term is
“mostly.” With no noise in the system, the variations due to D
are reproduced as variations in E which carry exactly the same
information. E will correlate perfectly with D. The only case in which it
will not correlate perfectly, in the absence of noise, is that in which K
= 1 to an infinite number of decimal places. But the probability of that
is zero.

This also means that the number of distinguishable states in D, the
variety of D, is perfectly preserved as distinguishable states in E,
despite the fact that R has reduced the magnitude of the fluctuations.
Note that I do not admit that R can bring the fluctations to zero: that
is physically impossible in the real world.

You may now ask “So what?” The Ashby controller can only reduce
the information to about the same level as the magnitude of noise in the
transmission channel – that’s still pretty good, isn’t it? Maybe so, but
the error-driven controller can do better.

Later in the book, Ashby devotes an
entire chapter (12) to “The Error-Controlled Regulator,” i.e.,
the PCT-type control system. There is another block diagram showing
that, for such as system, the variation in D cannot be eliminated in E,
for the simple reason that the variation in D must pass through E to get
to R.

Suppose the loop gain of the control system is one million (I have seen
control systems with gains a hundred times higher than that).Yes, there
is still an effect on the essential variable from the disturbance, but it
is reduced by a factor of a million below what the effect would have been
without the feedback. Ah, you say, but the noise level will be greater
than the error, so noise will limit the precision of control. There will
indeed be fluctuations in the transmission channel, I agree, but in the
negative feedback control system, it is not the transmission channel that
determines the precision of control. It is the perceptual channel, which
does not even exist in Ashby’s diagram. A negative feedback control
system can cancel out individual noise fluctuations in the output
transmission channel over its bandwidth of operation, provided that it
has low-noise sensors for detecting the state of E.

What this comes down to is that in the presence of transmission noise,
the optimum negative feedback control system will always be able to
defend against disturbances better than the optimum Ashby controller
(more properly called a compensator) could do. It can do so, furthermore,
despite significant changes in the transmission channel, the output
apparatus, or the effect of D on T. It can react to disturbancez faster,
and over a wider band of frequencies. Ashby showed no indication of
understanding those critical properties of negative feedback
control.

Of course very few others (but not zero) have understood them,
either.

See your block diagram above:
Variety can destroy variety, but the standard PCT control
system can, as you say, only reduce it to the point where it becomes
buried in statistical noise.

I hope I have shown you that variety cannot destroy variety in a
noise-free system except in the single case (of zero probability) that
the regulator is adjusted to cancel the effect of the disturbance
EXACTLY. And if we talk about real systems with noise, the negative
feedback controller is the one that wins.

Your conclusion does not follow.
Variety is what is being controlled by a control
system.

No. MAGNITUDE is what is being controlled by a control system. Variety is
defined as the number of distinguishable states, which is not directly
affected by making signal magnitudes larger or smaller – as long as
there is no noise or the signals are confortably above the noise level.
Variety IS affected by making the magnitudes smaller relative to a
background level of noise
– or, of course, by raising the noise level.
That is because the noise then becomes larger in proportion to the
magnitudes, and you can no longer make fine discriminations between
magnitudes because of the noise. You are confusing making the
fluctuations smaller with making the variety smaller. Those are not the
same thing. Think of the information on a printed page, measured in bits.
If you photograph that page and reproduce it half as large, the
information on the page is not reduced at all as long as you can still
read the characters as well as before (with or without a magnifier). Even
printing the Bible on the head of a pin does not reduce its information
content if you can still read it clearly with suitable optical aid. The
limit to minification is set by noise, not magnitude.

Less variation in the CV for
a given variation in disturbance is less information (in the
information-theoretic sense) being transmitted from disturbance to
CV.

No. Information is not the same thing as magnitude. Information has to do
with uncertainty-reduction, which is related to the number of
discriminable states of a variable, which is what relates information
both to variety and to entropy. Reducing the magnitude of a variation
does not change the amount of information or variety it carries unless
doing so increases the uncertainty in measuring the signal (I hope Martin
Taylor is going to validate this).

And remember, as soon as we introduce noise in the output transmission
channel, negative feedback control becomes the superior method of control
because it depends on direct sensing of the state of the controlled
variable, not blindly trying to compensate for the effects of
disturbances without being able to see the result…

Best,

Bill P.

From[Bill Williams 12 May 2004 12:25 PM CST]

[From Bill Powers(2004.05.11.1939 MST]

Bruce Abbott(2004.05.11.1940 EST]

You both may have a feeling that the other’s misconceptions preclude productive discussion. However, from the sidelines I see the discussion as exposing some fundamentally significant issues. Powers makes the point that a single equation can equally well describe very different causal systems. Patently obvious perhaps, if you think about the issues involved-- or think about them enough. But perhaps I don’t. So, it is nice to have this review by way of a disputation.

Bill Williams

from Mary Powers 2004.05.12

···

At 10:23 AM 5/11/2004, you wrote:

From[Bill Williams 11 May 2004 11:30 AM CST]

> [From Bill Powers (2004.05.11.0349 MST)]
>
I find your comments on Ashby quite interesting. And, I resent the time I
wasted wondering about whether what Ashby was saying made any sense.

But isn't what Ludwig von Bertalanffy did with his _General Systems Theory_
far worse?

Bill Williams

I read an article some years ago by von B in which he said that a control
system was nothing but a stimulus-response system with a feedback loop
added on. Brilliant.

I wish I still had the reference but it was lost years ago.

[From Bruce Abbott (2004.05.12.1910 EST)]

Bill Powers (2004.05.11.1939
MST)

Bruce Abbott (2004.05.11.1940 EST)–

Sorry Bill, but I must respectfully
disagree with your position on Ashby. You have misunderstood what
Ashby was trying to do, and from false premises come false
conclusions.
In Introduction to Cybernetics, Ashby presents an extended
tutorial on cybernetics, which Ashby saw as a field concerned with
developing a general theory of machines. It is not about the
mechanics of particular machines, but about machines in the abstract, as
organizations of interacting variables. In this sense, a hydraulic
amplifier and an electronic amplifier are the same machine if both reduce
to the same diagram.

But they aren’t the same machine – they only appear so if you ignore how
they actually work and look only at the describing equations. A negative
feedback control system with a leaky integrator for an output function,
controlling a proportional variable, is described by a second-order
differential equation. So is the behavior of a mass on a spring with
damping. So is the behavior of a pendulum swinging in a viscous fluid.
But these three types of device are certainly not “the same
machine.” The differences make a difference.

Those are not the same machines because they do not have the same
organization. What cybernetics (in Ashby’s time, at least) was
concerned with were general principles that depend, not on the specific
materials of which a machine is made, but rather on its
organization. At this level of abstraction, a negative feedback
control system with leaky integrator output is a negative feedback
control system with leaky integrator output, whether it is made of
neurons, sensory receptors, and muscles, electronic components, or steps
in a computer simulation.

I simply don’t believe that Ashby
knew enough about machines to come up with a general theory of machines,
and aside from that, I don’t think that a general theory of machines
would even be useful. But this, clearly, is a matter of opinion, and when
our opinions are so different there’s no point in going to the mat over
them. I think I can see that your opinion of Ashby’s work is not likely
to change as a result of anything I can say.

I think you are making the mistake of assuming that, in Introduction
of Cybernetics
, Ashby was inventing all these ideas, when for the
most part he was simply presenting some of the basic principles of
cybernetics as developed by others. All the stuff about variety,
entropy, and the like, for example, was taken from Information Theory as
pioneered by Shannon and Weaver.

You suggest that you are wasting your time trying to convince me of your
position, and you may be right. I am reminded of how many times you
have patiently attempted to correct the misunderstandings so many people
bring into their discussions of control theory. You probably
think I’m just being stubborn.

We can, however, talk about
specific points.

This is not Ashby’s proposed
alternative to a PCT-style controller. It is a diagram intended to
show how an ideal control system would “destroy” the
information (variation) present in the disturbance. Because D
simultaneously transmits its variation to both R and T, and because R and
T are connected inversely, none of the variation in D reappears as
variation in E. Whether this particular form of regulation is
practical would depend on the circumstances under which the regulator had
to operate.

This is true ONLY if you think of the variables involved as being
measured in small integers. However, if you use the real number scale,
it’s clear that the cancellation of D by R can never be exact to the last
decimal place, and in any real system if probably can’t be exact to more
than two or three decimal places – say, 0.1%.

Yes, and real springs do not behave quite like the ideal springs that
physicists are so fond of. You seem to be making an argument that
there is nothing to be gained by considering ideal systems.

Ashby’s ideal regulator does prevent all variation in the disturbance
from reaching the essential variables, and without requiring a system
with infinite gain. I agree that it won’t work so well in practice,
except perhaps in certain restricted environments. The device is
theoretically capable of perfect regulation; the error-controlled
regulator is not theoretically capable of perfect regulation, although
under the right conditions (permitting very high gains), it can come
close.

So in the absence of noise in the transmission channel T, variations in E
will be proportional to the disturbance, which is the point I am trying
to make here. True, delta may be very small, but these equations, like
Ashby’s, contain no noise terms: they are exact. If there is information
in D, it will be perfectly preserved in E, because simply scaling a
signal up or down in magnitude has no effect on the amount of information
it carries. It doesn’t matter that the effects of D on E are very small
after R has mostly cancelled out the effects of D. The critical term is
“mostly.” With no noise in the system, the variations due to D
are reproduced as variations in E which carry exactly the same
information. E will correlate perfectly with D. The only case in which it
will not correlate perfectly, in the absence of noise, is that in which K
= 1 to an infinite number of decimal places. But the probability of that
is zero.

This also means that the number of distinguishable states in D, the
variety of D, is perfectly preserved as distinguishable states in E,
despite the fact that R has reduced the magnitude of the fluctuations.
Note that I do not admit that R can bring the fluctations to zero: that
is physically impossible in the real world.

Now let’s return for a moment to a system with only a certain number of
“distinguishable states,” say the 256 distinguishable states
available in an 8-bit binary number. If the regulator cuts the
amplitude of the effect of the disturbance on the controlled variable in
half, there are only half as many distinguishable states available over
which the variable can vary, and information has been lost. I
rather suspect that the same would hold in the continuous case, although
I don’t have the mathematical skills to prove it. Any real system
has some degree of noise in it inherently limits the values that can be
reliably distinguished, so that reducing the signal to a small value and
then scaling it up again produces a degraded signal, compared to the
original.

You may now ask “So
what?” The Ashby controller can only reduce the information to about
the same level as the magnitude of noise in the transmission channel –
that’s still pretty good, isn’t it? Maybe so, but the error-driven
controller can do better.

Later in the book, Ashby devotes an
entire chapter (12) to “The Error-Controlled Regulator,” i.e.,
the PCT-type control system. There is another block diagram showing
that, for such as system, the variation in D cannot be eliminated in E,
for the simple reason that the variation in D must pass through E to get
to R.

Suppose the loop gain of the control system is one million (I have seen
control systems with gains a hundred times higher than that).Yes, there
is still an effect on the essential variable from the disturbance, but it
is reduced by a factor of a million below what the effect would have been
without the feedback. Ah, you say, but the noise level will be greater
than the error, so noise will limit the precision of control. There will
indeed be fluctuations in the transmission channel, I agree, but in the
negative feedback control system, it is not the transmission channel that
determines the precision of control. It is the perceptual channel, which
does not even exist in Ashby’s diagram. A negative feedback control
system can cancel out individual noise fluctuations in the output
transmission channel over its bandwidth of operation, provided that it
has low-noise sensors for detecting the state of E.

Well, yes – in an error-controlled regulator, you can reduce the error
to almost any desired value by making the gain high enough (so long as
the system in question remains stable). Ashby’s point is simply
that you can’t make it zero. To that I would and that, in many
systems, you can’t make the gain very high, either, without introducing
instability.
By the way, I’d really like to know how those Bose noise-cancelling
headphones work. One possibility is that there is a microphone that
picks up noise in the surroundings outside of the
headphones. Electronic circuitry would invert the signal, re-scale
it to match the intensity of the same noise passing through the sound
insulation of the headphones, and add it to the signal (e.g.,
music). If this is the way they work, then the headphones are an
example of Ashby’s compensator in action. Alternatively, the
microphone could pick up the sound inside the headphones (the CV)
and use the signal (music) as the reference – an error-controlled
regulator.

What this comes down to is that in
the presence of transmission noise, the optimum negative feedback control
system will always be able to defend against disturbances better than the
optimum Ashby controller (more properly called a compensator) could do.
It can do so, furthermore, despite significant changes in the
transmission channel, the output apparatus, or the effect of D on T. It
can react to disturbancez faster, and over a wider band of
frequencies. Ashby showed no indication of understanding those
critical properties of negative feedback control.

That may be so – I’d have to reread what he has said on the topic before
making a judgment – but such an analysis probably would have fallen
outside the scope of the book, which dealt with a broad range of ideas in
cybernetics and not just regulation. Remember, it was intended as
an introduction to the field, and to his great credit, I think,
Ashby was able to present these ideas in a way that even I could
understand. It’s still an extremely readable little book today, and
I would recommend it to anyone who wants to know what the heck we’ve been
talking about. (It’s available free on the Web, so there
goes that excuse for not taking a look.) It would be great if some
of those who try to make up their minds based on what you and I have to
say here would instead go see for themselves and then come back to CSGnet
with comments.

Of course very few others (but not
zero) have understood them, either.

See your block diagram above:
Variety can destroy variety, but the standard PCT control
system can, as you say, only reduce it to the point where it becomes
buried in statistical noise.

I hope I have shown you that variety cannot destroy variety in a
noise-free system except in the single case (of zero probability) that
the regulator is adjusted to cancel the effect of the disturbance
EXACTLY. And if we talk about real systems with noise, the negative
feedback controller is the one that wins.

Your conclusion does not follow.
Variety is what is being controlled by a control
system.

No. MAGNITUDE is what is being controlled by a control system. Variety is
defined as the number of distinguishable states, which is not directly
affected by making signal magnitudes larger or smaller – as long as
there is no noise or the signals are confortably above the noise level.
Variety IS affected by making the magnitudes smaller relative to a
background level of noise
– or, of course, by raising the noise level.
That is because the noise then becomes larger in proportion to the
magnitudes, and you can no longer make fine discriminations between
magnitudes because of the noise. You are confusing making the
fluctuations smaller with making the variety smaller. Those are not the
same thing. Think of the information on a printed page, measured in bits.
If you photograph that page and reproduce it half as large, the
information on the page is not reduced at all as long as you can still
read the characters as well as before (with or without a magnifier). Even
printing the Bible on the head of a pin does not reduce its information
content if you can still read it clearly with suitable optical aid. The
limit to minification is set by noise, not magnitude.

When you reduce the magnitude, you reduce the variety, unless you have
infinite fidelity.

Less variation in the CV for a given variation in disturbance is less
information (in the information-theoretic sense) being transmitted from
disturbance to CV.

No. Information is not the same thing as magnitude. Information has to do
with uncertainty-reduction, which is related to the number of
discriminable states of a variable, which is what relates information
both to variety and to entropy. Reducing the magnitude of a variation
does not change the amount of information or variety it carries unless
doing so increases the uncertainty in measuring the signal (I hope Martin
Taylor is going to validate this).

And remember, as soon as we introduce noise in the output transmission
channel, negative feedback control becomes the superior method of control
because it depends on direct sensing of the state of the controlled
variable, not blindly trying to compensate for the effects of
disturbances without being able to see the result…

Let’s not loose sight of the fact that I’m not arguing for the
superiority of compensator-type regulators over error-controlled
regulators in real-world applications. In fact, I don’t believe
that Ashby was either. He was simply pointing out the differences
in the performance of each design, considered in the abstract.
However, there do seem to be certain limited cases in which compensation
seems to have been implemented in biological systems. One I have in
mind at the moment is the system that regulates body temperature.
Although the main job of the system is to keep core body temperature
within certain bounds, we have skin sensors that provide the brain with
advance warning that the body is gaining or loosing heat. We begin
to take action against these disturbances long before core body
temperature is much affected.

One parting comment on Ashby. On Pp. 195-196, near the beginning of
the Chapter 10 (Regulation in Biological Systems), Ashby (1957) had this
to say:
The subject of regulation in biology is so vast that no single
chapter can do it justice. Cannon’s Wisdom of the Body
treated it adequately so far as internal, vegetative activities are
concerned, but there has yet to be written the book, much larger in size,
that shall show how all the organism’s exteriorly-directed activities –
its “higher” activities – are all similarly regulatory, i.e.,
homeostatic.For some reason, B:CP comes to mind.

Bruce A.

[From Bill Powers (2004.05.13.1249 MST)]

Bruce Abbott (2004.05.12.1910 EST)–

What cybernetics (in Ashby’s time,
at least) was concerned with were general principles that depend, not on
the specific materials of which a machine is made, but rather on its
organization. At this level of abstraction, a negative feedback
control system with leaky integrator output is a negative feedback
control system with leaky integrator output, whether it is made of
neurons, sensory receptors, and muscles, electronic components, or steps
in a computer simulation.

I think there has been an impression, not just in cybernetics but in
physics, too, that if two processes are described by the same equation,
there must be some mystical connection between them. I think this is a
misconception of the role of mathematics in science. I do not think that
mathematics reveals anything about the real world; it is a tool with
which we can represent real things in systematic ways, but it always
idealizes nature. It must, to make the equations solvable. And because of
that, it always lies when it seems to say that two different kinds of
mechanisms are really described by the same equations. The truth is that
none of the mechanisms is really described by any equations: a mechanism
is only approximated in an idealized form. This is basically why I treat
descriptions of nature with more skepticism as the level of abstraction
increases, and the messy details are smoothed over.

I’m sure that view would suffice to make me unpopular in many
places.

I think you are making the mistake
of assuming that, in Introduction of Cybernetics, Ashby was
inventing all these ideas, when for the most part he was simply
presenting some of the basic principles of cybernetics as developed by
others. All the stuff about variety, entropy, and the like, for example,
was taken from Information Theory as pioneered by Shannon and
Weaver.

I wasn’t swept up by Shannon and Weaver, either. Ashby presented the
basic principles quite uncritically, in my view.

You suggest that you are wasting
your time trying to convince me of your position, and you may be
right. I am reminded of how many times you have patiently attempted
to correct the misunderstandings so many people bring into their
discussions of control theory. You probably think I’m just
being stubborn.

Well, you are. So am I. The last thing I would want would be for you to
change a view just because I said you should. That would shake my
confidence in my own ideas, too.

Yes, and real springs do not behave
quite like the ideal springs that physicists are so fond of. You
seem to be making an argument that there is nothing to be gained by
considering ideal systems.

Not at all, but one has to approach ideal systems with an eye to reality.
Back when I first learned about negative feedback, in Navy electronics
school early in the winter of 1944-45, our instructor showed us the
equations for an amplifier with feedback, with particular regard to the
effect of the gain in the output vacuum tubes, 6L6s, on the overall
amplification factor. As the loop gain increased, the system became less
and less influenced by changes in the vacuum tube characteristics, until
the point whether they no longer had any measurable effect. At that point
the instructor asked the class, “Ok, so tell me, why can’t we just
pull the 6L6s out of their sockets and have the circuit go on working
without them?” Paradoxically, the only reason the circuit was
unaffected by changes in the vacuum tube properties was that the
vacuum tubes were in fact present and amplifying. The ideal case of
infinite loop gain, in this case, was totally misleading.

This is true of Ashby’s idealized compensatory circuit (I refuse to call
it a control system because it can’t defend against disturbances acting
directly on E without a link through R). My argument hinges on the
idealization of a system without noise, which is all that permits anyone
to say that the compensation can be perfect in this kind of system. If
Ashby can say that compensation is perfect here, I can say that an ideal
negative feedback system also controls perfectly, because its loop gain
is infinite. Neither claim would allow us to draw correct conclusions
about real systems designed in these respective ways. Neither idealized
design is achievable in practice.

Ashby’s ideal regulator does
prevent all variation in the disturbance from reaching the essential
variables, and without requiring a system with infinite
gain.

But it requires a system with zero noise, infinite precision, and perfect
knowledge of all laws of nature (including all those not discovered
yet)…

I agree that it won’t work
so well in practice, except perhaps in certain restricted
environments. The device is theoretically capable of perfect
regulation; the error-controlled regulator is not theoretically capable
of perfect regulation, although under the right conditions (permitting
very high gains), it can come close.

That’s more or less how Ashby and many others reasoned – Hans Blom
reasoned that way only a few years ago on CSGnet in defending
“modern control theory.” But just sit down and make a list of
all the things that have to work perfectly or be perfectly known or be
calculated with zero error and infinite speed, and this line of reasoning
rapidly loses its persuasiveness. Probably the pinnacle of the art in
designing compensators was reached in those clocks I spoke of with
compensating pendulums. If kept in a protected environment, carefully and
frequently lubricated with the best oils, powered by weights wound up
every few minutes by an electric motor, and compared with the stars at
frequent intervals, they could keep time within perhaps one minute per
week (I don’t know the real number, only that it’s pretty crude compared
with today’s crystal-controlled $5 wristwatches).

If there is information in D, it will be perfectly preserved in E,
because simply scaling a signal up or down in magnitude has no effect on
the amount of information it carries.

Now
let’s return for a moment to a system with only a certain number of
“distinguishable states,” say the 256 distinguishable states
available in an 8-bit binary number.

If the regulator cuts the amplitude of the effect of the
disturbance on the controlled variable in half, there are only half as
many distinguishable states available over which the variable can vary,
and information has been lost.

If there were only 8 bits, then E could be compensated against
disturbance only within 1 part in 256, which is pretty far from perfect
compensation. The “perfect” compensation occurs only with an
infinite number of discriminable states. When your 8-bit display showed
that E was undisturbed, it could actually be disturbed by up to one least
significant digit.

But you bring up a relevant point: the measuring instrument through which
we observe the state of this system. When we cut the amplitude of
disturbance effects in half, the measuring instrument with 8-bit
precision will indeed lose information, but not because information has
been lost in the system being measured. It has been lost only to the
external observer of this system, The only way for information to be lost
in the system itself is for R to have only n-bit accuracy, in which case
E cannot be compensated perfectly.

Well, yes – in an error-controlled
regulator, you can reduce the error to almost any desired value by making
the gain high enough (so long as the system in question remains
stable). Ashby’s point is simply that you can’t make it zero.
To that I would add that, in many systems, you can’t make the gain very
high, either, without introducing instability.

As I said, I have seen voltage regulators with a loop gain of 10^8 (and,
of course, rather low bandwidth). My point was simply that using negative
feedback we can approach perfect regulation much more closely than we can
with any open-loop compensatory scheme. It’s not really cricket to
compare Ashby’s ideal compensator in a noiseless universe with a control
system subject to practical limitations. Compare either the ideal forms
or the practical forms.

A good deal of the problem here is that there is a misconception to the
effect that open-loop compensation is simpler than negative feedback
control. It is not: it is immensely more complex, when you get right down
to designing such a system. In a negative feedback control system, the
only really critical parts are the sensor and the comparator, with the
comparator being the most critical. All the other parts can be fairly
sloppy with large tolerances and no great stability over time.
Furthermore, the negative feedback control system needs no knowlege of
the disturbance, as to its physical nature, its mode of action, or even
the number of separate and independent disturbances comprising it. It
needs no measure of the disturbance at all.

Compare that with the compensatory system, in which success depends
completely on being able to sense the state of the disturbing variable
with very high precision, on knowing the physical laws that convert that
state into an effect on the transmission channel T, and on knowing
the same information about the branch that passes from D to R and from R
to T. Any change in any of these properties will be reflected directly in
a perturbation of E. Any realistic assessment of the requirements on a
stimulus-response compensator will show just how impracticable such a
design really is, and how wasteful in comparison with the simple negative
feedback controller.

By the way, I’d really like to know
how those Bose noise-cancelling headphones work.

I’d prefer to call them noise-reducing headphones. I don’t know how they
work either, but you can be sure they don’t “cancel”
noise.

When you reduce the magnitude, you
reduce the variety, unless you have infinite fidelity.

My point precisely. Ashby’s design has infinite fidelity, which is all
that permits him to say it can compensate perfectly. As soon as you
revert to less than perfect fidelity – continuous noise or digitizing
noise – the precision of the compensation falls abruptly into the normal
modest range found in real compensating devices. And negative feedback
control, even in real devices, always works better and faster with fewer
requirements on component stability and adjustment.

However, there do seem to be
certain limited cases in which compensation seems to have been
implemented in biological systems. One I have in mind at the moment
is the system that regulates body temperature. Although the main
job of the system is to keep core body temperature within certain bounds,
we have skin sensors that provide the brain with advance warning that the
body is gaining or loosing heat. We begin to take action against
these disturbances long before core body temperature is much
affected.

That’s simply adding rate feedback to the proportional feedback, a common
method of stabilization. The key word is “much”. There will be
no correction unless the core body temperature is affected; if that
were not true, an error would be produced instead of corrected. But the
amplification is sufficient that not “mjuch” error is needed to
produce a considerable change in cooling or heating action, a requirement
of tight control. Adding the rate feedback allows the gain to be higher
without producing oscillations.

One parting comment on Ashby.
On Pp. 195-196, near the beginning of the Chapter 10 (Regulation in
Biological Systems), Ashby (1957) had this to say:
The subject of regulation in biology is so vast that no single
chapter can do it justice. Cannon’s Wisdom of the Body
treated it adequately so far as internal, vegetative activities are
concerned, but there has yet to be written the book, much larger in size,
that shall show how all the organism’s exteriorly-directed activities –
its “higher” activities – are all similarly regulatory, i.e.,
homeostatic.For some reason, B:CP comes to mind.

Thanks, buddy. The check is in the mail.

Best,

Bill P.

[From Bruce Abbott (2004.05.13.1120 EST)]

Bill Powers (2004.05.13.1249
MST)

Bruce Abbott (2004.05.12.1910 EST)–

Yes, and real springs do not behave
quite like the ideal springs that physicists are so fond of. You
seem to be making an argument that there is nothing to be gained by
considering ideal systems.

Not at all, but one has to approach ideal systems with an eye to reality.
Back when I first learned about negative feedback, in Navy electronics
school early in the winter of 1944-45, our instructor showed us the
equations for an amplifier with feedback, with particular regard to the
effect of the gain in the output vacuum tubes, 6L6s, on the overall
amplification factor. As the loop gain increased, the system became less
and less influenced by changes in the vacuum tube characteristics, until
the point whether they no longer had any measurable effect. At that point
the instructor asked the class, “Ok, so tell me, why can’t we just
pull the 6L6s out of their sockets and have the circuit go on working
without them?” Paradoxically, the only reason the circuit was
unaffected by changes in the vacuum tube properties was that the
vacuum tubes were in fact present and amplifying. The ideal case of
infinite loop gain, in this case, was totally
misleading.

The point is well-taken, but I would also note that one can often do an
excellent job of accounting for observed behavior without having to
resort to such detail. The control systems of our computer
simulations take no account of the real physical systems that come into
play during a human tracking exercise. You will not find any
mention of the masses of the shoulder-arm-mouse system, the stiffness of
the bones, or the mechanisms through which the person perceives the
target and cursor on the screen, yet the match between the behavior of
the simulated control system and the actual one is excellent. (Of
course, it could be improved by including such things.)

I agree that it [Ashby’s compensator] won’t work so well in practice,
except perhaps in certain restricted environments. The device is
theoretically capable of perfect regulation; the error-controlled
regulator is not theoretically capable of perfect regulation, although
under the right conditions (permitting very high gains), it can come
close.

That’s more or less how Ashby and many others reasoned – Hans Blom
reasoned that way only a few years ago on CSGnet in defending
“modern control theory.” But just sit down and make a list of
all the things that have to work perfectly or be perfectly known or be
calculated with zero error and infinite speed, and this line of reasoning
rapidly loses its persuasiveness. Probably the pinnacle of the art in
designing compensators was reached in those clocks I spoke of with
compensating pendulums. If kept in a protected environment, carefully and
frequently lubricated with the best oils, powered by weights wound up
every few minutes by an electric motor, and compared with the stars at
frequent intervals, they could keep time within perhaps one minute per
week (I don’t know the real number, only that it’s pretty crude compared
with today’s crystal-controlled $5 wristwatches).

I don’t understand why you argue so strongly against the compensator
mechanism. Under certain circumstances, it can work quite well, and
it is possible to design hybrid mechanisms that have the advantages of
both types of system. It wouldn’t undermine PCT if such mechanisms
were actually identified in the body. It isn’t as if one has to
choose between one or the other.

With respect to the need to match the compensator’s characteristics to
those of the system to be stabilized, adaptive systems exist that can
detect a mismatch and change the compensator’s parameters
accordingly. In essence, the adaptive component is just an
error-controlled regulator piggybacked onto the compensator.

Now
let’s return for a moment to a system with only a certain number of
“distinguishable states,” say the 256 distinguishable states
available in an 8-bit binary number.

If the regulator cuts the amplitude of the effect of the
disturbance on the controlled variable in half, there are only half as
many distinguishable states available over which the variable can vary,
and information has been lost.

If there were only 8 bits, then E could be compensated against
disturbance only within 1 part in 256, which is pretty far from perfect
compensation. The “perfect” compensation occurs only with an
infinite number of discriminable states. When your 8-bit display showed
that E was undisturbed, it could actually be disturbed by up to one least
significant digit.

In Ashby’s discrete-case analysis (used by Ashby as a teaching device
because it is relatively simple to understand), the disturbance would be
measurable to the same resolution as the compensator’s output. The
Law of Requisite Variety makes the very simple point that the compensator
would have to have at least the same variety of states as the
disturbance, if the effect of the disturbance on the essential variables
is to be completely nullified. So we might have a disturbance that
could range over the integers -5 to +5, and a compensator output with the
same capability. Below might be the input (disturbance) pattern
(D), compensator pattern (R), and effect on the essential variables (E),
measured at equal intervals of time:
T D R E
1 -1 +1 0
2 0 0 0
3 +1 -1 0
4 +3 -3 0
5 +5 -5 0
6 +2 -2 0
7 0 0 0
8 -1 +1 0
9 -4 +4 0
10 -5 +5 0As you can see, the variety in R completely destroys the variety in
D that is transmitted to E.

But you bring up a relevant point:
the measuring instrument through which we observe the state of this
system. When we cut the amplitude of disturbance effects in half, the
measuring instrument with 8-bit precision will indeed lose information,
but not because information has been lost in the system being measured.
It has been lost only to the external observer of this system, The only
way for information to be lost in the system itself is for R to have only
n-bit accuracy, in which case E cannot be compensated perfectly.

Well, yes – in an error-controlled
regulator, you can reduce the error to almost any desired value by making
the gain high enough (so long as the system in question remains
stable). Ashby’s point is simply that you can’t make it zero.
To that I would add that, in many systems, you can’t make the gain very
high, either, without introducing instability.

As I said, I have seen voltage regulators with a loop gain of 10^8 (and,
of course, rather low bandwidth). My point was simply that using negative
feedback we can approach perfect regulation much more closely than we can
with any open-loop compensatory scheme. It’s not really cricket to
compare Ashby’s ideal compensator in a noiseless universe with a control
system subject to practical limitations. Compare either the ideal forms
or the practical forms.

If a practical compensator can measure and reproduce the disturbance to a
given level of precision, it can compensate perfectly for the disturbance
at the same level of precision. The same limit applies to the
error-controlled regulator, which must amplify the error signal (residual
difference between reference and input) in order to produce an
output. This limits the useful gain. The loop gain in
biological control systems seems to me more on the order of 10 to 100
than 10*8, so the residual error between reference and input remains
relatively high unless additional design features are added to the
system.

A good deal of the problem here is
that there is a misconception to the effect that open-loop compensation
is simpler than negative feedback control. It is not: it is immensely
more complex, when you get right down to designing such a system. In a
negative feedback control system, the only really critical parts are the
sensor and the comparator, with the comparator being the most critical.
All the other parts can be fairly sloppy with large tolerances and no
great stability over time. Furthermore, the negative feedback control
system needs no knowlege of the disturbance, as to its physical nature,
its mode of action, or even the number of separate and independent
disturbances comprising it. It needs no measure of the disturbance at
all.

Compare that with the compensatory system, in which success depends
completely on being able to sense the state of the disturbing variable
with very high precision, on knowing the physical laws that convert that
state into an effect on the transmission channel T, and on knowing
the same information about the branch that passes from D to R and from R
to T. Any change in any of these properties will be reflected directly in
a perturbation of E. Any realistic assessment of the requirements on a
stimulus-response compensator will show just how impracticable such a
design really is, and how wasteful in comparison with the simple negative
feedback controller.

An excellent summary of the advantages of negative feedback control over
compensator action. However, evolution seems to have produced some
rather messy, complicated systems here and there, sometimes resembling
more the old compensator clocks you described earlier than the simple
negative-feedback controller. I wouldn’t reject altogether,
based purely on such arguments of relative efficiency, the possibility of
such compensator systems existing in the brain and body.

By
the way, I’d really like to know how those Bose noise-cancelling
headphones work.

I’d prefer to call them noise-reducing headphones. I don’t know how they
work either, but you can be sure they don’t “cancel”
noise.

You must be using the word “cancel” to mean “completely
eliminate.” I don’t think the designers of noise-canceling
headphones intended that meaning. Basically, it’s just a matter of
inverting the noise signal and beating it against the level of the same
noise entering the ear, i.e., using destructive interference to
“cancel” out the noise waveform. But you know all that,
you’re just being difficult.

Negative feedback control systems do much the same thing (within limits):
The disturbance and output waveforms are near mirror-images and their sum
(after appropriate weighting) gives the residual (post-cancellation)
effect of the disturbance on the CV. If the disturbance is noise,
then control systems are noise-canceling systems.

When
you reduce the magnitude, you reduce the variety, unless you have
infinite fidelity.

My point precisely. Ashby’s design has infinite fidelity, which is all
that permits him to say it can compensate perfectly. As soon as you
revert to less than perfect fidelity – continuous noise or digitizing
noise – the precision of the compensation falls abruptly into the normal
modest range found in real compensating devices. And negative feedback
control, even in real devices, always works better and faster with fewer
requirements on component stability and adjustment.

However, there do seem to be
certain limited cases in which compensation seems to have been
implemented in biological systems. One I have in mind at the moment
is the system that regulates body temperature. Although the main
job of the system is to keep core body temperature within certain bounds,
we have skin sensors that provide the brain with advance warning that the
body is gaining or loosing heat. We begin to take action against
these disturbances long before core body temperature is much
affected.

That’s simply adding rate feedback to the proportional feedback, a common
method of stabilization. The key word is “much”. There will be
no correction unless the core body temperature is affected; if that
were not true, an error would be produced instead of corrected. But the
amplification is sufficient that not “mjuch” error is needed to
produce a considerable change in cooling or heating action, a requirement
of tight control. Adding the rate feedback allows the gain to be higher
without producing oscillations.

I’m not convinced that your analysis is entirely accurate. Core
body temperature appears to be sensed in the hypothalamus, whereas skin
temperature is sensed in the skin. The temperature sensors in the
skin are very rate sensitive, so rapid heading or cooling produces an
enhanced sense of warmth or cold. When the skin temperature levels
off, the sensation diminishes. In addition, the sensors adapt
rapidly, so that what feels hot or cold depends most strongly on the
difference between the object being touched and skin temperature,
as opposed to the absolute temperature of the object. This is why
the same pail of room-temperature water will feel cold if your
hands have just been soaking in hot water and hot if your hands have been
soaking in ice water. The rate-control you mentioned might be
implemented via these receptors (which because of their location on the
skin have the ability to respond more rapidly to changes in heat flow
than the hypothalamus does) rather than via the brain taking the
derivative of the error signal for core body temperature.
Compensation could begin even before the core body temperature began to
change.

One
parting comment on Ashby. On Pp. 195-196, near the beginning of the
Chapter 10 (Regulation in Biological Systems), Ashby (1957) had this to
say:
The subject of regulation in biology is so vast that no single
chapter can do it justice. Cannon’s Wisdom of the Body
treated it adequately so far as internal, vegetative activities are
concerned, but there has yet to be written the book, much larger in size,
that shall show how all the organism’s exteriorly-directed activities –
its “higher” activities – are all similarly regulatory, i.e.,
homeostatic.For some reason, B:CP comes to mind.

Thanks, buddy. The check is in the mail.

I’m afraid can’t accept a check. On behalf of my fellow Knights Who
Say “Nee!” I demand that you bring me a shrubbery. Make
it a nice one, not too expensive!

Bruce A.

[From Bill Powers (2004.05.13.1101 MST)]

Bruce Abbott (2004.05.13.1120 EST)–

I don’t understand why you argue so
strongly against the compensator mechanism. Under certain
circumstances, it can work quite well, and it is possible to design
hybrid mechanisms that have the advantages of both types of system.
It wouldn’t undermine PCT if such mechanisms were actually identified in
the body. It isn’t as if one has to choose between one or the
other.

Of course not. There is a already place for compensator mechanisms in
PCT: it is called the output function of a control system. A signal sent
to the input of this device produces an output effect that joins with the
effects of disturbances on the input variable that is under control. As
it happens, this output effect from the Regulator very nearly cancels the
effect of the Disturbance on the controlled variable E. The difference is
that this result is obtained by monitoring the state of the controlled
variable instead of the state of the disturbing variable. This means it
is no longer necessary to know what is causing the disturbance, or all of
the external variables that are disturbing the controlled
variable…

It has been proposed frequently that control can be improved by providing
information about the state of the disturbance. Every test I have made of
that proposal has gone against it. What actually happens is that
attention is devoted to the disturbance at the expense of attention to
the controlled variable, and control simply gets worse. There may be
special circumstances where knowledge of the disturbance can be crucial
to success (where preparation for action is required for example) but
that is far from the default case. Normally we conrol systems don’t have
to know what is causing any deviation of the controlled variable to its
reference level. We just act directly on the controlled variable until
the error is made small again.

With respect to the need to match
the compensator’s characteristics to those of the system to be
stabilized, adaptive systems exist that can detect a mismatch and change
the compensator’s parameters accordingly. In essence, the adaptive
component is just an error-controlled regulator piggybacked onto the
compensator.

Yes, and that scheme has to involve perception of the controlled
variable. The particular scheme Hans Bloom was selling involved an
internal model of the controlled system which produced a predicted value
of the controlled variable. That predicted value was compared with actual
value (as sensed), and the difference was used to adjust the parameters
of the internal model. The theory was that this system would be able to
go on controlling even after losing the ability to sense the controlled
variable. Of course that would not work in the presence of any
unanticipated disturbances, and the accuracy of control would depend
entirely on the accuracy of the internal model. That takes us back to the
problem of how complete that model must be to accomplish anything of
interest in a realistic environment. However, I always agreed that this
mode of operation (which I called the “imagination connection”
in B:CP) could have its uses, within its inherent limitations. I do
believe we use it quite a lot, but only in the higher-level control
systems in which quantitative accuracy is not important.

In Ashby’s discrete-case analysis
(used by Ashby as a teaching device because it is relatively simple to
understand), the disturbance would be measurable to the same resolution
as the compensator’s output. The Law of Requisite Variety makes the
very simple point that the compensator would have to have at least the
same variety of states as the disturbance, if the effect of the
disturbance on the essential variables is to be completely
nullified.

But that puts it in terms of the objective states of the disturbance and
the essential variable, which in any macroscopic physical system are
continuous, not discrete, variables. The fact that our measuring
instruments, or models that use only a few bits to represent numbers,
have limited resolution says nothing about how the measured system works.
The essential variable that differs from its intended state by less than
one significant digit seems to be undisturbed, but it is not.

So we might have a
disturbance that could range over the integers -5 to +5, and a
compensator output with the same capability. Below might be the
input (disturbance) pattern (D), compensator pattern (R), and effect on
the essential variables (E), measured at equal intervals of time:
T D R E
1 -1 +1 0
2 0 0 0
3 +1 -1 0
4 +3 -3 0
5 +5 -5 0
6 +2 -2 0
7 0 0 0
8 -1 +1 0
9 -4 +4 0
10 -5 +5 0As you can see, the variety in R completely destroys the variety in
D that is transmitted to E.

This is exactly what I meant by saying that Ashby’s examples work only
because of the use of small whole numbers. The real system is continuous;
the model is discrete, so it does not accurately represent the real
system. Only an imaginary system could work the way you describe above.

Don’t overlook the fact that if you measured the behavior of a control
system in the same way as above, it would require a loop gain of only 11
or 12 to give the appearance of maintaining exactly zero error. You
wouldn’t see any errors less than one unit in magnitude: they would all
be reported as zero.

If
a practical compensator can measure and reproduce the disturbance to a
given level of precision, it can compensate perfectly for the disturbance
at the same level of precision.

That measurement and reproduction involves not just sensing the state of
the disturbing variable, but calculating the effect of the disturbing
variable on the transmission channel and (by a different path) on the
regulator , and the effect of the regulator on the transmission channel,
so as to produce the right amount of compensating output from the
regulator. And I think there is a considerable difference between
“compensate perfectly” and “compensate at the same level
of precision,” meaning imperfectly.

The same limit applies to the error-controlled regulator, which must
amplify the error signal (residual difference between reference and
input) in order to produce an output. This limits the useful
gain. The loop gain in biological control systems seems to me more
on the order of 10 to 100 than 10*8, so the residual error between
reference and input remains relatively high unless additional design
features are added to the system.

The error-controlled regulator does not need to sense the state of the
disturbing variable at all, nor, consequently, does it need to know
anything about how either the state of the disturbing variable or its own
output affects the essential (controlled) variable. The usable gain is
limited by time-delays in the loop, not by the magnitudes of the signals.
When dynamic stability is preserved, the error signal goes as (ref
signal)/(1+G) while the perceptual signal is G/(1+G) of the value of the
reference signal.Raise the gain (with a suitable adjustment of the
slowing factor) and the error simply gets smaller as the perception gets
closer to the reference signal…No particular amount of amplification is
implied; the amplification simply determines the closeness of control. If
you double the amplification, you halve the error.

An excellent summary of the
advantages of negative feedback control over compensator action.
However, evolution seems to have produced some rather messy, complicated
systems here and there, sometimes resembling more the old compensator
clocks you described earlier than the simple negative-feedback
controller.

I’d correct that to say that evolution has produced some systems which
have been routinely interpreted as compenatory systems, and among which a
few might prove actually to be compensatory systems. It’s easy to find
out which kind of system you have. If you disturb the essential variable
by a means that the controller can’t sense (except by sensing the
essential variable), a negative feedback system will be able to resist
the change, while the compensator will not. ,

I wouldn’t reject
altogether, based purely on such arguments of relative efficiency,
the possibility of such compensator systems existing in the brain and
body.

I don’t rejected them as a matter of principle; in fact the principles I
have adopted require setting up every experiment so that either
explanation might apply, and so we can tell which is correct. You’ll
notice that every control experiment I set up contains a disturbance
whose source cannot be sensed or predicted by the controlling person (or
by me, or by the program). If in any case, or even at any particular time
for a given person, a compensating system existed instead of a control
system, the person in the experiment would find it impossible to control
the variable. Since that has never, so far, happened, I conclude that we
have always been seeing the behavior of a negative feedback control
system, the only kind I know about that can work under those
circumstances.

What about circumstances where a compensating system could possibly work?
So far, as I said above, all tests in which information about the
disturbance is made available have resulted in poorer performance unless
the subject reported not noticing, or ignoring, that informtion. There’s
some wiggle room there, but not much.

By
the way, I’d really like to know how those Bose noise-cancelling
headphones work.

I’d prefer to call them noise-reducing headphones. I don’t know how they
work either, but you can be sure they don’t “cancel”
noise.

You must be using the word “cancel” to mean “completely
eliminate.”

That how I interpret the word “cancel.” In A + 3 = B + 3,
canceling the threes means removing them entirely, not leaving a little
bit on one side.

I don’t think the designers
of noise-canceling headphones intended that meaning.

Of course they did; it makes their product look better than saying the
headphones merely “reduce noise.”. It’s like saying that
aspirin prevents heart attacks, or more subtly, “helps prevent”
them. Preventing something means keeping it from happening, as most
people understand the term. the Supreme Court, at one point, called this
“harmless puffery.” I call it lying.

Basically, it’s just a matter
of inverting the noise signal and beating it against the level of the
same noise entering the ear, i.e., using destructive interference to
“cancel” out the noise waveform. But you know all that,
you’re just being difficult.

This presumes that you can separae the noise signal from the real signal.
As you guessed, an external microphone picking up room noise might work
in a compensating system. I’m not being difficult, I’m being
stubborn.

Negative feedback control systems
do much the same thing (within limits): The disturbance and output
waveforms are near mirror-images and their sum (after appropriate
weighting) gives the residual (post-cancellation) effect of the
disturbance on the CV. If the disturbance is noise, then control
systems are noise-canceling systems.

Control systems obtain their compensating outputs from the difference
between the essential variable and a signal represents its desired state.
Compensating systems obtain their outputs from sensing the state of the
disturbance, with the desired state existing in the head of the person
who does the tweaking of the amount of compensation. Control systems
reduce noise (unwanted disturbances of the controlled variable) by acting
directly on the controlled variable; compensating systems can’t reduce
that kind of noise at all.

Which suggests this: suppose you put a microphone in the earphone right
where the sound enters the ear,. You compare the microphone signal with
the electrical signal specifying what the sound pressure should be in
every successive microsecond, and have the control system dynamically
adjust the electrical input to the earphones to force a match. In that
case, the sound waves entering the ear would match the electrical input
waveforms, and any additional sounds not represented in the electrical
input would be errors. Those errors would change the excitation of the
earphones (very rapidly) and would oppose the extraneous variations.
Wanna bet that’s how they do it?

The compensating version of this system would place the microphone
outside the earphones, picking up the extraneous noises. The resulting
electrical signal would be subtracted from the signal entering the
earphones, with the earphone output reflecting the difference, carefully
adjusted for near-cancellation. This system would have difficulties with
noise that came from different directions and that occurred at different
frequencies. The real control system, which relies only on sensing the
actual sound waves as they enter the ear, would be the most
accurate.

That’s
simply adding rate feedback to the proportional feedback, a common method
of stabilization.

I’m not convinced that your
analysis is entirely accurate. Core body temperature appears to be
sensed in the hypothalamus, whereas skin temperature is sensed in the
skin. The temperature sensors in the skin are very rate sensitive,
so rapid heading or cooling produces an enhanced sense of warmth or
cold. When the skin temperature levels off, the sensation
diminishes. In addition, the sensors adapt rapidly, so that what
feels hot or cold depends most strongly on the difference between
the object being touched and skin temperature, as opposed to the absolute
temperature of the object. This is why the same pail of
room-temperature water will feel cold if your hands have just been
soaking in hot water and hot if your hands have been soaking in ice
water. The rate-control you mentioned might be implemented via
these receptors (which because of their location on the skin have the
ability to respond more rapidly to changes in heat flow than the
hypothalamus does) rather than via the brain taking the derivative of the
error signal for core body temperature. Compensation could begin
even before the core body temperature began to change.

All of that is very strong evidence for the negative feedback control
system with added rate feedback for damping. The proportional signal
comes from core. temperature, at least part of the rate component from
skin temperature… A disturbance that alters skin temperature generates a
transient error in the central control system right away; the core
temperature, which lags behind the peripheral temperature contributes to
the error later as the rate signal is dying away (after a step
disturbance). Both signals contribute to the same error signal, which
activates the mechanisms for heating and cooling. Any source of rate
information will provide damping; you;re describing a peripheral source.
There could also be computations of rate of change (run cold water over a
just-boiled egg and take it in your hand. You will feel the temperature
increasing, a central perception of rate of change of peripheral
temperature.)

Thanks,
buddy. The check is in the mail.

I’m afraid can’t accept a check. On behalf of my fellow Knights Who
Say “Nee!” I demand that you bring me a shrubbery. Make
it a nice one, not too expensive!

I have just the thing. It has nice little flowers like morning glories,
and it stays low to the ground so you needn’t trim or mow it. Also, it
has the nifty property of being able to wind tendrils around undesired
plants and pull them up by the roots (at least plants that IT doesn’t
desire to compete with). You can also apply pesticides, herbicides, or
dynamite without fear of disturbing it. I have a generous supply of it in
my front lawn, which I’m sure will not be significantly diminished by
ripping out three quarters of it and bringing it to the meeting in July.
By the time I get back home it will all have been replenished, which
makes maintenance (of it) quite inexpensive…

Generously,

Bill P.