Information and Control

CSGnet Archive CSG2012
1

[From Bruce Abbott (2012.12.13.1555 EST)]

I’ve thought of a way to make the role of information in a negative feedback control system clear – I hope. I’m modeling my approach on Ashby’s use of discrete variables to clarify issues that are more difficult to comprehend in the continuous case.

Imagine we have a control system that is attempting to keep a certain voltage constant at 0 volts, and that the range over which the voltage can vary is from 0 to 10 volts. It’s a high gain controller, so the error is very small and for our purposes can be ignored. The control system counteracts any disturbance by producing an equal and opposite voltage (range 0 to -10 volts), which is fed back to stabilize the controlled variable.

We, the observers of the system, cannot measure the disturbance that is acting on the system, but we do have access to the voltage that is being fed back to counteract the effect of the disturbance, and are recording that voltage. At first the recorded voltage is zero, but after some time has passed it suddenly jumps to -3 volts, holds that value for one second, then jumps to -7 volts for one second, etc. This pattern continues until we have the following recorded values: -3, -7, -1, -5, -9, -2, -7, -4, -10, the -10 holding for two seconds rather than the usual one second. After that sequence, the output stays at zero.

Question: what information was “in” the disturbance waveform? The answer is my office phone number.

The zeros indicate that the disturbance voltage is zero (thus the output is zero); a zero may be taken to be the idle state when no number is being transmitted. When the monitored voltage jumps to -3 volts, this -3 volts is counteracting a positive disturbance of 3 volts. But in my code, one volt represents the number zero, two volts the number 1, and so on up to 10 volts, which indicates the number 9. Applying this code to the sequence of recorded output values yields 2604816399.

Given that we know that single digits are being transmitted, each successive digit could have any value ranging from 0 to 9. After that digit has been transmitted to the controlled variable (by the disturbance voltage), the control system acts to nullify the disturbance by creating an opposing voltage, which we can read and translate via our code back to the original number. Uncertainty as to the transmitted value has been reduced by our reading of the output from ten possible values to one. The information transmitted, in information theoretic terms, is equal to this reduction in uncertainty, although it would be expressed as a number of bits. (I don’t have a calculator handy right now but it would be a little more than three bits. Three bits = 2 raised to the 3rd power, or 8 values, in this case ranging from 0 to 7. We have 10 possible values, more than 3 bits but less than 4 bits, which would allow 16 values). Let’s call it 3 bits.

Each successive second a new number is transmitted until the 10 digit series has been completed, so that’s a transmission rate of approximately 3 bits/second. In the end I have reduced the uncertainty about the possible 10-digit number to zero. This was accomplished without having any access to the disturbance values, nor any knowledge of the transformations going on inside the control system. (Recall that we are assuming our standard proportional controller here.)

We could add an additional wrinkle if we assume that a leaky integrator in the output function; in that case the output would require a little time to settle onto each new value, but otherwise the result would be the same after an exponential rise or fall toward the new value.

A spy attempting to read the coded phone number by intercepting the value of the controlled variable would see only a nearly constant voltage and probably conclude that no message was being transmitted.

Obviously, my office phone number is not being “used” by this control system for any purpose whatsoever, certainly not for the purpose of control. So much for that confusion. Obviously, information is being transmitted from disturbance to output. Obviously, the only channel through which this information can be transmitted is the control loop. Obviously, the information transmitted does not reflect the internal organization of the control system (no behavioral illusion at work here).

Bruce

2

[From Rick Marken (2012.12.13.1415)]

Bruce Abbott (2012.12.13.1555 EST)–

BA: I’ve thought of a way to make the role of information in a negative feedback control system clear – I hope. I’m modeling my approach on Ashby’s use of discrete variables to clarify issues that are more difficult to comprehend in the continuous case.

RM: Well this certainly does help. Apparently information theory is just used to learn about the disturbance “message” that is influencing a controlled variable. In order to use it this way you have to 1) know what variable the system is controlling (so you have to have done the test for the controlled variable before you can even use information theory) 2) know that the system is controlling that variable at a fixed reference level with very high gain 3) arrange to have the feedback and disturbance functions be constants and as noiseless as possible. Once you have all this in place you can be pretty sure that the outputs of the system are a refection of the disturbance to the controlled variable. The human control system is being used like a very expensive voltmeter to measure the disturbance waveform. Since I’m rarely interested in learning what “messages” are being sent by disturbances it seems that I wouldn’t have much use for information theory.

Maybe the use of information theory is in measuring the rate at which information is transferred from disturbance to output. Why we would want to know that is not immediately apparent to me, but, if we did, we would still have to know not only all those things I said above (all of which require a great deal of prior knowledge of how control systems work) and but also the disturbance itself (the thing we were using the person to find out about in the first place). And even then, if the system is a high gain controller (as it has to be for the output to accurately reflect the information in the disturbance) then the relationship between d and o depends almost entirely on the ratio of disturbance function to feedback function. So the information transmission rate measured reflects characteristics of the system’s environment, not of the system itself.

So I really just don’t see the relevance of information theory to the study of living systems. I think it’s probably a nice tool for analyzing (and making requirements for) communication systems, though.

Best

Rick

···

Imagine we have a control system that is attempting to keep a certain voltage constant at 0 volts, and that the range over which the voltage can vary is from 0 to 10 volts. It’s a high gain controller, so the error is very small and for our purposes can be ignored. The control system counteracts any disturbance by producing an equal and opposite voltage (range 0 to -10 volts), which is fed back to stabilize the controlled variable.

We, the observers of the system, cannot measure the disturbance that is acting on the system, but we do have access to the voltage that is being fed back to counteract the effect of the disturbance, and are recording that voltage. At first the recorded voltage is zero, but after some time has passed it suddenly jumps to -3 volts, holds that value for one second, then jumps to -7 volts for one second, etc. This pattern continues until we have the following recorded values: -3, -7, -1, -5, -9, -2, -7, -4, -10, the -10 holding for two seconds rather than the usual one second. After that sequence, the output stays at zero.

Question: what information was “in” the disturbance waveform? The answer is my office phone number.

The zeros indicate that the disturbance voltage is zero (thus the output is zero); a zero may be taken to be the idle state when no number is being transmitted. When the monitored voltage jumps to -3 volts, this -3 volts is counteracting a positive disturbance of 3 volts. But in my code, one volt represents the number zero, two volts the number 1, and so on up to 10 volts, which indicates the number 9. Applying this code to the sequence of recorded output values yields 2604816399.

Given that we know that single digits are being transmitted, each successive digit could have any value ranging from 0 to 9. After that digit has been transmitted to the controlled variable (by the disturbance voltage), the control system acts to nullify the disturbance by creating an opposing voltage, which we can read and translate via our code back to the original number. Uncertainty as to the transmitted value has been reduced by our reading of the output from ten possible values to one. The information transmitted, in information theoretic terms, is equal to this reduction in uncertainty, although it would be expressed as a number of bits. (I don’t have a calculator handy right now but it would be a little more than three bits. Three bits = 2 raised to the 3rd power, or 8 values, in this case ranging from 0 to 7. We have 10 possible values, more than 3 bits but less than 4 bits, which would allow 16 values). Let’s call it 3 bits.

Each successive second a new number is transmitted until the 10 digit series has been completed, so that’s a transmission rate of approximately 3 bits/second. In the end I have reduced the uncertainty about the possible 10-digit number to zero. This was accomplished without having any access to the disturbance values, nor any knowledge of the transformations going on inside the control system. (Recall that we are assuming our standard proportional controller here.)

We could add an additional wrinkle if we assume that a leaky integrator in the output function; in that case the output would require a little time to settle onto each new value, but otherwise the result would be the same after an exponential rise or fall toward the new value.

A spy attempting to read the coded phone number by intercepting the value of the controlled variable would see only a nearly constant voltage and probably conclude that no message was being transmitted.

Obviously, my office phone number is not being “used” by this control system for any purpose whatsoever, certainly not for the purpose of control. So much for that confusion. Obviously, information is being transmitted from disturbance to output. Obviously, the only channel through which this information can be transmitted is the control loop. Obviously, the information transmitted does not reflect the internal organization of the control system (no behavioral illusion at work here).

Bruce

–
Richard S. Marken PhD

rsmarken@gmail.com
www.mindreadings.com

3

[From Richard Kennaway (2012.12.14 09:40 GMT)]

An actual example of a disturbance being measured by looking at the output function of a control system is a phase-locked loop (PLL), used to decode FM transmissions.

FM radio transmission works by sending a carrier signal, typically in the region of 100MHz. The audio signal is encoded onto this by varying the frequency of the carrier proportionally to the instantaneous value of the audio signal. This can be done in analog circuitry: make an oscillator that runs at around 100MHz but whose frequency can be adjusted by a voltage input, then apply the audio signal to that input. This is called a voltage-controlled oscillator (VCO) (although that use of the word "control" is not quite what we're talking about when we talk about "control systems"). To decode this in the receiver, you have to have a way of measuring that deviation in frequency. This is done by a PLL, which is in effect another VCO inside a control loop that inverts its function.

You take a VCO, which is first tuned (by the dial you turn on the radio) so that its range of oscillation is around the carrier frequency of the station you want to receive. A control circuit senses the phase difference between that oscillator and the signals from the antenna, and varies the control voltage so as to keep that phase difference at a fixed reference value (usually 90 degrees). So if the incoming signal increases in frequency, it will begin to get ahead of the local oscillator. The control circuit senses this and changes the control voltage so as to speed up the oscillator and maintain the phase relation at 90 degrees. This is a phase-locked loop.

The result is that the disturbance (the deviation of the incoming signal from the carrier frequency) is exactly matched by the control voltage. That control voltage is proportional to the original audio signal.

···

--
Richard Kennaway, jrk@cmp.uea.ac.uk, http://www.cmp.uea.ac.uk/~jrk/
School of Computing Sciences,
University of East Anglia, Norwich NR4 7TJ, U.K.

4

[From Bruce Abbott (2012.12.14.0700 EST)]

Richard Kennaway (2012.12.14 09:40 GMT)--

RK: An actual example of a disturbance being measured by looking at the
output function of a control system is a phase-locked loop (PLL), used to
decode FM transmissions.

RK: FM radio transmission works by sending a carrier signal, typically in
the region of 100MHz. The audio signal is encoded onto this by varying the
frequency of the carrier proportionally to the instantaneous value of the
audio signal. This can be done in analog circuitry: make an oscillator that
runs at around 100MHz but whose frequency can be adjusted by a voltage
input, then apply the audio signal to that input. This is called a
voltage-controlled oscillator (VCO) (although that use of the word "control"
is not quite what we're talking about when we talk about "control systems").
To decode this in the receiver, you have to have a way of measuring that
deviation in frequency. This is done by a PLL, which is in effect another
VCO inside a control loop that inverts its function.

RK: You take a VCO, which is first tuned (by the dial you turn on the radio)
so that its range of oscillation is around the carrier frequency of the
station you want to receive. A control circuit senses the phase difference
between that oscillator and the signals from the antenna, and varies the
control voltage so as to keep that phase difference at a fixed reference
value (usually 90 degrees). So if the incoming signal increases in
frequency, it will begin to get ahead of the local oscillator. The control
circuit senses this and changes the control voltage so as to speed up the
oscillator and maintain the phase relation at 90 degrees. This is a
phase-locked loop.

RK: The result is that the disturbance (the deviation of the incoming signal
from the carrier frequency) is exactly matched by the control voltage. That
control voltage is proportional to the original audio signal.

Thanks, Richard, for that nice example. I didn't know that FM systems work
that way!

It is my understanding that telephone touch-tone decoders also employ
phase-locked loops, to detect which pairs of audio frequencies have been
sent over the line to the telephone switching system. Each button on the
phone generates a specific pair of audio frequencies whose combination
indicates which button has been pressed.

I've sometimes wondered whether the human memory system might work is some
analogous fashion. Something appears in the input (a person's face,
perhaps); to the extent that this image matches some representation of that
face stored in memory, it activates that representation.

Bruce

5

[From Bruce Abbott (2012.12.14.0735 EST)]

Rick Marken (2012.12.13.1415)–

BA: Bruce Abbott (2012.12.13.1555 EST)

BA: I’ve thought of a way to make the role of information in a negative feedback control system clear – I hope. I’m modeling my approach on Ashby’s use of discrete variables to clarify issues that are more difficult to comprehend in the continuous case.

RM: Well this certainly does help. Apparently information theory is just used to learn about the disturbance “message” that is influencing a controlled variable.

In the second sentence above I’d substitute “can be used” for “is just used.” Doubtless there are other uses.

In order to use it this way you have to 1) know what variable the system is controlling (so you have to have done the test for the controlled variable before you can even use information theory) 2) know that the system is controlling that variable at a fixed reference level with very high gain 3) arrange to have the feedback and disturbance functions be constants and as noiseless as possible. Once you have all this in place you can be pretty sure that the outputs of the system are a refection of the disturbance to the controlled variable. The human control system is being used like a very expensive voltmeter to measure the disturbance waveform. Since I’m rarely interested in learning what “messages” are being sent by disturbances it seems that I wouldn’t have much use for information theory.

Some might find information theory useful for other purposes to which an information-theoretic analysis is suited. I’m not really in a position to say what those purposes might be, but I do recognize that they may exist. I’m fairly sure that you wouldn’t be among those interested in such applications, but hey, that’s just you.

Maybe the use of information theory is in measuring the rate at which information is transferred from disturbance to output. Why we would want to know that is not immediately apparent to me, but, if we did, we would still have to know not only all those things I said above (all of which require a great deal of prior knowledge of how control systems work) and but also the disturbance itself (the thing we were using the person to find out about in the first place). And even then, if the system is a high gain controller (as it has to be for the output to accurately reflect the information in the disturbance) then the relationship between d and o depends almost entirely on the ratio of disturbance function to feedback function. So the information transmission rate measured reflects characteristics of the system’s environment, not of the system itself.

Not exactly. System characteristics can limit the rate of information transfer, just as they can limit the frequency range of disturbances that a control system can effectively counter.

So I really just don’t see the relevance of information theory to the study of living systems. I think it’s probably a nice tool for analyzing (and making requirements for) communication systems, though.

Fair enough. But there may be others who can and will demonstrate such relevance. I wouldn’t rule out that possibility in advance.

Bruce

6

[Martin Taylor 2012.12.15.14.01]

[From Rick Marken (2012.12.13.1415)]

              Bruce Abbott

(2012.12.13.1555 EST)–

              BA:

I’ve thought of a way to make the role of information
in a negative feedback control system clear – I hope.
I’m modeling my approach on Ashby’s use of discrete
variables to clarify issues that are more difficult to
comprehend in the continuous case.

      RM: Well this certainly does help.  Apparently information

theory is just used to learn about the disturbance “message”
that is influencing a controlled variable. In order to use it
this way you have to 1) know what variable the system is
controlling (so you have to have done the test for the
controlled variable before you can even use information
theory) 2) know that the system is controlling that variable
at a fixed reference level with very high gain 3) arrange to
have the feedback and disturbance functions be constants and
as noiseless as possible.

Point 1 is close to being true. The caveat is only that in the real

world several perceptions are being controlled at different
perceptual levels, and if the person is using just one of the
lower-level inputs to a higher-level perception as the way to
influence that higher-level perception you can disturb one of the
other inputs while observing the output to the variable the person
is using. I grant that this is an unusual situation, so for the
purposes of argument, I grant your point 1.

Point 2 is really two points (a) fixed reference level, and (b) high

gain. Each of those need be only approximately true. Changes of
reference level and low gain both have effects like the effects of
noise, obscuring but not eliminating the message unless the “noise”
effect is too large. Shannon showed how large is “too large”. But
again, for the purposes of argument, we can grant both parts of your
Point 2.

Point 3. Again this is two points (a) feedback function constancy

and (b) disturbance function constnacy. In the real world, feedback
functions usually don’t change a lot very fast unless the feedback
path is broken entirely (the effordance vanishes). If they do,
control becomes difficult if not impossible. As in your demo, a
large change in the feedback function can act like a high=level
noise to obscure the signal, but as with changes in reference value
and low gain, so long as the changes in the feedback function are
slow enough and small enough, they don’t matter. If they are known,
at one extreme this becomes Hedy Lamarr’s frequency hopping spread
spectrum system for secret communication, ensuring that a spy who
could observe the output could not find the disturbance value. For
the sake of argument, we can grant your point 3a.

As for (3b) I don't know what you intend, but since Bruce's example

involves applying the disturbance directly to to controlled
environmental variable, I suppose the disturbance function is a
unity multiplier. So we can grant your Point 3.

Having granted all your points for the sake of argument, without

considering whether they are likely to be important in the real
world, we ask “what does it all mean”.

What it means is that if you want to use Bruce's control-system

method of communication, which is really just an ongoing measurement
of the disturbance value, you first have to provide a finite number
of bits of information – an envelope or a decoding key. You have to
calibrate your measuring apparatus, the control system. Once you
have completed the calibration, you can proceed with the measurement
for as long as you want, with a continuous stream of information
passing around the circuit from the disturbance through the output
back to the CEV where the ever-changing disturbance acts as a noise
source to reduce and eventually remove the information about its
past values.

Volatile changes in the control system parameters and in the

reference value have an effect similar to having an elastic ruler.
If you have information about the forces stretching and compressing
the ruler, you can still make good measurements. If you don’t, your
measurements will be less accurate, but you can still make some
worthwhile measurements if the amounts of stretch and compression
are slow enough or small enough.

··· 
I think this is a reasonable place to make a comment about something

that has been bothering me for quite a long time. Rick would like
PCT to be widely understood (correctly, of course) and used, as do
we all, I imagine. But he frequently asserts that the only valid
research that should be done with PCT is to search for “the
controlled variable”, as though everyone controlled the same
variables all the time. Furthermore, in this and related threads,
Rick has strongly defended the position that PCT cannot tell you
anything about what happens inside a person’s head (except what
variable s/he is controlling). If you believe that PCT can tell you
something about what happens inside the head, Rick says you are an
S-R theorist, and don’t really understand PCT.

I do not see how Rick can expect PCT to be considered useful to

people who want to know what goes on inside people’s heads if PCT
cannot provide information about what goes on inside people’s heads
if you understand it properly. If such people believe Rick, they
will look at PCT, learn that it isn’t what they are seeking, and go
on to use other ways of thinking about why people do what they do.
That would be unfortunate, but I think we have seen it happen on
CSGnet more than once over the years.

Nor can I understand why the search for THE controlled variable is

so uniquely important in PCT-based research, since any one person
changes what s/he is controlling from moment to moment and since
most of the time no two people are likely to be controlling the same
variable.

Both of these two positions that Rick has strongly espoused seem to

suggest (at least to those who have not seriously understood PCT)
that though PCT might be useful in some applications, it cannot be
useful in understanding psychology.

It's a puzzle.

Martin
7

[From Rick Marken (2012.12.15.1645)]

Martin Taylor (2012.12.15.14.01)–

MT: I think this is a reasonable place to make a comment about something

that has been bothering me for quite a long time. Rick would like
PCT to be widely understood (correctly, of course) and used, as do
we all, I imagine. But he frequently asserts that the only valid
research that should be done with PCT is to search for “the
controlled variable”, as though everyone controlled the same
variables all the time.

RM: What I am pushing is the idea that research aimed at understanding behavior should be aimed at determining the variables that are being controlled when we see a person doing some behavior of interest, like walking, talking of playing chess. PCT does assume that people do control the same types of variables (intensities, sensations, configurations, transitions…etc). But this is just a hypothesis (based on Bill’s introspection) that can only be tested by doing lots of research and then looking it all over and seeing whether people do, indeed, control the same types of variables and whether these variables are, indeed, hierarchically related.

MT: Furthermore, in this and related threads,

Rick has strongly defended the position that PCT cannot tell you
anything about what happens inside a person’s head (except what
variable s/he is controlling).

RM: That’s news to me. It’s not only controlled perceptions that are in a person’s head (according to PCT); it’s the references for these perceptions and memories of these perceptions, and manipulations of these perceptions in your head, as imaginings, etc. I would hope that we will eventually find ways to study all of the “things in your head” that are posited by PCT.

MT: If you believe that PCT can tell you

something about what happens inside the head, Rick says you are an
S-R theorist, and don’t really understand PCT.

RM: No, I say you are an S-R psychologist if you believe that what is in your head are processes that convert input information into outputs. It’s the information processing view of mind: S-O-R. And PCT shows that it is wrong.

MT:I do not see how Rick can expect PCT to be considered useful to

people who want to know what goes on inside people’s heads if PCT
cannot provide information about what goes on inside people’s heads
if you understand it properly.

RM: I consider PCT useful to people who want to know what is inside people’s heads when they control. I know it will not be considered useful to people who don’t know that people control, the same people who want to know what goes on between S and R. So I’m trying to get those people to understand what control is, how it works and that behavior is control.

MT: If such people believe Rick, they

will look at PCT, learn that it isn’t what they are seeking, and go
on to use other ways of thinking about why people do what they do.
That would be unfortunate, but I think we have seen it happen on
CSGnet more than once over the years.

RM: If people left CSGNet because they learned that PCT wouldn’t help them study people as S-O-R or information processing systems then they were wise to leave because they were right. If they had stayed they would have just gotten madder and madder at PCT, unless they were willing to reorganize their ideas about how behavior works and start studying it in a new way.

MT: Nor can I understand why the search for THE controlled variable is

so uniquely important in PCT-based research, since any one person
changes what s/he is controlling from moment to moment and since
most of the time no two people are likely to be controlling the same
variable.

RM: You can't understand why the search for THE controlled variable is

so uniquely important in PCT-based research because it’s not. PCT research is not about finding THE controlled variable; it is (as I said above) about finding the perceptual variables around which particular examples of behavior are organized. It is also about find how these variables are controlled and why.

MT: Both of these two positions that Rick has strongly espoused seem to

suggest (at least to those who have not seriously understood PCT)
that though PCT might be useful in some applications, it cannot be
useful in understanding psychology.

It's a puzzle.

RM: It’s a puzzle to you only because you are misunderstanding my positions. But I blame myself for any misunderstanding and I’ll try to be more clear about what my position is. But in the mean time maybe you could tell be the kind of research you would like to do that you think I am ruling out.

Great topic Martin; thanks for bringing it up.

Best

Rick

···

–
Richard S. Marken PhD
rsmarken@gmail.com
www.mindreadings.com

8

from David Goldstein
(2012.12.15.2056)

from Rick Marken (2012.12.15.1645)]

What was the person who did the shooting in Newtown Ct controlling?

How can we find out given that he killed himself?

David Goldstein

···

Sent from my iPhone

On Dec 15, 2012, at 7:45 PM, Richard Marken <rsmarken@GMAIL.COM> wrote:

From Rick Marken (2012.12.15.1645)]

9

[From Rick Marken (2012.15.2145)]

David Goldstein (2012.12.15.2056)

Rick Marken (2012.12.15.1645)]

DG: What was the person who did the shooting in Newtown Ct controlling?

RM: Who knows? But when people ask why he did it they are asking what he was controlling for.

DG: How can we find out given that he killed himself?

RM: There’s no way to know now and we would be unlikely to have been able to find out even if he lived.

But who cares? What we can find out now is what the right wing dips who oppose any kind of gun control are controlling for. Whatever it is, a side effect of their controlling is the destruction of the country, ethically, morally and economically.

Best

Rick

···

–
Richard S. Marken PhD
rsmarken@gmail.com
www.mindreadings.com

10

[From Fred Nickols (2012.12.16.0525 AZ)]

David:

So far as I know the only way to find out is to do the test and since the
shooter is dead the test can't be done. All else is speculation.

Best regards,

Fred Nickols
Distance Consulting LLC
www.nickols.us

···

-----Original Message-----
From: Control Systems Group Network (CSGnet)
[mailto:CSGNET@LISTSERV.ILLINOIS.EDU] On Behalf Of D Goldstein
Sent: Saturday, December 15, 2012 7:02 PM
To: CSGNET@LISTSERV.ILLINOIS.EDU
Subject: Re: Information and Control

From David Goldstein
(2012.12.15.2056)

From Rick Marken (2012.12.15.1645)]

What was the person who did the shooting in Newtown Ct controlling?

How can we find out given that he killed himself?

David Goldstein

Sent from my iPhone

On Dec 15, 2012, at 7:45 PM, Richard Marken <rsmarken@GMAIL.COM> > wrote:

> From Rick Marken (2012.12.15.1645)]