Uncertainty, Information and control part I: U, I, and M

[MartinT]

[Martin Taylor 2012.02.23.16.39]

[From Rick Marken (2013.02.23.1330)]

      MT: I really and truly

don’t see why you keep repeating this, when I have so often
pointed out that it isn’t. I wish you would read even the
introductory section of my tutorial, down to the point where
“information” is defined, to see why this statement is so very
wrong.

      RM: That's one long tutorial. Could you just tell me where you

show that information theory is not an open loop model. I
really would like to see that.

It really isn't very far into the long tutorial that this is

explained. And I did try to make the tutorial easy to read without
having to go into the equations. But just for you – and this won’t
be strictly accurate – here’s a two-sentence summary.

However you define probability p, so long as the total probability

of all possibiities is 1.0, Uncertainty – U(system) – is defined
as Sum_over_all_possibilities (p(the
possibility)*log(p(thepossibility)). U is the fundamental quantity
of information theory. Information is a change in Uncertainty.
Information_about_X_consequent_on _Y = U(X|without Y) - U(X|with Y).

Even shorter, Information is a change in uncertainty because of

something else.

Shannon developed Information Theory in the context of open-loop

communication, and it remains useful there. The Receiver’s
uncertainty about what the message would be is changed by the
reception of the (possibly garbled) message. But Shannon’s
theoretical development did not depend on that context. It’s much
more general.

          RM: So I am afraid I will never

understand information theory because all the complex
mathematics (no pun intended) in the world will not
convince me that there is information about variations in
d(t) in the sum d(t) + qo(t).

        MT: And who has said there is? That's not the claim, as you

really ought to have understood by now, given the number of
times I have tried to correct you when you say it is.

      RM: I thought that your whole point has been that there is

information about the disturbance in perception; that that is
how control works (by the system knowing, based on the
information about the disturbance, what output to produce to
counter it). If that’s not your point then, once again, never
mind.

Right. Never mind.

        MT: Mathematics has nothing to do with the issue, anyway.

Let’s say it once again, more simply than I said the same
thing yesterday [Martin Taylor 2013.02.22.13.56] .

*** So long as the reference value for a
control system is constant, the only variable that can
influence the values in the control loop is the
disturbance. Therefore ALL values of signals in the
loop must be functions of the HISTORY OF the
disturbance and the properties of the paths and
functions in the loop.***
Can you see the difference between this and what
you are opposing?

      RM: As long as you add "and the output" after HISTORY OF the

disturbance.

No. I absolutely refuse that amendment. That's the critical point.

You still don’t see the difference between what you are saying and
the actual point. There is NOTHING apart from the HISTORY OF
the disturbance and the fixed properties of the control system that
can affect the current value of the output (and of any other signal
in the whole loop), so long as the reference is a fixed constant
value for all time.

        MT: Why can't you accept that the equations are done using

the conventional control loop analysis, and work only if the
loop is closed? You only had to look at them to know it.

        RM: Just that over-controlling thing. When you present

equations with d on the right and qo on the left its a
disturbance to my desire to see the emphasis on the
importance of understanding control with r on the right and
qi on the left.

Fine. That works, too, but it's not much use when r is permanently

zero, as in the analysis we have been using as an example. If r
varies, the Italic and bold quote above has to be
changed to say that the values are functions of the histories of the
reference and the disturbance.

        But in my experience, knowing all the mathematics of control

theory is no guarantee of understanding PCT (which is the
application of control theory to understanding the behavior
of living things); and, conversely, not knowing the
mathematics is no guarantee of not understanding PCT.

True, but as in most things, being able to take a variety of

viewpoints is useful for deep understanding. My old professor at
Johns Hopkins called it “triangulation”, though that word seems to
have taken on a different meaning nowadays.

        I think one does have to have a reasonably

good quantitative understanding of control theory to
understand PCT, which I believe one can get from doing the
computer modeling.

It takes a lot of modelling to cover the ground occupied by one

mathematical description, but neverhteless the mathematical models
can often be worked out only for simple cases. Because of that, some
of it is best done by modelling, but the problem is that every model
run deals with only one specific situation, one set of parameter
values, for example. How, using only computer modelling, would you
know whether your chosen parameter test set all fell on one side of
a bifurcation point? Models, mathematics, verbal analysis, they all
have their places, with verbal analysis being the least reliable but
the easiest to do.

        But knowing all the mathematics of control

theory in detail does not guarantee an understanding of PCT,
as has been demonstrated on this net (some early
participants in CSGNet discussions were control engineers
who did not get the idea that the behavior of living things
involves the control of input perceptual variables) and in
the works for various manual control psychologists, who
treat the input to the human controller as the cause of the
output, with the reference out in the environment.

All of this and what followed is fair comment. I might not have put

it the same way myself, but I can’t disagree with it.

Martin

[rsmarken]

[From Rick Marken (2013.02.23.1450)]

Martin Taylor (2012.02.23.16.39) –

      RM: That's one long tutorial. Could you just tell me where you

show that information theory is not an open loop model. I
really would like to see that.

MT: It really isn't very far into the long tutorial that this is

explained. And I did try to make the tutorial easy to read without
having to go into the equations. But just for you – and this won’t
be strictly accurate – here’s a two-sentence summary.

However you define probability p, so long as the total probability

of all possibiities is 1.0, Uncertainty – U(system) – is defined
as Sum_over_all_possibilities (p(the
possibility)*log(p(thepossibility)). U is the fundamental quantity
of information theory. Information is a change in Uncertainty.
Information_about_X_consequent_on _Y = U(X|without Y) - U(X|with Y).

Even shorter, Information is a change in uncertainty because of

something else.

RM: I’m sorry. I still don’t see any closed loop; Information goes from a source to the system. If it’s closed loop there should be a link that goes from system back to source, right?

*** MT: So long as the reference value for a
control system is constant, the only variable that can
influence the values in the control loop is the
disturbance. Therefore ALL values of signals in the
loop must be functions of the HISTORY OF the
disturbance and the properties of the paths and
functions in the loop.***

          Can you see the difference between this and what

you are opposing?

      RM: As long as you add "and the output" after HISTORY OF the

disturbance.

MT: No. I absolutely refuse that amendment. That's the critical point.

You still don’t see the difference between what you are saying and
the actual point. There is NOTHING apart from the HISTORY OF
the disturbance and the fixed properties of the control system that
can affect the current value of the output (and of any other signal
in the whole loop), so long as the reference is a fixed constant
value for all time.

RM: Ah, good, we still have stuff to disagree about. The current value of the output is affected by both the disturbance and the output that is currently occurring; the output is affecting the input at the same time and it is affecting itself. That’s what the simultaneous equations of control are about

qo(t) =G (r - p(t))

p(t) = H(d(t) + F(qo(t))

To the extent that history has anything to do with it, it would have to be the history of disturbance and output that affects output.

Best

Rick

···

–
Richard S. Marken PhD
rsmarken@gmail.com
www.mindreadings.com

[amatic]

[From Adam Matic 2012.02.24.1115cet]

(Martin Taylor 2013.02.23.10.58)

Yes.

[…]

I have been learning "Processing" (for which I thank you) by

creating just such an experiment (not a person with and without
glasses, though). It’s a simple tracking experiment with the target
and cursor separated by various distances. The basic form of it runs
right now, but I have to learn about files for data collection and
various other stuff before I let it out into the wild.

AM:

Great. I’m looking forward to seeing it.

Thank you for the extensive answer. I’m confused about some other aspects of your analysis. I agree that your introduction to information theory is perhaps overly technical, maybe only too abstract. I think it would help if you would start the tutorial with explaining what information, uncertainty, entropy, information source, receiver and other terms mean specifically in control system analysis.

This has probably been mentioned before, I don’t know for sure, but would you consider the following paragraph as using information theory (B:CP, chapter 3, on neural currents)?

“As a basic measure of nervous system activity, therefore, I choose to use neural current, defined as the number of impulses passing trough a cross section of all parallel redundant fibers in a given bundle per unit time. The appropriateness of this measure depends on the maximum neural current normally expected to occur in a given bundle of fibers. If the maximum in a bundle of 50 fibers is 200 impulses per second in each fiber, the maximum neural current will be 10,000 impulses per second, and statistical variations will not be important at any level of neural current in proportion to the whole normal range of operation (they will be roughly 1 percent of the maximum, or less).”

···

---

Another thing I’m having problems with is the mathematical analysis of error propagation trough the loop. I don’t see why it’s necessary to calculate it. Should we not compare control systems?

If X is some amount of random error in measurement, then px=Ki*(qi+X). Let’s say we have two equal control loops except one has a really accurate and precise input function p0, and the other an error-prone, px. We can write

p0 = Ki*qi.

px = Ki (qi +/- X)

= Kiqi +/- XKi

---

px = p0 +/- X*Ki

So, if all the other elements of those two loops are the same, we can see they will behave differently because they will be controlling different variables. Well, only in the case that they are fast enough to react to noise. If they are slow, the noise effects will cancel out, and their behavior will be the same.

MT: The point of adding this last paragraph is to make clear that the

degree of control is limited by the point in the loop that has the
lowest precision. It doesn’t matter whether the restriction is in
the perceptual system, the environmental feedback path, or anywhere
else in the loop.

AM:

This seems like a central point.

How do we evaluate the degree of control, is there a mathematical definition?

Adam

[MartinT]

[Martin Taylor 2013.02.24.09.56]

[From Adam Matic 2012.02.24.1115cet]

      Thank you for the extensive answer. I'm confused about some

other aspects of your analysis. I agree that your introduction
to information theory is perhaps overly technical, maybe only
too abstract. I think it would help if you would start the
tutorial with explaining what information, uncertainty,
entropy, information source, receiver and other terms mean
specifically in control system analysis.

I completely rewrote that tutorial four or five time, in an effort

not to elicit this kind of comment. I guess I should have tried six
or seven times

The main problem I found was that many of these terms have everyday

meanings, and I didn’t want to get involved with comments along the
lines of “That’s not what ‘uncertainty’ means to me”. The problem is
a bit like the problem I find happens quite often when I try to
introduce someone to PCT, when they can think of “perception” only
as something of which they are conscious, whereas in PCT it is an
abstract signal value at a particular place in a control loop.

The other point you make is what these terms mean specifically in

control systems. They don’t mean anything different in control
systems from what they mean in other contexts, but just as
“perception” means something specific in a particular control loop,
so the measures of information theory will mean something specific
in the context of a particular element of a control system.
Sometimes it will be in the original Shannon context of the
precision with which a message over a noisy transmission line can be
received. That’s what we deal with when we ask about the accuracy of
perception. But sometimes it isn’t, as when we ask about the mutual
information between the output and the disturbance in a loop that
has precisely accurate paths with no noise. With a precise loop the
mutual information depends on the effective transport lag, the
integrator limit, and the speed with which the disturbance varies.
There’s no “transmitter-receiver” link in that context.

I'll try to clarify some of this stuff in Part 2 of the tutorial,

but that may take as long to write as the unfinished Part 1 did. The
concepts are not too hard if you come at it from the right
background, but I don’t think many on CSGnet have that background,
so the problem is to try to furnish it without getting people lost
in a lot of unfamiliar ways of thinking – much like the problem in
trying to get people to think of “control” without thinking of
totalitarianism.

      This has probably been mentioned before, I don't know for

sure, but would you consider the following paragraph as using
information theory (B:CP, chapter 3, on neural currents)?

      "As a basic measure of nervous system activity, therefore,

I choose to use neural current, defined as the number of
impulses passing trough a cross section of all parallel
redundant fibers in a given bundle per unit time. The
appropriateness of this measure depends on the maximum neural
current normally expected to occur in a given bundle of
fibers. If the maximum in a bundle of 50 fibers is 200
impulses per second in each fiber, the maximum neural current
will be 10,000 impulses per second, and statistical variations
will not be important at any level of neural current in
proportion to the whole normal range of operation (they will
be roughly 1 percent of the maximum, or less)."

---

No, I wouldn't. One might analyze the situation as a Shannon-type

noisy communication channel, and there was a recent CSGnet
discussion of a paper in which they did exactly that. But the
paragraph itself essentially says “1% error doesn’t matter, so we
will call it zero”. In fact, when you consider the masses of
parallel neurons in many sensory pathways, the error is probably
much less than this paragraph suggests. My control analyses, at
least so far, deal only with systems that are noiseless apart from
well identified parts of the loop, such as between the CEV and the
perceptual signal – the issue we got involved with recently.

      Another thing I'm having problems with is the mathematical

analysis of error propagation trough the loop. I don’t see why
it’s necessary to calculate it. Should we not compare control
systems?

I guess I don't know what distinction you are making here. Would an

analysis of error propagation in different loops not be a comparison
of those control systems?

      If X is some amount of random error in measurement, then

px=Ki*(qi+X). Let’s say we have two equal control loops except
one has a really accurate and precise input function p0, and
the other an error-prone, px. We can write

p0 = Ki*qi.

px = Ki (qi +/- X)

= Kiqi +/- XKi

---

px = p0 +/- X*Ki

      So, if all the other elements of those two loops are the

same, we can see they will behave differently because they
will be controlling different variables. Well, only in the
case that they are fast enough to react to noise. If they are
slow, the noise effects will cancel out, and their behavior
will be the same.

You describe the two extremes of the problem. The question is how to

mathematize the middle ground, and I believe that is one place where
informational measures can be useful. My “Alice’s weights” example
suggests one way of thinking about this, if instead of a hard
rectangular limit on the resolution of the digital tilt-meter we
substitute a continuous Gaussian distribution. (I may use this
notion in Part 2).

        MT: The point of adding

this last paragraph is to make clear that the degree of
control is limited by the point in the loop that has the
lowest precision. It doesn’t matter whether the restriction
is in the perceptual system, the environmental feedback
path, or anywhere else in the loop.

AM:

This seems like a central point.

      How do we evaluate the degree of control, is there a

mathematical definition?

Conventionally (i.e. by Rick and Bill), you measure how the CEV

would vary in the absence of control as compared to its variation
when the corresponding perception is controlled. The “control ratio”
is the ratio of the two variance or RMS values (it’s not consistent,
but usually it is stated which measure is used). The equivalent in
Informational analysis terms is the difference in uncertainty of the
CEV between controlled and uncontrolled states.

Here's an example from my "Processing" experiment, using myself as a

subject yesterday (I have progressed to the point of being able to
store my data in a form suitable for a LibreOffice spreadsheet). The
display has two small horizontally oriented ellipses, a purple
target one and a green “cursor” one. They move vertically, and the
subject’s task is to keep the green one level with the purple one.
The movements were mostly very slow.

The experimental "independent variable" is the lateral separation

between the ellipses. 4.5 bits of control means that the RMS error
without control is 26 times the error when controlling. A
performance of 3.5 bits means a ratio of 11.3. The plotted points
are an average of three runs each. They don’t mean too much, because
the range between best and worst at a given pixel separation
averages 0.6 bits. All the same, the ranges of the two closes
separations do not overlap the ranges of the two widest, so I guess
that does lend the slope a little credibility.

To save you having to compute the lateral separations from their

logarithmic values, they are 20, 40, 70, 220, 470 and 820 pixels
between the centres of the ellipses. At the lowest separation, the
ellipses are just touching.

![CtrlPerfBits2.jpg|522x408](upload://e7yQoT3b3FyEPfoRkvtvd3QjYLN.jpeg)

You should take the actual values in this figure with a large grain

of salt, since the data were taken while I was debugging the
program. However, they do show the anticipated reduction of the
control performance with increasing lateral separation between
cursor and target.

At the time I took these data, my precision of mouse movement was

limited to 2 pixels, which would have allowed around 8 bits of
control, but my personal ability to move the mouse accurately
towards and away from me was less, at least while the target was
moving. So we have the question I posed in my last response to a
message from you “I wonder whether it would be possible to tease out
where in the loop the limit lies?” [Martin Taylor 2013.02.23.10.58].
At least this figure suggests that part of it in this experiment is
in the accuracy of perceiving the relative height of the cursor and
target.

Martin

[rupert]

[Rupert Young 2013.02.20 09.00]

···

[Martin Taylor 2013.01.01.12.40]

              > As you can see by the date stamp, I started this

on New Year’s Day, expecting it to be quite short and
to be finished that day. I was wrong on both
counts…

              > [Martin Taylor 2013.02.20.10.22]

              > I am following Shannon's 1949 use of the term, as

a reduction (a differential or derivative quantity) in
what Shannon called “Entropy” and I call
“Uncertainty”…

I have read through your tutorial and also http://en.wikipedia.org/wiki/Information_theory .
The definition of information theory in the latter seems
pretty consistent with my existing understanding, and it
describes information theory as “involving the
quantification of information” and is about “compressing
data and on reliably storing and communicating data” and
states “A key measure of information is known as entropy,
which is usually expressed by the average number of bits
needed to store or communicate one symbol in a message.”

          Essentially it is about communication systems

and how information can represented within transmission
signals. That is, the transmitted signals contain
information that represents something else. This also
seems to be implied by the concept of uncertainty
(entropy), in that there is a correct value that the
signal should be, or represents, and uncertainty measures
the deviation from that “correct” value.

          It should be possible to discuss the validity,

or otherwise, of information theory with respect to living
systems at the conceptual level without reference to the
mathematics; at least initially, and can be accepted or
rejected as such.

          The argument, seems to me, to be the

equivalent of the Representation debate in the Artificial
Intelligence community. That is, that neural signals
“represent” something outside of their own activation
levels. In other words they “model” or contain
“information” about the external world. The problem, the
argument goes, that AI needs to address is how can the
world be modelled and what computational processes can be
employed to transform one representational state to
another.

          This was the approach initiated by Alan Turing

in the '50’s and has pretty much held sway ever since.
Well, he may well be known as the “the father of computer
science”, but he is the grim reaper of AI as far as I am
concerned as his computational approach set the community
off on entirely the wrong track and has led to decades of
wasted research.

          In recent years, at least, there has been some

recognition of this with a more bottom up approach and a
move to Artificial Life research. However, there still
seems to be an overriding assumption that neural systems
model, or contain information about, the world, but it is
just an assumption, made from the viewpoint of complex
beings (humans) who are misguided by their perception of
the world around them that gives them a sense that metal
states represent the world.

          Luckily we don't need to refer to humans, with

all their complexity, to address the issue of
representation and information theory, we can look at very
simple creatures such as the zooplankton mentioned
elsewhere. For if we can show that information theory is
or is not applicable to the zooplankton then we can safely
assume the same for all other living systems.

          Martin, where is "information" in a simple

system like the zooplankton? From my viewpoint I do not
see how information is relevant to the zooplankton. The
neural signals do not represent anything, apart from
themselves; they do not contain messages or information,
about which there is uncertainty as there is no “correct”
value about which to be uncertain. The only reason for the
signals being the values they are is in order to achieve
control so that the organism can survive within its
environment and pass on its genes. There is no need for
information.

          In fact the beauty, and radical nature, of PCT

is that it shows that control systems (and any nervous
systems) are successful precisely without any need for
information and representation.

          The danger, which I think Rick has alluded to,

is if you start introducing other theories, which are not
really relevant as if they are then people may start using
that approach to model control systems which would lead to
a completely incorrect divergent path of research, as has
been the case with AI.

          Although I am open-minded to pretty much

anything I am still not able to see the applicability or
necessity of information theory with respect PCT,

Regards,
Rupert Young

[MartinT]

[Martin Taylor 2013.02.24.16.26]

[Rupert Young 2013.02.20 09.00]

There's a confusing issue of nomenclature in information theory

discussions. I wish people had kept to Shannon’s usage, which was
clear and easily understood. You quotes from the Wikipedia entry
make “information” mean what Shannon called “entropy” and I
(following Garner) call “uncertainty”. Shannon used “information”
for “change in uncertainty”, and that’s how I think the term should
be used.

Here's what I wrote in a recent NATO report:

------begin quote-------

    The relation between

Shannon’s
transmitter-receiver language and Garner’s observer-observed
language may be
encapsulated in Table D.1:

    Table D.1:

Mapping between Shannon’s communication terminology and the
language of
observation

Shannon communication

Natural Observation

Transmitter

Observable World

Receiver

Observer

Channel

              Sensors, displays,

and processing

              Transmitted

Message

Sensor data

Received Message

              Observer’s

perception and understanding

              Possible distinct

messages

              Possible distinct

states of the observed world

Entropy

Uncertainty

Information

Information

      “Information”

is a term
widely used, and, as a technical construct, almost as widely
misunderstood. The
problem is that the same word is used to mean several
different but related
concepts, three in particular within the Shannon conceptual
structure

      Information

gained from a
message, which is the change in uncertainty about the
transmitter’s intent as a
consequence of receipt of the message; one bit of information
equates to a
doubling of precision.

      Information

about the state of
the transmitter, which is obtained from consideration of all
the messages so
far received from that transmitter;

      Information

that could yet be
obtained about the state of the transmitter, which is the
remaining uncertainty
about the transmitter’s state. Sometimes this is misleadingly
called
“Information in” the transmitter. This usage is the one that
causes most
misunderstanding, as there is no sense in which “information”
resides in
anything waiting to get out. It will not be used in this
chapter. However, a
legitimate form of statement is “information potentially
available about” the
transmitter, and occasionally a form such as this may be used
in what follows.

---------end quote------

That's just one potential use of information theory. It's the

context in which Shannon created the theory, so it has a long
history. But it is VERY far from being all that information theory
is about.

I agree that uncertainty and information are always about something.

That is something also that Shannon said, and that has been widely
misrepresented. It has been mixed up with something else Shannon
said, that when measuring the information transmission capacity of a
communication channel, the measures must be independent of the
meaning of the messages to be passed through the channel. That’s
quite different from what a lot of people take as dogma in
information theory, that information has nothing to do with meaning.

I wish that were true, but so far I have not found it possible.

Maybe someone who has more facility with language could do it, but
whenever I try, I find there are too many ambiguities left hanging.

Red Flag! That pair of words "contain information" always is a

warning that something is going conceptually off the rails.

I don't see it so much an assumption as a way of regarding the

analysis. If from the state of X you can deduce something about the
state of Y, it seems harmless to say that in some way X “represents”
Y (and as the maths tell you, vice-versa).

Nowhere. Information is never "in" any system.

I don't know whether to say "true" or to point out that those

signals have values that depend on what is happening in the
environment outside the organism. That explicitly does mean that
there is positive mutual information between the environment and the
signals.

The mind boggles at this. The teleological statement that the

signals somehow manipulate themselves in order to achieve control,
and do this with no reference to the state of the outside world,
seems really weird. Even if the output required for control were
somehow achieved through ESP, the informational relations would be
the same. The loop HAS to be closed from input through the organism
to output and back to input, or you don’t have control.

Obviously, I strongly disagree about "information". If the

perceptual signal did not vary at least moderately consistently with
the state of something in the environment, how would the output be
able consistently to move the perception in the direction of
decreasing error? As for “representation”, I suppose whether I
disagree or not would depend on what you mean by the term. Clearly
you means something other than “varies consistently with”, but I
don’t have any other possible meaning in my mind.

I don't think any paths of research are "incorrect". They may turn

out to be fruitless, but you can’t say whether that will be the case
without trying them. They may not be the way you personally would
choose to do the research, but someone who did choose that path
would be equally legitimate in calling your path “incorrect”. I
would say you would both be wrong.
Neither can Rick. I agree about “necessity”. There’s no need of
information theory in PCT. PCT has been getting on well without it
for decades. There’s no need for the internet, but communication is
easier because it is available. You don’t need a saw or an axe to
cut down a tree, since lots of string and months of work would
enable you to do it by rubbing the string back and forth along the
groove you make. But it’s quicker to use a saw or an axe. Likewise,
if it is quicker or helpful in understanding some aspect of a
control system by using information theory, then why not use it? If
you can’t see how to use it, the situation is much the same as
wanting to cut down the tree when you don’t know how to use a saw
(in fact, I avoid using a chain-saw for just that reason, though I
know it would be quicker).

I think your objections come from that anti-Shannon Wikipedia entry.

If you don’t try to verbalize and just use the math, I don’t think
you would come up with these objections. Now would you (probably),
if you just use the Shannon definition of “information” as “change
in uncertainty”. When you start mixing up ideas of some kind of
representation other than consistent co-variation, you are going
well beyond what I think of a classical information theory. It’s a
snare and a delusion.

Martin

···

[Martin Taylor 2013.01.01.12.40]

                > As you can see by the date stamp, I started

this on New Year’s Day, expecting it to be quite
short and to be finished that day. I was wrong on
both counts…

                > [Martin Taylor 2013.02.20.10.22]

                > I am following Shannon's 1949 use of the term,

as a reduction (a differential or derivative
quantity) in what Shannon called “Entropy” and I
call “Uncertainty”…

I have read through your tutorial and also http://en.wikipedia.org/wiki/Information_theory .
The definition of information theory in the latter seems
pretty consistent with my existing understanding, and it
describes information theory as “involving the
quantification of information” and is about “compressing
data and on reliably storing and communicating data” and
states “A key measure of information is known as
entropy, which is usually expressed by the average
number of bits needed to store or communicate one symbol
in a message.”

            Essentially it is about communication

systems and how information can represented within
transmission signals.

            That is, the transmitted signals contain

information that represents something else. This also
seems to be implied by the concept of uncertainty
(entropy), in that there is a correct value that the
signal should be, or represents, and uncertainty
measures the deviation from that “correct” value.

            It should be possible to discuss the

validity, or otherwise, of information theory with
respect to living systems at the conceptual level
without reference to the mathematics; at least
initially, and can be accepted or rejected as such.

            The argument, seems to me, to be the

equivalent of the Representation debate in the
Artificial Intelligence community. That is, that neural
signals “represent” something outside of their own
activation levels. In other words they “model” or
contain “information” about the external world.

          ... However, there still seems to

be an overriding assumption that neural systems model, or
contain information about, the world, but it is just an
assumption, made from the viewpoint of complex beings
(humans) who are misguided by their perception of the
world around them that gives them a sense that metal
states represent the world.

            Luckily we don't need to refer to humans,

with all their complexity, to address the issue of
representation and information theory, we can look at
very simple creatures such as the zooplankton mentioned
elsewhere. For if we can show that information theory is
or is not applicable to the zooplankton then we can
safely assume the same for all other living systems.

            Martin, where is "information" in a simple

system like the zooplankton?

            From my viewpoint I do not see how

information is relevant to the zooplankton. The neural
signals do not represent anything, apart from
themselves; they do not contain messages or information,
about which there is uncertainty as there is no
“correct” value about which to be uncertain.

            The only reason for the signals being the

values they are is in order to achieve control so that
the organism can survive within its environment and pass
on its genes. There is no need for information.

            In fact the beauty, and radical nature, of

PCT is that it shows that control systems (and any
nervous systems) are successful precisely without any
need for information and representation.

            The danger, which I think Rick has alluded

to, is if you start introducing other theories, which
are not really relevant as if they are then people may
start using that approach to model control systems which
would lead to a completely incorrect divergent path of
research, as has been the case with AI.

            Although I am open-minded to pretty much

anything I am still not able to see the applicability or
necessity of information theory with respect PCT,

[MartinT]

[Martin Taylor 2013.02.24.17.18]

···

Correctio of a typo…

[Martin Taylor 2013.02.24.16.26] to

[Rupert Young 2013.02.20 09.00]

  I think your objections come from that anti-Shannon Wikipedia

entry. If you don’t try to verbalize and just use the math, I
don’t think you would come up with these objections. Nor [not
“Now”] would you (probably), if you just use the Shannon
definition of “information” as “change in uncertainty”. When you
start mixing up ideas of some kind of representation other than
consistent co-variation, you are going well beyond what I think of
a classical information theory. It’s a snare and a delusion.

  Martin

[rupert]

[Rupert Young 2013.02.25 23.30]

(Martin Taylor 2013.02.24.16.26)

There's a confusing issue of nomenclature in information theory discussions. I wish people had kept to Shannon's usage, which was clear and easily understood. You quotes from the Wikipedia entry make "information" mean what Shannon called "entropy" and I (following Garner) call "uncertainty". Shannon used "information" for "change in uncertainty", and that's how I think the term should be used.

If your use of the the word "information" doesn't correspond to the common understanding, or to that used by practitioners of IF than maybe you should use another word. In your table you have information on both sides, distinct from uncertainty, yet you define information in terms of uncertainty. So I hope you'll excuse me if I am having trouble understanding what it means, and its relevance to PCT.
But, if we go with your definition, "Information gained from a message, which is the change in uncertainty about the transmitter’s intent as a consequence of receipt of the message; one bit of information equates to a doubling of precision", this still seems to me to be talking about a communication system, in that there is a transmitting agent who is "putting" meaning, intention, within a transmitted message; that is, it "represents" something.
Furthermore communication systems are designed to be that way, to transmit meaning. Natural control systems are not communication systems, there is no designer (external agent), there is no intention, or representation, no meaning (apart from that inferred by an external observer).
Even if we take information just as "change in uncertainty", what is there, in natural control systems, about which we are uncertain? I still infer from this the concept of something being regarded as "correct".

It should be possible to discuss the validity, or otherwise, of information theory with respect to living systems at the conceptual level without reference to the mathematics; at least initially, and can be accepted or rejected as such.

I wish that were true, but so far I have not found it possible. Maybe someone who has more facility with language could do it, but whenever I try, I find there are too many ambiguities left hanging.

Well, maths is pretty meaningless without any conceptual framework. If we can't get to the point where we see that the IF is conceptually relevant to PCT then there's no point going further, as far as I am concerned. We have to deal with our conceptual assumptions first.

The argument, seems to me, to be the equivalent of the Representation debate in the Artificial Intelligence community. That is, that neural signals "represent" something outside of their own activation levels. In other words they "model" or contain "information" about the external world.

Red Flag! That pair of words "contain information" always is a warning that something is going conceptually off the rails.

What's the difference between "contain information" and representation, which you support later?

... However, there still seems to be an overriding assumption that neural systems model, or contain information about, the world, but it is just an assumption, made from the viewpoint of complex beings (humans) who are misguided by their perception of the world around them that gives them a sense that metal states represent the world.

I don't see it so much an assumption as a way of regarding the analysis. If from the state of X you can deduce something about the state of Y, it seems harmless to say that in some way X "represents" Y (and as the maths tell you, vice-versa).

Well, coming from an AI background this is not at all harmless, but the most fundamental question. There is a major philosophical difference between saying X is (causally?) related to Y and X "represents" Y. The latter, I would say, can lead to an invalid view of how living systems work, and lead to invalid research.

Martin, where is "information" in a simple system like the zooplankton?

Nowhere. Information is never "in" any system.

Ok, so how does, or can, IF be useful in the context of studying or understanding the zooplankton?

From my viewpoint I do not see how information is relevant to the zooplankton. The neural signals do not represent anything, apart from themselves; they do not contain messages or information, about which there is uncertainty as there is no "correct" value about which to be uncertain.

I don't know whether to say "true" or to point out that those signals have values that depend on what is happening in the environment outside the organism. That explicitly does mean that there is positive mutual information between the environment and the signals.

Ok, but where does uncertainty come in?

The only reason for the signals being the values they are is in order to achieve control so that the organism can survive within its environment and pass on its genes. There is no need for information.

The mind boggles at this. The teleological statement that the signals somehow manipulate themselves in order to achieve control, and do this with no reference to the state of the outside world, seems really weird.

Who said that, I didn't?

In fact the beauty, and radical nature, of PCT is that it shows that control systems (and any nervous systems) are successful precisely without any need for information and representation.

Obviously, I strongly disagree about "information". If the perceptual signal did not vary at least moderately consistently with the state of something in the environment, how would the output be able consistently to move the perception in the direction of decreasing error?

This last point sounds valid at first glance, but I think there are some subtleties here that might point to the differences in our viewpoints. Your point seems to indicate that you see a strong deterministic link or relationship between the external and internal signals. I see them as very independent entities that may sometimes be related, but other times not. For example, if you are perceiving lights that are too bright (your perceptual signal is "related" to the external light signal), but then you put your hands over your eyes the perceptual signal changes. The external signal remains the same, yet there is no longer a relationship.

The danger, which I think Rick has alluded to, is if you start introducing other theories, which are not really relevant as if they are then people may start using that approach to model control systems which would lead to a completely incorrect divergent path of research, as has been the case with AI.

I don't think any paths of research are "incorrect". They may turn out to be fruitless, but you can't say whether that will be the case without trying them. They may not be the way you personally would choose to do the research, but someone who did choose that path would be equally legitimate in calling your path "incorrect". I would say you would both be wrong.

Incorrect, fruitless; potato, potatoe. The issue is about paradigm creep. If one introduces concepts and processes that aren't particularly relevant from different paradigms then it is more or less inevitable that we end up doing something that is some bastardised form of PCT. Rick may seem sensitive and over-zealous in his defense of PCT, but I think he is quite right to protect the "purity" of the theory against being tainted in this way. I'm sure you can understand that.

Likewise, if it is quicker or helpful in understanding some aspect of a control system by using information theory, then why not use it?

Fine, but I am still at a loss to see how it is helpful, or even that the concepts of IF are relevant to PCT. From what you say it seems that the key is explaining how the concept of uncertainty applies to PCT or to the variables in the loop.
Regards,
Rupert

[amatic]

[From Adam Matic, 2013.02.26.0200cet]

(Martin Taylor 2013.02.24.09.56)

I completely rewrote that tutorial four or five time, in an effort not to elicit this kind of comment. I guess I should have tried six or seven times

The main problem I found was that many of these terms have everyday meanings, and I didn't want to get involved with comments along the lines of "That's not what 'uncertainty' means to me".

AM:
Shannon's choice of terms doesn't make the job of explaining IT very
easy, I guess. Information in information theory has nothing to do
with the everyday meaning of information. It's also not defined
properly. It is described mathematically, but a proper definition
would be a bit wider than 'a change in uncertainty', where uncertainty
is also a fuzzy concept.
Nevertheless, the theory proved to be useful in a wide range of fields.

Sometimes it will be in the original Shannon context of the precision with which a message over a noisy transmission line can be received. That's what we deal with when we ask about the accuracy of perception.

AM: That is the kind of "meaning of IT in a control loop" I had in
mind. I can understand it when explained that way.

By the way, is it precision or accuracy?

But sometimes it isn't, as when we ask about the mutual information between the output and the disturbance in a loop that has precisely accurate paths with no noise. With a precise loop the mutual information depends on the effective transport lag, the integrator limit, and the speed with which the disturbance varies. There's no "transmitter-receiver" link in that context.

[...]

My control analyses, at least so far, deal only with systems that are noiseless apart from well identified parts of the loop, such as between the CEV and the perceptual signal -- the issue we got involved with recently.

AM:
I don't get the concept of 'mutual information' and how it depends on
the other elements you mention.
I'll wait for Tutorial part 2 for detail.

AM: Another thing I'm having problems with is the mathematical analysis of error propagation trough the loop. I don't see why it's necessary to calculate it. Should we not compare control systems?

I guess I don't know what distinction you are making here. Would an analysis of error propagation in different loops not be a comparison of those control systems?

AM: What I'm saying is, from inside the loop, the effects of noise on
p are indistinguishable from effects of disturbances on p. There is
already a whole analysis of effects of disturbances on the loop. No
need to repeat it.
What we can do is compare equivalent (apart from the perceptual
function) loops. Maybe a visual and an auditory tracking task, with
equal output functions. We could answer a question such as "what is
the range of sound frequency that produces equaly good control as a
range of distances of 500 pixels". Does that make sense in the context
of information theory?

And there are probably other kinds of analyzing I don't know of. I'm
hoping to see them in Tutorial pt 2.

Here's an example from my "Processing" experiment, using myself as a subject yesterday (I have progressed to the point of being able to store my data in a form suitable for a LibreOffice spreadsheet). The display has two small horizontally oriented ellipses, a purple target one and a green "cursor" one. They move vertically, and the subject's task is to keep the green one level with the purple one. The movements were mostly very slow.

The experimental "independent variable" is the lateral separation between the ellipses. 4.5 bits of control means that the RMS error without control is 26 times the error when controlling. A performance of 3.5 bits means a ratio of 11.3. The plotted points are an average of three runs each. They don't mean too much, because the range between best and worst at a given pixel separation averages 0.6 bits. All the same, the ranges of the two closes separations do not overlap the ranges of the two widest, so I guess that does lend the slope a little credibility.
To save you having to compute the lateral separations from their logarithmic values, they are 20, 40, 70, 220, 470 and 820 pixels between the centres of the ellipses. At the lowest separation, the ellipses are just touching.

AM:
So, when the ellipses are laterally further apart, control is worse.
What does that tell us? How is that interpreted from an IT
perspective?

Adam

[chadtgreen]

[Chad Green 2013.02.26.12.26 EST]

RY: The argument, seems to me, to be the equivalent of the Representation
debate in the Artificial Intelligence community. That is, that neural
signals "represent" something outside of their own activation levels.
In other words they "model" or contain "information" about the
external world.

MT: Red Flag! That pair of words "contain information" always is a warning
that something is going conceptually off the rails.

CG: Martin, what are your thoughts on Tononi's integrated information theory (IIT)? For example, he posits the following:

"Information that is not integrated, I have argued, is not associated with experience, and thus does not really exist as such: it can only be given a vicarious existence by a conscious observer who exploits it to achieve certain discriminations within his main complex. Indeed, the same 'information' may produce very different consequences in different observers, so it only exists through them but not in and of itself."

Source: http://www.biolbull.org/content/215/3/216.full

Chad T. Green, PMP
Program Analyst
Loudoun County Public Schools
21000 Education Court
Ashburn, VA 20148
Voice: 571-252-1486
Fax: 571-252-1633
Web: http://cmsweb1.loudoun.k12.va.us/50910052783559/site/default.asp

There are no great organizations, just great workgroups.
-- Results from a study of 80,000 managers by The Gallup Organization

[MartinT]

[Martin Taylor 2103.02.25.23.20]

(Response to
[(nightowl) From Adam Matic, 2013.02.26.0200cet] at the end of this

message).

If I have "information" about something, my uncertainty about it has

been reduced. How is this different from the “common understanding”?
The problem I have is with people who are talking technically, and
treat “information in” something as though it was some kind of
phlogistonic variable waiting to get out.

Motion is change of location. Information is change of uncertainty.

It’s easy to understand that location and motion are related but
different. Nobody uses one word when the other is appropriate. Why
do it for information and uncertainty, which are equally clearly
distinct?

It really is most unfortunate that the two terms have become mixed

up, when Shannon was so clear about their difference. Words
shouldn’t matter, but they do when the same word is used to
represent two different things, in this case a measure and its
derivative or differential. That leads only to confusion. Would you
approve if people used “velocity” indiscriminately to mean
“location” as well as “speed” and perhaps “acceleration”?

That was Shannon's area, which was indeed about communication

systems. But I still don’t see how your idea of “representation”
differs from mine – that to “represent” something, the
representation must change when the something changes and not when
the something doesn’t change.

Are you trying to contradict me? Because if you are, you are not

succeeding.

I fail to see the necessity of something being "correct" in order

for there to be a change in uncertainty about it. Suppose for some
unknown reason you had a bet that the first man you see tomorrow
will be wearing brown shoes. You may believe that the probability of
the first man wearing black is about 0.5, of wearing brown is about
0.25, and of wearing some other colour is about 0.25. You have 1.5
bits of uncertainty about it. Now you conspire with a friend to
arrive at your house very early in the morning wearing brown shoes.
Your probability that the first man you see will have brown shoes is
now about 0.9, and “other” is about 0.1. Your uncertainty is now
just under half a bit. You have gained just over 1 bit of
information. Yet at this point there is no “correct” fact as to
whether the first man you see tomorrow will be wearing brown shoes.
Tomorrow, your uncertainty will change again, probably to near zero
(unless you can’t tell whether the shoes are black shoes brown from
mud or brown shoes blacked with soot), as soon as you see your first
man of the day.

When you are dealing with control, all you need is to be "correct"

enough that your actions do influence your perceptions sufficiently
consistently to allow you to bring them within tolerance bounds of
their reference values. Since we cannot know what is truly “out
there”, the ability to control precisely is our best bet for
correctness. We have no other “ground truth”.

There's a strong conceptual framework. Why do you suggest there

isn’t? Granted, there is disagreement about the philosophical status
of “probability”, but in practice the different ways of approaching
it usually come to the same or very nearly the same result. Once you
have probability, the rest is rock solid.

What would you want? Doesn't the concrete example of Alice's weights

that I gave in response to Adam show where the information theoretic
concepts have a place? Would it not be conceptually helpful to
mention that if you can’t see clearly where something is, it’s going
to difficult to control its location precisely? I can’t really be
very helpful here if I don’t know what would make IT feel
conceptually relevant in your mind. It is highly relevant in mine,
but that’s just me.

"Contain information" is, to me, a meaningless pair of words, for

reasons I stated. “Representation” isn’t meaningless to me, but I am
sure that the meaning I have for the word is not the meaning you
intend to convey when you use it.

OK. Please explain the difference, and what you could possibly mean

by “X represents Y” other than that the state of X is in some way
related to the state of Y. I cannot think of any other way it could
be. You obviously can.

Do you mean Information Theory when you say IF? You used IF several

times, and it brought me up short each time I ran into it.

In the fact that the states of the environment and the internal

states of the zooplankton are variable, and in some respects
co-variable.

OK. Rephrase what you did say, because I thought I "represented" you

quite precisely.

Why? There must be consistency, or there could be no control. There

doesn’t have to be deterministic invariant consistency. Control
systems can deal with variation in loop properties if the variation
is slow enough compared with the speed of control. Control is lost
if the loop properties or disturbances change too fast.

The external signal does not remain the same in that case. The

external signal is only what the sensor systems report. The eyes
don’t get the same light, so the external signal has changed. But
the perception of brightness has a very variable relation to the
amount of light entering the eye. It’s not deterministic at all,
though usually the sign of a change in perceived brightness is the
same as the sign of a change in light intensity.

There's a more important point here. Think about the mantra

“correlation does not imply causation”. There’s a similar mantra for
information: “Positive mutual information does not imply
communication”. Just as X may be correlated with Y because they are
both subject to a common influence, so also may there be positive
mutual information between X and Y because they are subject to a
common influence. In the case of PCT, if control is good, the output
influence on the CEV has a high mutual information with the
disturbance, not because the disturbance communicates with the
output – indeed, the current value of the disturbance has no
influence at all on the current value of the output in any physical
control system. The high mutual information value is because of the
fact that the properties of the control system retain effects of
past values of the disturbance. In the case of the leaky integrator,
it’s just the sum of past influences, with the influence decreasing
into the more distant past. Recent past values of the disturbance do
influence its present value, so output and disturbance have a
positive mutual information (and a negative correlation) because
they are both influenced by the same set of past values.

I suppose I care less about the form of PCT than I do about

scientific correctness. PCT isn’t (or shouldn’t be) a religion in
which something is incorrect if the Pope says it is incorrect. The
test is consistency with all the observations of the world, in
whatever context. If PCT were seen to violate the laws of
thermodynamics, I wouldn’t hesitate to “bastardise” it. But it
doesn’t. In fact, I have argued strongly that PCT is required by
the laws of thermodynamics. But the specific form of HPCT built by
Bill Powers isn’t. Nor are there (or at least nor should there be)
restrictions by fiat against trying different way of analysing PCT
structures.

Yes, I understand it in the same way as I understand the control

processes of the officers of the Spanish Inquisition. However, I
fail to see how any particular method of analysis can threaten the
purity of the religion in this case, unless, as in the case of the
Inquisition, it is the simple act of enquiry that is threatening.

I hope that some day you will see how it is helpful, or that I will

find that it isn’t. Either way, our understanding of science will
have progressed.

···

[Rupert Young 2013.02.25 23.30]

(Martin Taylor 2013.02.24.16.26)

                There's a confusing issue of nomenclature in

information theory discussions. I wish people had
kept to Shannon’s usage, which was clear and easily
understood. You quotes from the Wikipedia entry make
“information” mean what Shannon called “entropy” and
I (following Garner) call “uncertainty”. Shannon
used “information” for “change in uncertainty”, and
that’s how I think the term should be used.

            If your use of the the word "information"

doesn’t correspond to the common understanding, or to
that used by practitioners of IF than maybe you should
use another word. In your table you have information on
both sides, distinct from uncertainty, yet you define
information in terms of uncertainty. So I hope you’ll
excuse me if I am having trouble understanding what it
means, and its relevance to PCT.

            But, if we go with your definition,

“Information gained from a message, which is the change
in uncertainty about the transmitter’s intent as a
consequence of receipt of the message; one bit of
information equates to a doubling of precision”, this
still seems to me to be talking about a communication
system, in that there is a transmitting agent who is
“putting” meaning, intention, within a transmitted
message; that is, it “represents” something.

Furthermore communication systems are designed
to
be that way, to transmit meaning. Natural control
systems are not communication systems, there is no
designer (external agent), there is no intention, or
representation, no meaning (apart from that inferred by
an external observer).

            Even if we take information just as "change

in uncertainty", what is there, in natural control
systems, about which we are uncertain? I still infer
from this the concept of something being regarded as
“correct”.

                              It should be possible to discuss

the validity, or otherwise, of
information theory with respect to
living systems at the conceptual level
without reference to the mathematics;
at least initially, and can be
accepted or rejected as such.

                I wish that were true, but so far I have not found

it possible. Maybe someone who has more facility
with language could do it, but whenever I try, I
find there are too many ambiguities left hanging.

            Well, maths is pretty meaningless without

any conceptual framework.

            If we can't get to the point where we see

that the IF is conceptually relevant to PCT then there’s
no point going further, as far as I am concerned. We
have to deal with our conceptual assumptions first.

                              The argument, seems to me, to be

the equivalent of the Representation
debate in the Artificial Intelligence
community. That is, that neural
signals “represent” something outside
of their own activation levels. In
other words they “model” or contain
“information” about the external
world.

                Red Flag! That pair of words "contain information"

always is a warning that something is going
conceptually off the rails.

            What's the difference between "contain

information" and representation, which you support
later?

                          ... However, there

still seems to be an overriding assumption
that neural systems model, or contain
information about, the world, but it is
just an assumption, made from the
viewpoint of complex beings (humans) who
are misguided by their perception of the
world around them that gives them a sense
that metal states represent the world.

                I don't see it so much an assumption as a way of

regarding the analysis. If from the state of X you
can deduce something about the state of Y, it seems
harmless to say that in some way X “represents” Y
(and as the maths tell you, vice-versa).

            Well, coming from an AI background this is

not at all harmless, but the most fundamental
question. There is a major philosophical difference
between saying X is (causally?) related to Y and X
“represents” Y. The latter, I would say, can lead to an
invalid view of how living systems work, and lead to
invalid research.

                              Martin, where is "information" in a

simple system like the zooplankton?

Nowhere. Information is never “in” any system.

            Ok, so how does, or can, IF be useful in the

context of studying or understanding the zooplankton?

                              From my viewpoint I do not see how

information is relevant to the
zooplankton. The neural signals do not
represent anything, apart from
themselves; they do not contain
messages or information, about which
there is uncertainty as there is no
“correct” value about which to be
uncertain.

                I don't know whether to say "true" or to point out

that those signals have values that depend on what
is happening in the environment outside the
organism. That explicitly does mean that there is
positive mutual information between the environment
and the signals.

Ok, but where does uncertainty come in?

                              The only reason for the signals

being the values they are is in order
to achieve control so that the
organism can survive within its
environment and pass on its genes.
There is no need for information.

                The mind boggles at this. The teleological statement

that the signals somehow manipulate themselves in
order to achieve control, and do this with no
reference to the state of the outside world, seems
really weird.

Who said that, I didn’t?

                            In fact the beauty, and radical

nature, of PCT is that it shows that
control systems (and any nervous
systems) are successful precisely
without any need for information and
representation.

                Obviously, I strongly disagree about "information".

If the perceptual signal did not vary at least
moderately consistently with the state of something
in the environment, how would the output be able
consistently to move the perception in the direction
of decreasing error?

            This last point sounds valid at first

glance, but I think there are some subtleties here that
might point to the differences in our viewpoints. Your
point seems to indicate that you see a strong
deterministic link or relationship between the external
and internal signals.

            I see them as very independent entities

that may sometimes be related, but other times not. For
example, if you are perceiving lights that are too
bright (your perceptual signal is “related” to the
external light signal), but then you put your hands over
your eyes the perceptual signal changes. The external
signal remains the same, yet there is no longer a
relationship.

                              The danger, which I think Rick has

alluded to, is if you start
introducing other theories, which are
not really relevant as if they are
then people may start using that
approach to model control systems
which would lead to a completely
incorrect divergent path of research,
as has been the case with AI.

                I don't think any paths of research are "incorrect".

They may turn out to be fruitless, but you can’t say
whether that will be the case without trying them.
They may not be the way you personally would choose
to do the research, but someone who did choose that
path would be equally legitimate in calling your
path “incorrect”. I would say you would both be
wrong.

            Incorrect, fruitless; potato, potatoe. The issue is

about paradigm creep. If one introduces concepts and
processes that aren’t particularly relevant from
different paradigms then it is more or less inevitable
that we end up doing something that is some bastardised
form of PCT.

            Rick may seem sensitive and over-zealous in his

defense of PCT, but I think he is quite right to protect
the “purity” of the theory against being tainted in this
way. I’m sure you can understand that.

                Likewise, if it is quicker or helpful in

understanding some aspect of a control system by
using information theory, then why not use it?

Fine, but I am still at a loss to see *
how* it is helpful, or even that the concepts of IF are
relevant to PCT. From what you say it seems that the key
is explaining how the concept of uncertainty applies to
PCT or to the variables in the loop.

[rupert]

[Rupert Young 2013.02.26 23.30]

Sorry meant IT but wrote IF.

(Martin Taylor 2103.02.25.23.20)

MT: If I have "information" about something, my uncertainty about it has been reduced. How is this different from the "common understanding"?

RY: But it was you who were disputing the usage of the "common understanding" in the wiki page; you wrote,

MT: There's a confusing issue of nomenclature in information theory discussions. I wish people had kept to Shannon's usage, which was clear and easily understood. You quotes from the Wikipedia entry make "information" mean what Shannon called "entropy" and I (following Garner) call "uncertainty". Shannon used "information" for "change in uncertainty", and that's how I think the term should be used.

MT: I fail to see the necessity of something being "correct" in order for there to be a change in uncertainty about it.

RY: Well if you are uncertain about something surely there must be an ideal against which you measure that uncertainty? In your example it's the brown shoes.

MT: There's a strong conceptual framework. Why do you suggest there isn't?

RY: I mean with respect to control systems.

MT: Doesn't the concrete example of Alice's weights that I gave in response to Adam show where the information theoretic concepts have a place?

RY: That seemed to be about a measurement system which I didn't see as applicable to PCT. Maybe that's where our difference is, as I don't see natural perceptual systems as measuring the external world.

MT:OK. Please explain the difference, and what you could possibly mean by "X represents Y" other than that the state of X is in some way related to the state of Y.

RY: Yes, that's right, but not the other way around. That is, if X is sometimes in some way related to the state of Y doesn't mean that "X represents Y".

RY: Ok, but where does uncertainty come in?

MT: In the fact that the states of the environment and the internal states of the zooplankton are variable, and in some respects co-variable.

RY: But, uncertainty of what?

RY: I see them as very independent entities that may sometimes be related, but other times not. For example, if you are perceiving lights that are too bright (your perceptual signal is "related" to the external light signal), but then you put your hands over your eyes the perceptual signal changes. The external signal remains the same, yet there is no longer a relationship.

MT: The external signal does not remain the same in that case. The external signal is only what the sensor systems report. The eyes don't get the same light, so the external signal has changed.

RY: By external I mean in the environment, how does that (the light) change?

MT: Yes, I understand it in the same way as I understand the control processes of the officers of the Spanish Inquisition. However, I fail to see how any particular method of analysis can threaten the purity of the religion in this case, unless, as in the case of the Inquisition, it is the simple act of enquiry that is threatening.

RY: Well, if I take the Copernican stance I am going to resist vigorously any attempts to apply a method of analysis that requires the claim that the Sun moves around the Earth.

All these varying interpretations of terminology aside would I be along the right lines to say that you are looking at a control system, or the perceptual signal, as measuring some aspect of the environment? And the uncertainty is an indication of how good control is; that is, how close the perceptual signal and the the environmental aspect is?

Rupert

[rsmarken]

[From Rick Marken (2013.02.26.1500)]

Martin Taylor (2013.02.24.09.56)–

RM: There are thee topics in this discussion that I want to quickly address.

1. Closed -loop nature of information theory.

You have never shown that information theory is a closed loop model. Your continued insistence that it is simply does not make it so.

2. The claim that information theory can tell us something about control that control theory can’t.

To demonstrate this claim you present data from a study you did to show information theory can contribute to our understanding of control. The study involve measuring control performance in a tracking task in which the distance between cursor and target was an independent variable. The results, presented i the nice graph below, shows that control performance becomes worse as the separation between cursor and target increases.

![CtrlPerfBits2.jpg|522x408](upload://e7yQoT3b3FyEPfoRkvtvd3QjYLN.jpeg)

RM: You interpret this result as being due to a decrease in the accuracy of perceiving the vertical separation between cursor and target as the horizontal separation increases. You say:

MT: So we have the question I posed in my last response to a

message from you “I wonder whether it would be possible to tease out
where in the loop the limit lies?” [Martin Taylor 2013.02.23.10.58].
At least this figure suggests that part of it in this experiment is
in the accuracy of perceiving the relative height of the cursor and
target.

RM: So you interpret these results to mean that poorer control with increasing separation results from decreased accuracy of perceiving the vertical separation of cursor and target. But this isthe way you see it when looking through information theory glasses. Looking at these results through control theory glasses give a very different perspective because you would first look for an explanation in terms of the variable under control: the controlled variable.

One reasonable possibility is that the controlled variable is the angular difference of the line connecting cursor and target from the line representing perfect horizontal alignment between cursor and target (with the reference for this variable being that this angle = 0). I tried this variable out in a control model, with horizontal separation a variable, as in you experiment, and found that control is poorer when the horizontal separation between cursor and target increases. This seems to be because the change in the angle is smaller per unit of vertical change in cursor position when the horizontal separation is wider and this decreases the loop gain of the control system. So it seems to me (if I did the modeling correctly) that you can account for these results without introducing the concept of accuracy. You just need the correct definition of the controlled variable.

3. You say that information theory is just a method of analysis that won’t lead one down the incorrect path. Rupert Young suggested that it might:

RM: To which you replied:

MT: I don’t think any paths of research are “incorrect”.

RM: In fact, PCT shows the path based on the use of open loop causal model of living systems, such as the GLM, is incorrect (as demonstrated in Powers 1978 Psych Review paper and several papers of my own). Since information theory is an open loop model of systems, the path of research based on information theory (research aimed at understanding the behavior of living organisms) is just as incorrect. Which is why I have been so high gain in arguing against it’s relevance to understanding behavior when behavior is understood to be a process of control.I don’t think PCT will ever get off the ground until people throw away all their open-loop glasses – information theory, the GLM, etc – and put on their control theory glasses so they can start seeing what they have never seen before: the controlled variables around which behavior is organized.

Best

Rick

···

RY: The danger, which I think Rick has alluded
to, is if you start introducing other theories, which
are not really relevant as if they are then people may
start using that approach to model control systems which
would lead to a completely incorrect divergent path of
research, as has been the case with AI.

[MartinT]

[Martin Taylor 2013.02.27.00.01]

When have I ever suggested that information theory is a closed loop

(or any other kind of) model of anything?
I guess it would help if you described “control theory”. My
understanding of “control theory” is that it is the general theory
of how control systems behave. To have a complete understanding of
control theory is extremely difficult, and I don’t think that anyone
has it. To be able to say some things about how some control systems
behave, there are various tools such as Bode plots, spectral
analysis, correlational analyses, … Information theory is among
those tools. So I don’t see how it is possible that information theory, being a
tool within control theory, can tell us anything that control theory
can’t. It may be able to tell something that other tools can’t.
True, and you are correct to question this interpretation.
That’s not a contrast between information theory glasses and control
theory glasses. The same questions would arise when calculating the
uncertainty involved in controlling the angle.
You are quite right that the angle may well be the controlled
variable. It doesn’t make a difference to the Control performance in
bits, but it makes a big difference to the apparent resolution of
perception.
How would we test for whether the absolute difference in level or
the angle between the ellipses is the controlled variable? They seem
to be inextricably confounded at any one separation. One thing I did
think of when I started programming this experiment was to draw a
non-horizontal reference line an ask the subjects to keep the angle
between the ellipses parallel to it, but that seemed to be subject
to the same problem.
The results of research show that the open-loop model doesn’t fit
the data as well as does perceptual control. That doesn’t mean that
it was incorrect to try that model, which is what a path of research
is. Only by trying different paths of research can one converge on a
“best theory at the moment” result. I don’t think any paths of
research are “incorrect”, no matter how wrong later work shows them
to be. All my academic life I have believed that every theory we now
have will eventually be proved to be wrong. That doesn’t mean the
research leading to the theory followed an incorrect path.
Why continue repeating this nonsense? Information theory is not any
kind of model of systems, any more than is Fourier spectral
analysis. Is circuit analysis an “open-loop model of systems”
because by using circuit analysis you can tell the current through a
resistor if you know its resistance and the voltage between input
and output?
I thought we had mutually thrown this nonsense into the garbage a
week or so ago. I’m sorry to see it resurrected. Why does the zombie
of “information theory denies perceptual control” keep coming to
life?
Martin

···

[From Rick Marken (2013.02.26.1500)]

        Martin Taylor

(2013.02.24.09.56)–

      RM: There are thee topics in this discussion that I want to

quickly address.

      1. Closed -loop nature of information theory.



      You have never shown that information theory is a closed loop

model. Your continued insistence that it is simply does not
make it so.

      2. The claim that information theory can tell us something

about control that control theory can’t.

      To demonstrate this claim you present data from a study you

did to show information theory can contribute to our
understanding of control. The study involve measuring control
performance in a tracking task in which the distance between
cursor and target was an independent variable.

      RM: You interpret this result as being due to a decrease in

the accuracy of perceiving the vertical separation between
cursor and target as the horizontal separation increases.

You say:

        MT: So we have the

question I posed in my last response to a message from you
“I wonder whether it would be possible to tease out where in
the loop the limit lies?” [Martin Taylor 2013.02.23.10.58].
At least this figure suggests that part of it in this
experiment is in the accuracy of perceiving the relative
height of the cursor and target.

  RM: So you interpret these results to mean that poorer control

with increasing separation results from decreased accuracy of
perceiving the vertical separation of cursor and target. But this
isthe way you see it when looking through information theory
glasses. Looking at these results through control theory glasses
give a very different perspective because you would first look for
an explanation in terms of the variable under control: the
controlled variable.

  One reasonable possibility is that the controlled variable is the

angular difference of the line connecting cursor and target from
the line representing perfect horizontal alignment between cursor
and target (with the reference for this variable being that this
angle = 0).

  3. You say that information theory is just a method of analysis

that won’t lead one down the incorrect path. Rupert Young
suggested that it might:

              RY: The danger, which I think Rick has alluded to,

is if you start introducing other theories, which are
not really relevant as if they are then people may
start using that approach to model control systems
which would lead to a completely incorrect divergent
path of research, as has been the case with AI.

RM: To which you replied:

    MT:  I

don’t think any paths of research are “incorrect”.

  RM: In fact,  PCT shows the path based on the use of open loop

causal model of living systems, such as the GLM, is incorrect (as
demonstrated in Powers 1978 Psych Review paper and several papers
of my own).

  Since information theory is an open loop model of

systems,

  the path of research based on information theory

(research aimed at understanding the behavior of living organisms)
is just as incorrect. Which is why I have been so high gain in
arguing against it’s relevance to understanding behavior when
behavior is understood to be a process of control.I don’t think
PCT will ever get off the ground until people throw away all their
open-loop glasses – information theory, the GLM, etc – and put
on their control theory glasses so they can start seeing what they
have never seen before: the controlled variables around which
behavior is organized.

[amatic]

[From Adam Matic, 2013.02.27.1330cet]

(Martin Taylor 2103.02.25.23.20)
I believe that if you think about it a bit, you will see that the mathematical descriptions of both “information” and “uncertainty” are not very different from the everyday meanings. Shannon originally used “Uncertainty” for what he later called “Entropy”. He called it “Uncertainty” precisely because it seemed to make precise the feeling we have when we are uncertain about something. When we get information about something of which we have been uncertain, we usually become less uncertain.

In the maths, it is “probability” that is the fuzzy concept, not “Uncertainty”. In everyday speech, most words have fuzzy edges to their meanings. “Uncertainty” is no exception, which is why I tried to avoid referring in my tutorial to everyday meanings, and insisted on using only the mathematical description. But as I said, when you look at it carefully, you find that the mathematical description very closely tracks the everyday meaning – it just puts hard edges on the concept.

Well… when you put it that way… it does seem to make a lot of sense. If I imagine myself as the receiver of a message, or someone measuring a variable, I can understand the meaning of “information” as precision. In the control loop I would imagine myself as reading out ‘p’ as the only source of CEV magnitude. If I only knew if the light is on or off, I would still be very uncertain about the intensity of light; although, I would have some information about it (exactly one bit of information).

I guess I didn’t understand what “I was uncertain about”. And the probability calculations didn’t fit in the picture either, especially in the context of control.

Ok. So IT is a tool to consider when exploring control.

AM: What I’m saying is, from inside the loop, the effects of noise on
p are indistinguishable from effects of disturbances on p. There is

already a whole analysis of effects of disturbances on the loop. No
need to repeat it.

MT: I don’t know that we would be repeating an analysis by using different measures. Would you say you were repeating a time-series analysis if you did a Fourier spectral analysis of the same system? Sometimes you see something more clearly from one angle than from another. I usually like to be able to take a variety of viewpoints if it helps me to understand.

AM:

Well, what I saw with noise propagation analysis was that it was impossible to determine weather the change in p was due to a disturbance or it was due to noise. Now, I’m not very sure about this. If I’m in the loop, reading out p, and there are ‘small random variations’, I could consider them to be noise, but there seems to be an arbitrary definition of ‘small’.

What about more abstract variables? Like, if you’re doing a project, would more information about the current stage of the project also mean better precision and better control? I’m having problems trying to imagine this in mathematical form, but it seems intuitively correct.

MT: My quantitative problem at the moment goes back to an issue that will recur if we do the experiment you proposed just above – how to tease out the uncertainty contributed by low accuracy output (e.g. jerky mouse movements when I want them to be smooth). Until I solve that problem, I think it will be hard to match control systems using different perceptions. I do have a couple of notions, but they probably won’t pan out, and first I want to program a simulation model to see whether the control parameters (such as integrator gain rate and loop transport lag) vary with perceptual accuracy.

Why would it be necessary to find out how much uncertainty is contributed by the output? I mean, if we are comparing two systems differing only in perceptual functions, then their difference in control is due to their perceptual functions.

Adam

[MartinT]

[Martin Taylor 2013.02.27.11.13]

[From Adam Matic, 2013.02.27.1330cet]

> AM: What I'm saying is, from inside the loop, the effects of noise on
> p are indistinguishable from effects of disturbances on p. There is
> already a whole analysis of effects of disturbances on the loop. No
> need to repeat it.
>
> MT: I don't know that we would be repeating an analysis by using different measures. Would you say you were repeating a time-series analysis if you did a Fourier spectral analysis of the same system? Sometimes you see something more clearly from one angle than from another. I usually like to be able to take a variety of viewpoints if it helps me to understand.

AM:
Well, what I saw with noise propagation analysis was that it was impossible to determine weather the change in p was due to a disturbance or it was due to noise. Now, I'm not very sure about this. If I'm in the loop, reading out p, and there are 'small random variations', I could consider them to be noise, but there seems to be an arbitrary definition of 'small'.

The problem is exactly as you say. There is no way for the perceptual function to discriminate between variation caused by changes in the CEV and variation caused by changes in the path between the CEV and the perceptual function. How would you know if there were "small random variations" when the only measure you have is the current value of the variables, and as Rupert has been pointing out, they just "do their thing" without worrying about anything else?

Obviously (which usually means "I can't think of another answer"), the perception of "small random variations" must be a different perception, not the one being controlled. Sometimes I find one can make these variations conscious by paying very close attention to something when the conditions are a little difficult -- such as in dim light. We usually are not conscious of these slight fluctuations, but with some practice, one can become conscious of them.

Here's a heretical suggestion. If we have two different perceptions, one -- call it X -- being the one whose control we are considering, and the other -- call it Y -- looking at some statistic of X variation over time, then the Y perception might be controlled to some reference value if its output were to influence the properties of the X perceptual input function. Y could not be a normal part of the HPCT hierarchy (and needs some experiments and simulations to see whether it is even feasible) because it would not be acting on anything in the outer environment, but it might assist the standard hierarchy to operate effectively in the presence of perceptual difficulty.

If you think of this problem as being an example of Shannon's message communication link with redundancy, there is a problem. Shannon's receiver could reconstruct a noisy message by knowing the coding used by the transmitter. Certain messages would be impossible according to the agreed code, so the receiver could know that something needed to be corrected. That is not the case with a control system in which all the signals are simple scalar variables. All values are possible. A perceptual input function might average over time or fail to respond to "too rapid" changes to reduce the effect of "small random variations", but that wouldn't change the problem.

What IS the case for a simple control loop is that somewhere in the loop there must be a time-binding function. In the model usually used to fit tracking data, that time-binding is a leaky integrator in the output function, but it could be in an environmental feedback function that spreads the effect of the output on the CEV over time. What is also the case is that if the output compensates for a perception that does not consistently track changes in the CEV, the perception will not be well controlled. So the "Y" perception suggested above could take the form of rapid reorganization, in which the dynamics of the error signal are perceived at Y and affect the properties of the control loop rather than of its connections to other parts of the hierarchy. (I'm not sure whether this suggestion is equally heretical).

What about more abstract variables? Like, if you're doing a project, would more information about the current stage of the project also mean better precision and better control? I'm having problems trying to imagine this in mathematical form, but it seems intuitively correct.

One of the neat things about the information-theoretic approach is that it is just as applicable when the variables in question are discrete as it is when they are continuous. In your example, "better precision" has to mean that there is less uncertainty about the actual state of the project -- is Joe likely to finish his module in time to meet Ben's timeline for fitting it into the overall design of the bridge? Asking Joe might make you less uncertain of his answer, increasing your precision of perception of the project state (though not necessarily your accuracy of perception, Joe being an eternal optimist

> MT: My quantitative problem at the moment goes back to an issue that will recur if we do the experiment you proposed just above -- how to tease out the uncertainty contributed by low accuracy output (e.g. jerky mouse movements when I want them to be smooth). Until I solve that problem, I think it will be hard to match control systems using different perceptions. I do have a couple of notions, but they probably won't pan out, and first I want to program a simulation model to see whether the control parameters (such as integrator gain rate and loop transport lag) vary with perceptual accuracy.

Why would it be necessary to find out how much uncertainty is contributed by the output? I mean, if we are comparing two systems differing only in perceptual functions, then their difference in control is due to their perceptual functions.

True, but if the output uncertainty is the limiting factor, increasing the perceptual precision will not influence the result. If the question were a cut-and-dried issue of the channel capacities of the different segments of the loop, then the segment with the lowest capacity would dictate the entire result. Either the output or the perception would provide the limit, but not both.

However, it isn't that simple. Forget about "information" and "uncertainty" for the moment, and think about variance in a low-level tracking loop. If there is some output variance that is independent of the disturbance variance, the two variances add numerically at the CEV. The CEV variance will be the sum of the two, and that summed variance will propagate around the loop, with a result I have not calculated.

Uncertainty is proportional to standard deviation if the distributions are Gaussian, so a similar argument applies to uncertainties, except that, like standard deviations, they don't simply add at the CEV. Again, I haven't computed how this propagates around the loop, but it should mean that to some extent variation due to output inaccuracy and variation due to perceptual inaccuracy will both be reflected in the output variation. However, I don't think you can get at the effect of one by simply subtracting the effect of the other. Both channel capacity limits and uncertainty propagation may come into play under different circumstances, making it all quite tricky to compute for a particular situation.

Martin

[MartinT]

[Martin Taylor 2013.02.27.12.31]

[Rupert Young 2013.02.26 23.30]

(Martin Taylor 2103.02.25.23.20)

MT: If I have "information" about something, my uncertainty about it has been reduced. How is this different from the "common understanding"?

RY: But it was you who were disputing the usage of the "common understanding" in the wiki page; you wrote,

MT: There's a confusing issue of nomenclature in information theory discussions. I wish people had kept to Shannon's usage, which was clear and easily understood. You quotes from the Wikipedia entry make "information" mean what Shannon called "entropy" and I (following Garner) call "uncertainty". Shannon used "information" for "change in uncertainty", and that's how I think the term should be used.

"Common understanding" in the WIki page is different from everyday "common understanding". You asked why I did not conform to the common understanding that information theory was only about communication. In what you question above, I talked about the everyday usage of the terms.

MT: I fail to see the necessity of something being "correct" in order for there to be a change in uncertainty about it.

RY: Well if you are uncertain about something surely there must be an ideal against which you measure that uncertainty? In your example it's the brown shoes.

Not at the time we are talking about new information and changing uncertainty. The brown or not-brown shoes haven't happened yet.

MT: There's a strong conceptual framework. Why do you suggest there isn't?

RY: I mean with respect to control systems.

There's no special conceptual framework for uncertainty and information in control systems, any more than there is a special conceptual framework when we use the techniques of circuit analysis in control systems. You just use the tool.

MT: Doesn't the concrete example of Alice's weights that I gave in response to Adam show where the information theoretic concepts have a place?

RY: That seemed to be about a measurement system which I didn't see as applicable to PCT.

Did you not notice that Alice's weight measurement is structured as a canonical perceptual control system?

Maybe that's where our difference is, as I don't see natural perceptual systems as measuring the external world.

Nor do I, but that doesn't mean perceptual control fails to do exactly what a measurement system would do if the objective was a readout of the output value at the CEV when the perception matched its reference value.

MT:OK. Please explain the difference, and what you could possibly mean by "X represents Y" other than that the state of X is in some way related to the state of Y.

RY: Yes, that's right, but not the other way around. That is, if X is sometimes in some way related to the state of Y doesn't mean that "X represents Y".

Well, what does "represent" mean to you that makes it different? Is it that if X is to "represent" Y there must be a direct causal link from Y to X? I can't say if I agree or disagree with you about "representation" unless I know what you mean when you use the word.

RY: Ok, but where does uncertainty come in?

MT: In the fact that the states of the environment and the internal states of the zooplankton are variable, and in some respects co-variable.

RY: But, uncertainty of what?

Of what the values of those variables might be, and of how their variations relate to each other.

RY: I see them as very independent entities that may sometimes be related, but other times not. For example, if you are perceiving lights that are too bright (your perceptual signal is "related" to the external light signal), but then you put your hands over your eyes the perceptual signal changes. The external signal remains the same, yet there is no longer a relationship.

MT: The external signal does not remain the same in that case. The external signal is only what the sensor systems report. The eyes don't get the same light, so the external signal has changed.

RY: By external I mean in the environment, how does that (the light) change?

Unless you can sense it, you can't perceive it. We have no clairvoyant capabilities of determining properties of teh environment that don't affect our sensor organs, so far as I am aware. Lots of thing happen in the environment, but you can't call them "signals" in a biological control loop until they influence the values reported by the sensor organs to the interior of the body's skin bag.

MT: Yes, I understand it in the same way as I understand the control processes of the officers of the Spanish Inquisition. However, I fail to see how any particular method of analysis can threaten the purity of the religion in this case, unless, as in the case of the Inquisition, it is the simple act of enquiry that is threatening.

RY: Well, if I take the Copernican stance I am going to resist vigorously any attempts to apply a method of analysis that requires the claim that the Sun moves around the Earth.

I very much doubt that was the problem, and such an analysis would probably be refuted by observations, in any case. I think the problem was more along the lines of "If he gets away with denying the Pope's authority on this, the floodgates will be open for denying more important things. We have to stop the questioning." The unknown is often threatening, and if unrestricted enquiry is permitted, who knows what results might be obtained that might contradict something else the Pope said, particularly if he said it _ex cathedra_?

All these varying interpretations of terminology aside would I be along the right lines to say that you are looking at a control system, or the perceptual signal, as measuring some aspect of the environment? And the uncertainty is an indication of how good control is; that is, how close the perceptual signal and the the environmental aspect is?

Not really. You would be closer if you reversed some of what you say. An extended (and probably philosophically naive) discussion of the relation of perceptual control to "real reality" that I wrote in a different group is downloadable from <http://pctweb.org/martintaylorcontrol,pdf&gt;\. Some of what I said there might be helpful in this discussion, and might make sense of my "if you reversed...".

As for uncertainty as an indicator of the quality of control, that's one use of the uncertainty measure. Usefully, it is applicable at all levels of the perceptual hierarchy. One can use it in the same way as one can use variances to indicate the quality of control in low-level tracking tasks, and in those tasks one can use either measure, since uncertainty is proportional to the square root of variance if the shape of the distribution remains unchanged (e.g. Gaussian). But there are lots of other ways uncertainty measures might be useful in considering control.

Martin

[rsmarken]

[From Rick Marken (2013.02.27.1200)]

Martin Taylor (2013.02.27.00.01)–

MT: When have I ever suggested that information theory is a closed loop

(or any other kind of) model of anything?

RM: I though you did. But if you are not, that’s nice.

      RM2. The claim that information theory can tell us something

about control that control theory can’t.

MT: ... To be able to say some things about how some control systems

behave, there are various tools such as Bode plots, spectral
analysis, correlational analyses, … Information theory is among
those tools.

RM: I have seen no evidence that information theory contributes anything other than obfuscation to our understanding of control, particularly the controlling done by living systems. I think that, like the GLM, it’s just the wrong tool for understanding control.

  RM: So you interpret these results to mean that poorer control

with increasing separation results from decreased accuracy of
perceiving the vertical separation of cursor and target. But this
isthe way you see it when looking through information theory
glasses. Looking at these results through control theory glasses
give a very different perspective because you would first look for
an explanation in terms of the variable under control: the
controlled variable.

MT: That's not a contrast between information theory glasses and control

theory glasses. The same questions would arise when calculating the
uncertainty involved in controlling the angle.

RM: But I don’t see what calculating the uncertainly gets you.Your measure of uncertainly seems to me to be nothing more than a mathematical (log) transformation of rms error. And the calculation only makes sense once you’ve determined the controlled variable, a concept that doesn’t even exist in informatoin theory.

There is a huge contrast between information theory and control theory glasses which is clear in your experiment. You designed it under the assumption that increasing the separation between cursor and target would reduce the accuracy of your perception of the relative vertical positions of cursor and target. So informatoin theory led you to assume that there was an entity out there – vertical distance between cursor and target – that was to perceived. And that horizontal distance would affect the accuracy (amount of information) that you gt about the state of that perception. So the poorer control seen with the larger horizontal separation was consistent with the way you saw the situation through information theory glasses. It would never occur to someone doing this experiment (which is an experiment that would be done in the context of the standard causal paradigm) that the perception controlled by the subject is not the same as the one assumed by the experimenter. Information theory glasses (like the causal model glasses currently worn by most research psychologists) simply blind you to the question of what perceptual variable the subject is controlling. Information theory is not only not relevant to the study of living control systems; it blinds one to the most important (and up until PCT the most unnoticed) aspect of the controlling done by living systems: the perceptual variables around which their behavior is organized.

MT: How would we test for whether the absolute difference in level or

the angle between the ellipses is the controlled variable?

RM: One way is the way I did it: using modeling. If the model with absolute distance doesn’t should reduced control with separation (which, of course, it wouldn’t) then try a different controlled variable. Of course, you could add noise to the absolute difference model in proportion to the separation and account for the data as well as the angle control model. But that is quite a bit less parsimonious. And there are surely other experiments you could devise to tease this out.

  RM: 3. You say that information theory is just a method of analysis

that won’t lead one down the incorrect path. Rupert Young
suggested that it might:

MT: The results of research show that the open-loop model doesn't fit

the data as well as does perceptual control. That doesn’t mean that
it was incorrect to try that model, which is what a path of research
is.

RM: Right. There was nothing wrong with trying it, especially since there was no alternative. But at least you admit that it was wrong and led down a wrong path. Unfortunately, Psychology has followed that path for over 100 years and shows no sign of interest in trying a new one.

  RM: Since information theory is an open loop model of

systems,

MT: Why continue repeating this nonsense? Information theory is not any

kind of model of systems, any more than is Fourier spectral
analysis. Is circuit analysis an “open-loop model of systems”
because by using circuit analysis you can tell the current through a
resistor if you know its resistance and the voltage between input
and output?

RM: Yes, circuit analysis is based on an open loop model of the system as is information theory. Fourier analysis is not a model of systems; it’s a tool for the analysis of waveforms.

  RM: the path of research based on information theory

(research aimed at understanding the behavior of living organisms)
is just as incorrect. Which is why I have been so high gain in
arguing against it’s relevance to understanding behavior when
behavior is understood to be a process of control.I don’t think
PCT will ever get off the ground until people throw away all their
open-loop glasses – information theory, the GLM, etc – and put
on their control theory glasses so they can start seeing what they
have never seen before: the controlled variables around which
behavior is organized.

MT: I thought we had mutually thrown this nonsense into the garbage a

week or so ago. I’m sorry to see it resurrected. Why does the zombie
of “information theory denies perceptual control” keep coming to
life?

RM: Informatoin theory doesn’t deny control theory any more than the causal model does. You are misidentifying zombies. The zombie that won’t go away is the idea that using information theory as a basis for understanding control blinds one to the most essential aspect of control: controlled variables. This is actually nicely illustrated by the experiment you described. I’m really glad that you proposed (and carried out!) an actual experimental study to show what one could learn about control from information theory. But, from my perspective, it illustrated exactly the opposite; it showed rather clearly what you miss when you do research based on an understanding of control based on information theory.

Best

Rick

···

–
Richard S. Marken PhD
rsmarken@gmail.com
www.mindreadings.com

[MartinT]

[Martin Taylor 2013.02.27.15.31]

[From Rick Marken (2013.02.27.1200)]

        Martin Taylor

(2013.02.27.00.01)–
MT: … To be able to
say some things about how some control systems behave,
there are various tools such as Bode plots, spectral
analysis, correlational analyses, … Information theory
is among those tools.

      RM: I have seen no evidence that information theory

contributes anything other than obfuscation to our
understanding of control, particularly the controlling done by
living systems. I think that, like the GLM, it’s just the
wrong tool for understanding control.

Then don't try to use it, or to learn enough about it to be able to

discuss why you might be right or wrong.

            RM: So you interpret these

results to mean that poorer control with increasing
separation results from decreased accuracy of perceiving
the vertical separation of cursor and target. But this
isthe way you see it when looking through information
theory glasses. Looking at these results through control
theory glasses give a very different perspective because
you would first look for an explanation in terms of the
variable under control: the controlled variable.

        MT: That's not a contrast between information theory glasses

and control theory glasses. The same questions would arise
when calculating the uncertainty involved in controlling the
angle.

        RM: But I don't see what calculating the uncertainly gets

you.

I know you don't. And won't.

        Your measure of uncertainly seems to me to be nothing

more than a mathematical (log) transformation of rms error.

For low-level tracking, that's exactly what it is. That's the main

reason I wanted to use such a task in the experiment – to show that
the information approach does the same thing as the conventional
approach under those conditions. I thought that would make it easier
to understand how it works when applied to control in conditions
where “rms error” is a meaningless concept.

        And the calculation only makes sense once you've

determined the controlled variable, a concept that doesn’t
even exist in informatoin theory.

Of course it doesn't. Why on earth would you think it might or

should? You are in a different domain of discourse. “Controlled
variable” is a concept applicable to considering what to examine.
Information theory is a way of examining whatever is reasonable to
analyze once you have some guesses as to what the controlled
variable might be.

        There is a huge contrast between information theory and

control theory glasses

Such a comparison is not possible, since one would normally be

wearing both at once.

        which is clear in your experiment. You designed it under

the assumption that increasing the separation between cursor
and target would reduce the accuracy of your perception of
the relative vertical positions of cursor and target.

Correct.

        So informatoin theory led you to assume that there was

an entity out there – vertical distance between cursor and
target – that was to perceived.

That is certainly a possible controlled variable in the experiment.

        And that horizontal distance would affect the accuracy

(amount of information) that you gt about the state of that
perception. So the poorer control seen with the larger
horizontal separation was consistent with the way you saw
the situation through information theory glasses.

True. Your point is?

        It would never occur to someone doing this experiment

(which is an experiment that would be done in the context of
the standard causal paradigm) that the perception controlled
by the subject is not the same as the one assumed by the
experimenter.

It certainly occurred to me, as I mentioned in my earlier response

to you.

        Information theory glasses (like the causal model

glasses currently worn by most research psychologists)
simply blind you to the question of what perceptual variable
the subject is controlling.

And a strong control system with a reference for not perceiving X

will often allow the perception of X not to occur. You control so
strongly for NOT seeing what information theory is about that you
issue a flood, rather than a stream, of nonsense about it.

        MT: How would we test for whether the absolute difference in

level or the angle between the ellipses is the controlled
variable?

        RM: One way is the way I did it: using modeling. If the

model with absolute distance doesn’t should reduced control
with separation (which, of course, it wouldn’t) then try a
different controlled variable.

I don't understand this. It isn't English and none of my attempts at

correcting it into English make any sense.

My problem with distinguishing the two suggested possibilities for

the controlled variable (vertical location difference and angle of
imaginary connecting line between the ellipses) is that I can’t
think of a way of doing The Test that would distinguish them.
Everything you do to disturb the vertical separation also disturbs
the angle to the same proportionate extent. That’s why I asked if
you could think of such a test.

        Of course, you could add noise to the absolute

difference model in proportion to the separation and account
for the data as well as the angle control model. But that is
quite a bit less parsimonious. And there are surely other
experiments you could devise to tease this out.

You are good at thinking of critical experiments for purposes such

as this. I hoped you would think of one now, because I couldn’t.

          RM: 3. You say that information

theory is just a method of analysis that won’t lead one
down the incorrect path. Rupert Young suggested that it
might:

        MT: The results of research show that the open-loop model

doesn’t fit the data as well as does perceptual control.
That doesn’t mean that it was incorrect to try that model,
which is what a path of research is.

      RM: Right. There was nothing wrong with trying it, especially

since there was no alternative. But at least you admit that it
was wrong

Nice implication here. Remember that I came to PCT from a different

starting point than most people do. I had independently arrived at
the Layered Protocol Theory of human-human or computer-human
interaction, some time before I discovered PCT and realized that LPT
was actually a special case of PCT. PCT seemed very natural to me as
a consequence. So I don’t think I now “admit” something I
acknowledged over 20 years ago was the general case of my particular
theory.

          RM: Since information theory is an

open loop model of systems,

      MT: Why continue repeating this nonsense? Information theory

is not any kind of model of systems, any more than is Fourier
spectral analysis. Is circuit analysis an “open-loop model of
systems” because by using circuit analysis you can tell the
current through a resistor if you know its resistance and the
voltage between input and output?

      RM: Yes, circuit analysis is based on an open loop model of

the system as is information theory.

Then, please, what DO you mean by "control theory". I thought you

meant the kind of analysis that goes

p = qo + d = G*(r-p) +d  etc. etc.

But now you say that's NOT what you mean, because these equations

are curcuit analysis, and circuit analysis is based on an open loop
model of the system, So what is it that you mean when you appeal to
the universal “control theory” in the sky?

      Fourier analysis is not a

model of systems; it’s a tool for the analysis of waveforms.

Yes, and circuit analysis is not a model of systems. It is a way of

analyzing circuits. Information theory is not a model of systems. It
is a way of analyzing the variations and relations among the
variations of different variables. So what?

I hope you will be able to suggest an experiment that uses The Test

for the controlled variable to distinguish between the two suggested
possibilities for the controlled variable in the ellipse experiment.
And, though this is probably too much to hope for, that you will
offer a description of “control theory” that does not use circuit
analysis but that does allow “control theory” to be subject to
scientific investigation.

Martin

[amatic]

From Adam Matic 2013.02.28.0000cet]

(Martin Taylor 2013.02.27.11.13)
The problem is exactly as you say. There is no way for the perceptual function to discriminate between variation caused by changes in the CEV and variation caused by changes in the path between the CEV and the perceptual function. How would you know if there were “small random variations” when the only measure you have is the current value of the variables, and as Rupert has been pointing out, they just “do their thing” without worrying about anything else?

Obviously (which usually means “I can’t think of another answer”), the perception of “small random variations” must be a different perception, not the one being controlled. Sometimes I find one can make these variations conscious by paying very close attention to something when the conditions are a little difficult – such as in dim light. We usually are not conscious of these slight fluctuations, but with some practice, one can become conscious of them.

Here’s a heretical suggestion. If we have two different perceptions, one – call it X – being the one whose control we are considering, and the other – call it Y – looking at some statistic of X variation over time, then the Y perception might be controlled to some reference value if its output were to influence the properties of the X perceptual input function. Y could not be a normal part of the HPCT hierarchy (and needs some experiments and simulations to see whether it is even feasible) because it would not be acting on anything in the outer environment, but it might assist the standard hierarchy to operate effectively in the presence of perceptual difficulty.

AM:

I suppose it’s possible to have adjustable-length temporal averaging (filtering) in the input function.

We do come prewired with receptors with different filtering properties.

For light receptors:

“The period of integration is up to 0.1 seconds or 100 ms for rods and 10 to 15 ms for cones. The advantage of long integration time is that under limited light level conditions, threshold will be reached, whereas when light levels are not limiting (cone or photopic vision), a short integration time is preferable to improve temporal resolution.”

http://webvision.med.utah.edu/book/part-viii-gabac-receptors/temporal-resolution/

On the other hand, I’m not sure if we come prewired with an exact amount of redundant fibers per bundle (spatial averaging). I should check a textbook on neural development to see how nerve bundles form. It might be that the amount of noise is a CV, controlled by ‘adding new fibers’. Noise could simply be any high-frequency change. Noise could also be too low frequency change… I won’t go further with speculation.

So, you’re saying that information analysis might prove useful for exploring development of complex, stable input functions, reorganization in the input part and such mysteries?

MT: One of the neat things about the information-theoretic approach is that it is just as applicable when the variables in question are discrete as it is when they are continuous. In your example, “better precision” has to mean that there is less uncertainty about the actual state of the project – is Joe likely to finish his module in time to meet Ben’s timeline for fitting it into the overall design of the bridge? Asking Joe might make you less uncertain of his answer, increasing your precision of perception of the project state (though not necessarily your accuracy of perception, Joe being an eternal optimist

AM:

Nice. Asking Joe, then, might not be the best strategy for improving control. An analogy to spatial averaging would be to ask a lot of people what they think about where the project is, or add some other “objective” sources of information. Temporal averaging would be to just look at week reports instead of daily reports.

There are trade-offs to both. In spatial averaging, you need a complex input function. In temporal averaging you loose the ability to react quickly.

MT: Uncertainty is proportional to standard deviation if the distributions are Gaussian, so a similar argument applies to uncertainties, except that, like standard deviations, they don’t simply add at the CEV. Again, I haven’t computed how this propagates around the loop, but it should mean that to some extent variation due to output inaccuracy and variation due to perceptual inaccuracy will both be reflected in the output variation. However, I don’t think you can get at the effect of one by simply subtracting the effect of the other. Both channel capacity limits and uncertainty propagation may come into play under different circumstances, making it all quite tricky to compute for a particular situation.

AM:

I’m not sure I understand exactly what propagates and what effects what, but simulating such a loop could be faster than computing noise propagation effects.

Adam