Info theory: putting some ideas together

[From Bill Powers (930314.0900)]

Martin Taylor and I have been having a go-round privately. In one
of his posts (930312) he said

If I (the sensor) know that those equations are being used, and
there is no added noise or phase uncertainty, then I need not
look at the sample values. There is no uncertainty, and the
signal conveys no information.

I was very puzzled about why the sensor had to know anything, but
evidently the last sentence above stuck, and worked away in my
head to lead to my post on the net yesterday, first about the
control system rejecting information from the disturbance and
then the understanding that there is one virtual channel from
disturbance to output, and another from reference signal to
controlled variable. The better the control system (the smaller
it can keep its error signal and the wider the bandwidth over
which it can do so), the more faithfully do these channels
reproduce their input information at their outputs.

Information in higher levels of control inside the organism can
be expressed in the environment as the behavior of visible
controlled variables. That is the output part of communication.
It represents a flow of information from the organism into the
environment. But the obvious source of information flowing from
the enviroment into the organism -- a disturbance acting on the
controlled variable -- is not an actual source, because that
information is cancelled by the actions of the control system,
and does not show up as changes in the perception of the
controlled variable. Environmental information from the
disturbance shows up in the output actions of the organism, again
in the environment, cancelling the effects of the disturbance. So
disturbance information enters the system, but comes out again
immediately as opposing actions.

So how DOES information get into the organism from the
environment? Clearly, only through uncontrolled perceptions.
Uncontrolled perceptions may be uncontrolled at many successive
levels, but eventually they join with controlled perceptions to
become inputs to a high-level control system. There they act as
disturbances of the perception to which they contribute, _and
their information content is therefore rejected by that system_.
The actions of that system that oppose the effects of the
uncontrolled perception result (after passing through layers of
reference signals on their way toward the periphery) in
variations of visible controlled variables. Those visible
variations carry the information about what the highest-level
system involved did to cancel the disturbing effect of the
incoming information. And perception of them joins with the
uncontrolled perceptions to leave the highest-level perception
undisturbed.

This leads to a rather odd concept of communication: the output
information from a system produced by intentional variations in
controlled variables reflects the attempt of a higher-level
control system to prevent uncontrolled input information from
disturbing a higher-level controlled variable.

This has a negative ring to it, but perhaps this is not as
negative a picture as it seems. Suppose that the highest-level
system involved (which could be at any level) receives
information via uncontrolled input channels that disturbs a
world-view that is maintained by manipulating controlled
perceptions. The first reaction to the uncontrolled information
input will be an adjustment of lower-level reference signals that
cancels the disturbance of the world view which the uncontrolled
information tends to produce. If this is verbal communication, a
statement that upsets the world view is countered by adjusting
controllable details of perception. When I introduce a black
rocket scientist to a bigot, the bigot immediately adjusts the
controllable perceptions to explain how a dumb black person could
be a rocket scientist. Oh, he's their token black, he's probably
being carried by his colleagues so he appears brilliant, but the
real work is probably done by someone else. This leaves the
bigoted world-view of black people undisturbed; the information
is rejected.

This works as long as the adjustments of controlled perceptions
suffice as a way of rejecting the information and maintaining the
world-view perception. But if the right kind of information is
received, control may become more and more difficult, or fail.
Then the person must start to reorganize -- find another world
view and other ways of adjusting lower reference signals in which
the adjustments will again succeed in preventing a disturbance
from the uncontrolled information input. This can lead to a
change in world view so that the disturbance is no longer a
disturbance. The person now thinks, "Ho, hum, another black
rocket scientist, so what's new?."

Or apply this to a conventional psychologist observing a rubber-
band demonstration pf PCT. At first, this uncontrolled new
information leads to strenuous attempts to adjust other
perceptions and prevent a disturbance of the conventional view of
how behavior works. But the longer additional information about
control behavior enters the system, the more problems there are
in finding adjustments of already-controlled perceptions that
will in fact cancel the disturbance. Eventually the perception
will change, and the former disturbance will be seen as a proper
aspect of behavior, behavior the way it is now understood to
work.

We say that we understand and accept communications from others
when doing so does not constitute a disturbance of our controlled
perceptions -- in other words, when no information is received
from the communications. This is what makes gatherings of like-
minded people so boring. None of their communications with each
other cause any disturbances, or carry any information. It is
what makes the CSG, and CSGnet, so interesting and attention-
grabbing: there are plenty of disturbances, and world-views (or
whatever) are undergoing continuous reorganization.

How'm I doing, you information theorists out there?

ยทยทยท

-----------------------------------------------------------
We can now see where Ashby made his mistake about "requisite
variety." He reasoned that in an error-driven control system
information from a disturbance must affect the regulator via its
(the disturbance's) effects on the essential variable, so the
disturbance information was used by the regulator as the basis
for creating opposing actions. This led to a paradox, because the
better the regulation, the less information about the disturbance
was available to direct the actions of the regulator. This led to
the idea that error-driven regulation must be inherently
imperfect -- not just in the technical sense that a tiny bit of
error had to remain, but in a much more serious sense. If the
regulator were to lose as much as half of the information about
the disturbance, how could it possibly cancel out more than half
of the disturbing effects? Ashby didn't actually reason that way
out loud, but the way he used the idea of error-driven regulation
subsequent to that argument showed that he didn't expect much of
it by way of precision.

Now we can see that the error-driven regulator is simply a
different beast from the disturbance-driven regulator. The
information that drives the output of the regulator comes from
the essential variable, not from the disturbance. In fact, the
effects of the disturbance are excluded from the control loop;
the information about the disturbance is rejected. An
information-theoretic analysis of the error-driven regulator must
start froms scratch -- it isn't simply an extension of the
analysis of the disturbance-driven regulator.
--------------------------------------------------------------
Best,

Bill P.

[Martin Taylor 930315 19:30]
(Bill Powers 930314.0900)

Bill,

I think we need to stand back a bit. I see you as getting tangled in
your own rhetoric. Probably you see me the same way. But your posting
on how information about the environment gets into the organism seems
to solve an unnecessary problem. Yes, we know that information gets
in through uncontrolled perceptions, but the same is true of controlled
perceptions. The information is about the state of the world, whether
it be controlled or not. Whether this contains information about
disturbances is irrelevant, but that is what has been giving you
heartburn.

If a perception is controlled, there is information that it is so, or
that there is error that will lead to action. But the perceptual signal
flows up through the layers of ECSs whether or not they are actively
controlling.

Did you read my "information for action" paper yet? In it, I describe
four modes of perception, one of which would be controlled in PCT terms,
the other three being through uncontrolled perceptions.

However, you are now expressing something of what I have been trying
to get across from time to time when I return to the degrees-of-freedom
argument:

Uncontrolled perceptions may be uncontrolled at many successive
levels, but eventually they join with controlled perceptions to
become inputs to a high-level control system. There they act as
disturbances of the perception to which they contribute, _and
their information content is therefore rejected by that system_.

Except for that last (emphasized) clause, that's an aspect of what
I have been saying. I'd even agree with the last clause, if "countered"
were substituted for "rejected."

The actions of that system that oppose the effects of the
uncontrolled perception result (after passing through layers of
reference signals on their way toward the periphery) in
variations of visible controlled variables. Those visible
variations carry the information about what the highest-level
system involved did to cancel the disturbing effect of the
incoming information. And perception of them joins with the
uncontrolled perceptions to leave the highest-level perception
undisturbed.

Wouldn't it be nice if we could resolve all our misunderstandings
so easily?

This works as long as the adjustments of controlled perceptions
suffice as a way of rejecting the information and maintaining the
world-view perception. But if the right kind of information is
received, control may become more and more difficult, or fail.
Then the person must start to reorganize -- find another world
view and other ways of adjusting lower reference signals in which
the adjustments will again succeed in preventing a disturbance
from the uncontrolled information input.

Reorganization if necessary, but not necessarily reorganization :wink:
(Sorry for the Canadiam in-joke).

Before reorganization, the normal effect would be to control some of
those otherwise uncontrolled perceptions. That's part of the function
of the alerting system. What we control changes all the time, out of
our vast repertoire of what we can control. Reorganization presumably
happens all the time when there is ongoing uncontrolled error, but if
a shift in what is being controlled fixes the problem, reorganization
will stop as if it had succeeded. My guess is that a reorganization
solution to the problem is normally quite rare. But I don't know how
I would test that.

Come to think of it, how does one test for reorganization in a human,
anyway? And how does one fit the model to it so that less than 5% of
the time it fails to predict the timing and the result of the reorganization
event?

Martin