Bandwidth

[from Gary Cziko 921026.1205 GMT]

re. Bill Powers (921025.0800)

Bill,

Thanks so much for your response to my speed and deafferentation questions.
And thanks for your patience. I know you've said all this stuff (probably
many times) before, but it may take a while (and a few repetitions) before
it sinks into the brains of us non-engineer-background PCTers. I'm sure
many people on the net found it useful as well.

But there is a word which often creeps up in technical discussions
(especially between you and Martin Taylor) which I don't yet feel I have a
good understanding of--bandwidth. To wit:

This is exactly what H. S. Black discovered in 1929. He found that
vacuum tube amplifiers with a certain inherent gain and bandwidth
could be used to achieve not only far more stable gain but a much
wider bandwidth, through the use of negative feedback.

I have a pretty good idea of how radio works (at least a better
understanding than Fred's in Greg Williams's _Fred & Bill_ story), so for
me bandwidth refers to the amount of frequency spectrum a signal takes
up--for example broad-brand FM (normal commercial broadcasting of moldy
oldies) takes up more bandwidth than narrow-band FM (as used in police and
amateur radio communications). But when you guys talk about it, you seem
to refer to the amount of information being transmitted. In radio, the
amount of information is the same from the perspective of words, but more
from the perspective of fidelity (normal FM broadcasting has much higher
fidelity than narrow-band).

Perhaps you could apply the bandwidth concept to an ECS and its meaning
might become clearer for me.--Gary

P.S. Greg, thinking of Fred and Bill made me think of the fast approaching
Christmas/New Year's season. Is this because you posted this story last
year around Christmas? Anyway, I like this story so much that I would like
it to become a Christmas/New Year's tradition on CSGnet. Perhaps you could
even make a few minor changes so that it fits the season better.

···

------------------------------------------------------------------
Gary A. Cziko Telephone: (217) 333-8527
Educational Psychology FAX: (217) 244-7620
University of Illinois E-mail: g-cziko@uiuc.edu
1310 S. Sixth Street Radio: N9MJZ
210 Education Building
Champaign, Illinois 61820-6990
USA
------------------------------------------------------------------

[Martin Taylor 921026 12:30] Back today only, then gone for a week or so.
(Gary Cziko 921026.1205)

Gary,

Bandwidth is an interesting issue to bring up. In a linear system, it is
quite easy to deal with, in a simplified form, but even in linear systems
there are subtleties that are not always evident. In a non-linear system
the concept is almost metaphoric, but useful. I have tended to mix the
technical with the metaphoric in my postings, because I know of no easy
corresponding concept other than informational channel capacity that works
in a non-linear system. I'll try to give a brief explanation that applies
to a linear system, and to indicate why it matters.

···

----------
We start with the idea that things vary over time, and we will consider only
a scalar variable x (x can at any moment be described by a single real
number between minus infinity and plus infinity). The variation over time
of this variable is called its waveform, x(t). A couple of hundred years ago,
Fourier proved that x(t) can be described by a unique infinite sum of sinusoidal
variables. There are an infinite number of these, because to specify x(t)
over infinite time one needs to use all frequencies between zero (a steady
level) and upward further than any specified number. Frequency is the number
of oscillations of the sinusoid per second, so a high frequency means that
rapid variations in x(t) can be described. The set of parameters that specify
the magnitude and phase of the sinusoid at each frequency needed to match x(t)
is called the Fourier Transform of x(t), and is sometimes written as X(f).

If x(t) has no moments of very rapid variation, X(f) will have magnitude near
zero for all frequencies higher than some maximum value Xc (should be a
subscript c). The value of Xc is called the cutoff frequency of X(f) ("cutoff"
is sometimes used in other ways, so don't let it confuse you).

Certain kinds of waveform for x(t) have interesting and important Fourier
transforms. One, in particular, is central in our discussions. Call it
w(t) (w = white noise). W(f) is zero above a certain frequency Wc, and has
a uniform magnitude for all frequencies below Wc. W(f) is said to have a
rectangular spectrum. In addition, if you look at w(t) (that is, you sample
w(t)) at regular intervals spaced further apart than 1/2Wc the values you get
have zero correlation (at this point, that's a further part of the definition
of w(t), not a consequence of the shape of W(f)). One sample provides no
information about the distribution of values of the next. If you sample at
time intervals closer than that, the successive samples are necessarily
correlated, and you get no more information about the waveform w(t) than you
would have if you sampled at an interval 1/2Wc. Sampling at 1/2Wc is fast
enough to tell you everything there is to know about the waveform w(t), or
about any other waveform that uses no frequencies about Wc. 1/2Wc is called
"the Nyquist limit" or "the Nyquist rate" after its discoverer.

Most signals are not like w(t), in two ways. Firstly, in real signals there
is no fixed value of Wc above which the magnitudes of the frequency components
are all exactly zero. Rather, the magnitudes tail off over some range until
they become too small to detect. Secondly, the magnitudes below Wc are usually
not all the same. There are peaks and valleys in the magnitudes as a function
of frequency.

So far, I did not mention phase, but it is important, because it determines
the actual shape of the waveform x(t). If the phases (the moment at which the
sinuoid crosses zero can be used as an indicator) are in some easily described
relation to each other, then even if the spectrum is rectangular, like W(f),
there will be correlation among the successive samples of w(t) even if the
sampling is slower than the Nyquist rate. The waveform will convey less
information than will w(t). Of all signals with a Fourier transform
contained entirely within a cutoff frequency Xc, white noise (w(t)) requires
the most information to describe it. The amount of that information depends
on the precision with which each successive sample must be described.

In dealing with signals of less randomness than w(t), it is conventional to
talk about "equivalent rectangular bandwidth", which is the bandwith (i.e.
the cutoff frequency) of a white noise signal that would require the same
amount of information to describe it. So, when we talk about bandwidth, we
are talking about two things: (1) the fastest rate at which the samples of
an equivalent white noise could be sampled without forcing correlation among
the values of successive samples, and (2) the rate at which the signal can
convey information, which depends not only on the sampling rate of the
equivalent white noise, but also on the precision with which those samples
would have to be specified. In the second sense, we mean "information rate"
or "channel capacity" whereas in the first sense we are dealing with time-
related phenomena that will show up in specific application. For PCT purposes,
the most important is probably the intrinsic transport delay around a
feedback loop, which affects the stability of the loop. The wider the
bandwidth, the faster the loop.

The concept of bandwidth is often applied to filters. In this case, it means
that the filter reduces the magnitude of frequencies outside the "filter band."
An equivalent rectangular filter with a cutoff Fc would reduce to zero all
the frequency components of X(f) higher than Fc, leaving a waveform x'(t)
that could not carry any more information than a white noise with cutoff
Wc = Fc.

When we come to non-linear systems, the Fourier transform approach no longer
applies. But the concept of information transfer does apply, and so do the
questions of loop delay and stability. So we continue to use the word
"bandwidth" even though it is technically wrong to do so. However, it is
a good metaphor, and much of the time you won't go too wrong by thinking of
the effects of bandwidth in linear systems. Sometimes you will go wildly
wrong, but that's life.

Hope this helps.

Martin

(Today, according to the polls, we in Canada commit national Hara Kiri. For
what perception of honour does this control, Greg?)

[From Bill Powers (921027.0500)]

Gary Cziko (921026) --

In case Martin Taylor's interesting treatise on bandwidth left some
people scratching their heads:

Here's an example of the bandwidth of a control system. Hold up your
forefinger about 18 inches in front of your nose and move it slowly
from side to side over a total distance of three or four inches, like
a slow metronome. Now, keeping the average position and the amplitude
of movement the same, gradually speed up the movement, like a
metronome going faster and faster. Keep going faster until you
absolutely can't do it any faster. At that point you will be using
your whole arm, and you will feel quite large muscular efforts, even
though the movement from side to side is still only three or four
inches (try to keep it that way).

The fastest movement you can produce is at a frequency essentially
equal to the bandwidth of your finger position control system.
Obviously you can perform this back-and-forth pattern at any slower
speed (lower frequency) with no great difficulty, right down to zero
frequency (stationary finger). But when you try to produce an
oscillating movement at a frequency higher than the bandwidth, your
control system simply won't obey. You can _imagine_ a faster movement,
but you can't _produce_ a faster movement.

Why is there a bandwidth? One explanation might be that your muscles
simply can't reverse the motion of your arm any faster, because they
reach the limits of force that they can produce. If that were the only
limit, you ought to be able to move your finger faster if you move it
over a span of only a quarter of an inch instead of three to four
inches. The maximum force needed to maintain an oscillation goes as
the square of the frequency, so when you move your finger 1/10 as
much, you should be able to oscillate your finger about three times as
fast.

In fact, you can move perhaps a LITTLE faster, but certainly not three
times as fast. You can oscillate your finger with an amplitude of,
say, four inches or less at about 4 or 5 cycles per second, but not
significantly faster, even for the smallest movements (I assume you're
not a concert pianist, and anyway concert pianists don't have much
occasion to practice sideways trills).

If you _increase_ the amplitude to a foot or eighteen inches, you will
indeed find a decreasing speed limit set by muscle strength: the force
required increases linearly with amplitude in a linear system (which
your arm is not). At large amplitudes of movement you slow down
because your muscles won't produce enough force to maintain the same
frequency of oscillation you could maintain with a small amplitude.
But below a certain amplitude, the speed limit is no longer set by
muscle force. Something else is limiting the speed.

When you slowly speed up a small movement, keeping its amplitude the
same, you'll notice another phenomenon. At low frequencies, you see a
finger waving slowly back and forth. But at the highest frequency you
can produce, you can see the finger only at the end of each movement
where it reverses. Between those positions it's just a blur; you can
see right through it. Obviously you couldn't track anything with your
finger at that speed, because you couldn't see its movements, much
less track irregular movements of something else. What you're seeing
is the bandwidth of your visual perceptions of position. The frequency
at which your finger just ceases to be a blur and becomes a finger
again is the bandwidth of retinal position detection (actually you
have to suppress eye movement by fixating on the background to find
the true bandwidth, which is quite low, only 2 to 3 Hz).

It's interesting that the bandwidth or maximum frequency for small
movements is higher than the bandwidth for retinal position detection.
Something is limiting kinesthetic control at a frequency higher than
that at which position control takes place, but at a lower frequency
than is set by muscle strength. This probably involves a perceptual
limit, too, in that kinesthetic position sensors do have speed limits,
but more likely it is caused by temporal filtering that is required in
order to make the kinesthetic control systems (that position your
finger in the dark) STABLE.

The kinesthetic position control systems contain time delays of
something like 50 milliseconds of neural transit time and synaptic
delay around the loop. The muscles themselves have viscous damping.
The noisy nature of neural signals, trains of impulses, requires that
some smoothing take place in order to turn barrages of neural impulses
into smooth changes in neurochemical concentration levels. All these
factors mean that there is an unvoidable lag in these systems of about
100 milliseconds, part of it a transit-time delay and part of it an
integrative or smoothing lag. That would imply that to switch as fast
as possible from one position to another under kinesthetic control
should take a little longer than 100 milliseconds, and to switch back
another 100 milliseconds, for a total of 200 milliseconds for one
cycle of a repetitive movement. That would give a frequency for
continuous switching of 4 to 5 Hz, which is pretty close to what you
see when you do it. Not bad for a ball-park estimate.

You can easily see the relationship between speed of movement and
bandwidth. Try the experiment again, with small movements, only this
time switch as fast as possible from one position to another 4 inches
away, pause, then switch back as fast as possible, and pause. You're
trying to generate a square wave. At low frequencies, each switch is
discrete. You finger blurs over to the other position and is
stationary for a while, then blurs back again. But as you increase the
frequency of the square wave, still making each movement as fast as
you can, the movements begin to blend into a continuous movement, so
that when you reach the maximum frequency you're back to a continuous
sine-wave movement. In fact, even at the low frequencies, each switch
has been like half a cosine wave -- a high-frequency cosine wave at
just about the bandwith frequency. So the slow square wave you started
with was actually rounded off a little, and that rounding off meant
that the movements actually never exceeded the maximum bandwidth for
continuous oscillations.

It is possible for you to generate oscillations at higher frequencies.
The only way to do it, however, is to destabilize your spinal control
systems, the lowest level of control. If you press your hands together
very hard and maintain the push until the muscles begin to fatigue,
you may see "clonus" oscillations, at a frequency of about 8 to 10 Hz.
This results from changing the force-tension curve in the muscles
enough to make the control systems unstable. They break into
spontaneous oscillation. But you can't produce this kind of frequency
voluntarily. (You may see lower-frequency oscillations -- the next
level may get unstable first. Shivering is probably a clonus
oscillation of this kind, produced by destabilizing the control
systems in some other way. So climb naked into the refrigerator if you
want to see 10-Hz oscillations).

We're now approaching Rick Marken's territory. For visual tracking
using control of finger position to follow a target, you obviously
have to be able to see a finger while it's moving. This means that the
bandwidth for following a randomly moving target is about 2 to 3 Hz,
the frequency at which the finger just stops being a blur. This
bandwidth is set by perception and output functions, not muscles. The
kinesthetic systems clearly have a wider bandwidth; they can execute
faster movements than you can control visually. And the lowest level
of kinesthetic control, the spinal reflexes, have the widest bandwidth
of all.

What's most interesting to me is that these nested bandwidths are just
about what is necessary to maintain stable control at each level.
There would be no point in being able to see movements beyond a
bandwidth of 2 to 3 Hz because the kinesthetic control systems used by
a visual-motor control system have a bandwidth only slightly higher --
4 to 5 Hz. Therefore we DON'T see faster movements! In fact, if we
could see faster movements, the bandwidth of the visual control
systems would be so high that the lags of the lower control systems
would be too long for stable control at the higher level. In technical
terms, at a frequency where the phase shift of a sine-wave disturbance
passing around the loop is 180 degrees, the gain would still be above
1. Negative feedback would turn into positive feedback at that
frequency, and the whole system would oscillate. Oscillation is not
good for control.

Rick Marken has explored several of the higher levels of perception,
showing that as the (hypothetical) level increases, the bandwidth of
perception continues to decrease. This is only logical, once you do
some experiments yourself. For example, while moving your finger back
and forth as fast as you can, _vary the amplitude_ between, say, a
four-inch amplitude and a two-inch amplitude. Obviously, you can't
even SEE "amplitude" in a time smaller than the fastest oscillation.
And to VARY amplitude, you have to have a couple of oscillations of
each size. In principle you could do one large oscillation and one
small one, and so forth. In practice, you can't perceive changes in
amplitude that fast. So you can't control amplitude as fast as you can
control position. Rick's demonstrations are simple and elegant, as
usual, showing the effect clearly. So naturally he can't get them
published.

The relationships between bandwidths at different levels are, once you
understand why they exist, perfectly simple and logical. It seems that
bandwidth follows from physical principles and obvious relationships
among physical phenomena, such as between frequency and amplitude.
It's obvious that you can't change amplitude in less than one complete
cycle, because amplitude doesn't even exist until at least one cycle
is completed. Ho hum.

But remember that this is a constructed reality we're talking about.
This relationship holds because of the way we perceive amplitude as a
function of movements. Having constructed a perception of amplitude,
we then discover that it has properties, and that it is related to
lower levels of perception such as movement and position. The ho-hum
self-evident relationship suddenly becomes evidence about how
perception is constructed -- much more so than evidence about the
natural universe. The bandwidth relationships also tell us that higher
perceptions must be functions of lower ones, and that higher control
systems use lower ones to accomplish their control. The evidence just
continues to pile up that we are looking at -- and WITH -- a hierarchy
of perceptual control systems.

When is the world going to wake up to what is going on here?

···

--------------------------------------------------------------------