[From Peter Cariani (2000.09.21.1130 EDT)]
Hi, everyone. My apologies for the tardy response -- I get the
DIGEST version a day late, and didn't have time yesterday to
read and respond. If you could cc: to me directly your
responses to me along with posting to the group, that would
be much appreciated.
I'm very happy my talk was thought-provoking!
Yes, when I talk about multiplexing I mean it in the sense that
many different signals can be sent through the same channels.
This is different from a system that has separate dedicated
channels for each different kind of signal and which takes the
value of each signal as some scalar quantity (e.g. average firing rate
or signal rms over some integration period).
I agree that what we experience is the representation itself and that
no homunculus is needed to "decode" the neural signals. (My apologies
for the use of "code" and "representation" language -- different
people have different connotations of these words & concepts, I know,
but they are the common words that have the closest meanings to the notion
that acoustic forms are transformed into neural signal patterns in
an orderly way by the ear and the central auditory system. There is
always the tradeoff between using more general words that people
understand vs. specific ones that one defines.)
While we don't need to decode the neural signals in order to
directly experience them, sometimes we do need to attend to
particular aspects of a sound (if we are trying to match the
pitches of two instruments vs. matching their timbres or
loudnesses or perceived locations). In this case, then, we do
need to separate/extract one kind of signal from the
multiplexed aggregate (my working hypothesis is that
different aspects of the population response subserve
the representation of these different, independent
perceptual qualities). It may be that the signals themselves
don't need to be separated out, but different analysis
operations could be done on the multiplexed aggregate.
I am highly sympathetic to the idea of "lower" levels
of processing (the PCT ladder of control) and bidirectional
influence between levels. My two high level working hypotheses
are: 1) that there are central time codes that are multiplexed
and the fine information is available at central stations, and
2) that central stations organize the lower stations to carry out
pattern comparisons (e.g. by opening up gates that permit
signals to reverberate and be compared with incoming ones).
There are advantages and disadvantages to either scheme --
the second one is closer to the PCT hierarchical structure,
except that instead of scalar input and reference signals, one
has multiplexed signals that are being compared. In order
to focus attention on one or another aspect, different
analyses of the multiplexed signal that comes out of the
comparison process need to be effected.
In the traditional PCT scheme:
What happens next is that a hundred, or a thousand or more input functions
of the second level pick out of the intensity information a large number of
auditory sensations such as pitches and loudnesses. These input functions
act like filters which are tuned to particular frequencies, and they all
respond in parallel, each to the aspect of the basic intensity signals to
which they are individually sensitive. These new signals then pass to the
third level where they enter input functions concerned with sound
configurations, such as chords and discords or the characteristics waveform
shapes of instruments, individual strings, and voices. These configuration
signals become inputs to the level of transitions where dynamics are
recognized, and so forth up the hierarchy to the perceptions of more
sublime aspects of the Jesu, many of which may exist only in the listener's
brain. We do not, unfortunately, have any recordings of Dinu Lipatti's
higher-level perceptual signals.
The problem is that there don't seem to be nice, separate channels for the
different frequencies, especially at high levels where the frequency response
areas of most neurons are very broad. Now there are some complicating
factors here, so it is not possible to rule out some kind of covert and/or
specialized scheme where only neurons 205, 556, 891, and 903 fire
maximally when a 1000 Hz tone at 87 dB SPL is presented. Such coding
schemes are not robust (the pattern of activity will change with small
changes in
level and/or the addition of background sounds), and they don't handle
combinations of stimulus atributes very well (1000 different pitches at
100 different levels in 100 different places in space, etc. etc. etc.).
A multiplexed system can handle this combinatoric explosion because the
different stimulus attributes are represented by different aspects of the
population response (one no longer needs to know that
neurons 205, 556, 891, and 903 fire maximally when a 1000 Hz tone at
87 dB SPL is presented, but neurons 201, 337, 406, and 908
fire maximally when the stimulus is 1050 Hz at 93 dB SPL.)
I think form perception is more primitive than intensity perception, that
forms are not built from an array of perceived intensities, but that
forms are represented in an iconic manner that reflects the temporal
correlation structure of the acoustic signal. One can infinite peak
clip speech signals with very little loss of intelligibility -- the amplitudes
are not as important as the zero-corssings. It's very hard to escape
the "pixel" concept of sensory coding and its close relative the atomic "feature"
(and most of psychology and neuroscience lies unaware that this is the
tacit assumption), which
depends on the detection of unitary events (the intensity of each pixel)
taht become the perceptual atoms of the system. The alternative is
a perceptual system based on relations as primitives (e.g. an interspike
interval being a relation, a temporal correlation, between two spike
events).
I think such a view of perception is compatible with PCT-style
control. THe comparisons are then correlation operations between
patterns rather than comparisons between scalars. This may seem
more complicated, but it actually simplifies enormously the
problems related to form recognition in a control system.
Obviously, I (we) should build such a system and try it out,
even if only in simulo. Time is in such short supply for me
these days, but eventually it will happen.
Best to everyone,
Peter
Peter Cariani, Ph.D.
Eaton Peabody Laboratory of Auditory Physiology
Massachusetts Eye & Ear Infirmary
243 Charles St., Boston, MA 02114 USA
tel@EPL (617) 573-4243
tel@MGH (617) 726-5419
FAX (617) 720-4408
Email peter@epl.meei.harvard.edu
Web: www.cariani.com
Date: Tue, 19 Sep 2000 03:26:41 -0600
From: Bill Powers <powers_w@FRONTIER.NET>
Subject: multiplexing[From Bill Powers (2000.09.19.0137 MDT)]
What woke me at this hour was a thought about Peter Cariani's concept of
"multiplexing" of neural "codes." I haven't liked either term, especially
not "coding" because I think it gives a false picture of the relationship
between experience and reality. What we experience _are_ the codes: no
decoding necessary.I finally got a flash, howeever, of what Peter may may have in mind when he
says "multiplexing." Multiplexing is a term from communications
electronics; it means putting many different signals into a single channel
(a single wire) by using some sort of orderly time-sharing or
bandwidth-sharing scheme that permits splitting the combined signals into
their individual channels (demultiplexing) at the destination, recreating
the original independent signals.Peter Cariani is interested mainly in auditory perception, where an effect
like (but only like) multiplexing is very prominent. What got me started on
this was a little mental replay of a segment of one of my favorite CDs that
drifted through my half-waking mind. The CD is a re-release of a recording
made in the 1940s by Dinu Lipatti of the Myra Hess piano arrangement of J.
S. Bach's "Jesu, joy of man's desiring." This is a melody played against a
background of running eighth notes in 6/8 time, the melody being a stately
slow progression of dotted fourth notes (one for every three notes of the
obbligato), with an even slower pattern of bass notes to add another
harmony line. These three lines are woven together, the emphasis shifting
back and forth among them with various themes coming and going, and
appearing in different registers (octaves).The thing about this wonderful piece is that it was entirely captured on a
78 RPM hill-and-dale recording which I still have. In other words, this
whole complex piece was reduced to a series of up-and-down waves in a
groove in a piece of plastic, a completely one-dimensional medium. I can't
think of a better example of the intensity level of perception. All of the
melody lines, the changing sonorities, the voicing of the notes (the
emphasis in loudness and continuity), and the harmonies, together with the
varying dynamics, are represented by a single physical signal in which
there is only one dimension of variation: up and down. The up and down
variations are changed electronically into peaks and valleys of air
compression, which travel from a loudspeaker through the air and enter the
ear, where they vibrate the cochlea and are translated into neural signals
at the lowest level of perception.Clearly there was some sort of multiplexing that went on here. It started
with Dinu Lipatti focusing his eyes on a page of music where the notes to
be played were represented in symbolic visual form. Somehow Dinu's brain
converted the visual symbols into a series and parallel set of
reference-sounds, in which there were (at least) three melody lines, which
were woven together by striking and pressing keys harder and less hard and
for longer and shorter durations as a mere ten fingers packed the melodies
into the piano. It's almost impossible to express the complexity of this
piece -- many people including me think that Lipatti was the greatest
pianist ever to have lived (he died in his 30s), and that his rendition of
the "Jesu" the most perfect ever accomplished. This particular job of
multiplexing is so flawlessly done that to the listener it sounds like (at
least) three different people playing together. When, as often happens, two
of the melody lines happen to require playing exactly the same note at the
same time, the melodies pass into the conjunction and out again without the
slightest bobble, in spite of the fact that one of the melodies requires
the note to be played in the foreground (to be "voiced") while the other
requires it to be played softly. The listener's brain effortlessly sorts
out the two flows without any confusion, just as Lipatti flawlessly put
them together with his ten fingers (a great deal would be explained if he
had had about twenty fingers, but I have seen pictures proving that he did
not).So here we have the multiplexing and demultiplexing phenomenon -- only the
multiplexing took place outside the brain, while the whole brain of the
listener was engaged in the demultiplexing. Peter described an image in
which the multiplexed signal rose through various layers of the brain while
various receiving circuits picked off the individual messages, leaving the
rest in place to rise to still higher centers. But the HPCT model offers a
different image. Not drastically different, but different enough.At the lowest level we have sensory nerves generating signals that so
resemble the incoming one-dimensional air-waves that an electrode in the
auditory nerve of a cat can, I have read, pick up signals that, amplified
and converted to sound again, are recognizable as music or voices. Although
spread out over different nerves, the basic intensity information is still
there. But in the HPCT model, this level of information never gets any
higher in the brain.What happens next is that a hundred, or a thousand or more input functions
of the second level pick out of the intensity information a large number of
auditory sensations such as pitches and loudnesses. These input functions
act like filters which are tuned to particular frequencies, and they all
respond in parallel, each to the aspect of the basic intensity signals to
which they are individually sensitive. These new signals then pass to the
third level where they enter input functions concerned with sound
configurations, such as chords and discords or the characteristics waveform
shapes of instruments, individual strings, and voices. These configuration
signals become inputs to the level of transitions where dynamics are
recognized, and so forth up the hierarchy to the perceptions of more
sublime aspects of the Jesu, many of which may exist only in the listener's
brain. We do not, unfortunately, have any recordings of Dinu Lipatti's
higher-level perceptual signals.This concept is actually quite different from the concept of multiplexing
that works in terms of time or frequency-band sharing. There is no
interleaving of different kinds of information in the basic intensity
recording. That recording is simply a continuous representation of
fluctuations of air pressure caused by the piano. There is no subchannel
carrying pitch information or harmony information or melody information.
All there is is the continuous recording of air pressure.Perhaps the concept of coding comes into this because it seems that
something in Lipatti's brain is encoded into sound-waves and then neural
firings, so that what was encoded in Lipatti's mind can be decoded again
and appear in the listener's mind. But the PCT model is not organized that
way. What the recording does is freeze into vinyl the air-pressure waves
emitted by the piano -- the same air waves that entered Lipatti's ears as
he played. Lipatti controlled those air waves by using his fingers and feet
on the piano, adjusting them so that as the resulting neural signals passed
upward through all the levels of his brain, each level experienced the
perceptions that were necessary for the next level of perceptions to be
controlled. As he did this, the lowest level of physical effects was
recorded. When that recording was later played back by someone else, very
nearly the same sound waves entered the listener's ears, and passed upward
through the listener's levels of perception, where the listener experienced
whatever the listener's brain was organized to get out of the basic sound
waves. If the listener had been present while Lipatti played, the
perceptions would have arisen level by level in both brains at the same
time, driven simltaneously by the piano, although they would by no means
have been exactly the same perceptions. I know that it took me many times
through the Jesu recording to perceive all that was going on, and I am sure
I am still missing a great deal. I may, of course, also be experiencing
things that Lipatti did not experience or control as he played.The main point here is that nothing passed out of Lipatti's brain and into
mine. That is why there is no encoding and decoding. Lipatti created a
series of physical effects which _both of us_ perceived -- I perceived them
anywhere from a year to 50 years later, of course, although if I had been
in the recording studio with Lipatti, we both would have perceived them
simultaneously.Notice that this applies to all auditory communication, including speech.
Nothing passes from the speaker's brain into the hearer's brain. Instead,
the speaker controls a physical utterance in such a way as to experience,
at many levels, what the speaker intends to experience. The same physical
utterance is heard simultaneously by a listener, and it gives rise to many
levels of perception in the listener's brain, according to the way the
listener's perceptions have come to be organized. Nothing is encoded by the
speaker for the listener to decode. Speaker and listener, separately, make
what they will of the same series of sound-waves.In fact this applies to all communication, doesn't it, including the very
words I am causing to appear here on this screen, and which will appear on
your screen at some later time. If you were reading over my shoulder as I
type, you and I would be recognizing the words and deriving meaning from
them simultaneously. Of course only I would know why I pause so often,
delete a phrase, and type something similar but not quite the same. If you
read only the final product, you don't even know I did that.What do you think of this, Peter?
Best,
Bill P.
------------------------------
Date: Tue, 19 Sep 2000 03:28:58 -0600
From: Bill Powers <powers_w@FRONTIER.NET>
Subject: Re: The Informant[From Bill Powers (2000.09.19.0327 MDT)]
Bruce Gregory (2000.0918.1433)--
>Rick Marken (2000.09.18.1045)
>
>You really are clueless, aren't you. This latest effort to demonstrate
>that you are right shows that you have learned nothing from your
>experiences on CSGnet.I consider this a completely counterproductive post. Don't you, on reflection?
Best,
Bill P.
------------------------------
Date: Tue, 19 Sep 2000 03:39:28 -0600
From: Bill Powers <powers_w@FRONTIER.NET>
Subject: Re: Domjan et al.[From Bill Powers (2000.09..19.0330 MDT)]
Bruce Gregory (2000.0918.1640)--
>> At the end of
>> this process is a final model, the one that correctly represents the
>> organization and predicts its behavior under any specific
>> circumstances to
>> the limits of measurement.
>
>That's what I'm disputing. I'll change my mind when such a model exists.My point is that if we follow a process of testing and refining models, at
the end of the process there will be a model with perfect descriptive and
predictive power. I'm not saying we have such a model yet (how long has
anyone consciously been trying to follow this procedure?). Do you dispute
that pi, the ratio of the circumference of a circle to its diameter, has a
specific exact value, despite the fact that nobody has yet computed all the
digits in its expression? To describe a process that produces a convergent
infinite series can be done without demonstrating the entire series.>(Of course all we can know is that up to now the model has never let us
>down. That was true of Newton's model for at least two hundred years. So
>how long should we wait before declaring the model is "final"?)There's no need ever to declare it final. The point is that we can tell
when a model is getting better and when it's getting worse, and roughly by
how much.>> By "correctly" I mean simply that
>> the structure
>> and behavior of the final model is exactly isomorphic to the
>> structure and
>> behavior of the real system.>So far we haven't developed any such models. Clearly you are more
>optimistic than I am.Optimism has nothing to do with it. Basically, all I'm saying is that
reality has exactly one shape, not a shape that changes with every
theoretical fad.
>> And if two models have contradictory properties, then at
>> least one of them must be wrong, since the real system can have no
>> self-contradictory properties.
>
>Such as being both a particle and a wave?Yes, to the extent that we consider these properties "contradictory." Both
models are probably wrong.Best,
Bill P.
------------------------------
End of CSGNET Digest - 18 Sep 2000 to 19 Sep 2000 (#2000-214)
*************************************************************
u a m