Rick Marken wrote:
I am curious about your ultimate conclusion:
that temporal coding "may lead
us to coding mechanisms that
drastically simplify (seemingly) complex sensory-
motor tasks". How could a neural coding mechanism
do this?
Since I am working as a sensory neurophysiologist, most of my
examples are on the sensory side.
Extracting the pitch of a complex sound is a very elaborate
task if one has to use representations of the sound based
solely upon average discharge rates. In the auditory
literature, these are called "rate-place" representations,
"place" being the position of the neuron in a tonotopic,
frequency map (or alternately, the "place" on the basilar
membrane to which the neuron is ultimately connected. In order
for the auditory system to distinguish the very definite low
pitch (100 Hz) of a harmonic complex consisting of frequency
components 1300, 1400, 1500, 1600, 1700, and 1800, the excitation
peaks corresponding to the frequency components must first be
resolved in the rate-place maps and then a
harmonic analysis must be carried out (comb filter) on the
estimates of where the peaks lie.
It is quite unclear how (or where) such a mechanism would be
carried out in the vertebrate auditory system, given the kinds
of neural elements and response types that are available. The
general problem of using scalar outputs of tuned neural maps is
that every time one wants to combine two or more sensory
qualities (e.g. pitch, timbre, loudness or color, texture, and
brightness), one needs another set of neurons to encode their
association, and the representation of multimodal objects
rapidly requires more and more associative elements (the number
goes up combinatorically if one tries to represent the entire
space of perceptual qualities, yet we can distinguish many
different pitches, timbres, colors, and tastes some of
which we may never have before experienced).
What makes the pitch problem so hard for rate-place schemes
is that in these processing schemes the relationships
between different frequencies is discarded once one does the
frequency analysis at the cochlea; after this analysis all one
has left are excitations in different channels, and one then
has the onerous task of reconstructing the harmonic structure
of the stimulus.
Alternately, there are other kinds of stimulus representations,
which in effect preserve harmonic structure by virtue of
a time domain encoding (such as interspike interval distri-
butions), in effect implementing an autocorrelation-like
representation in the time domain rather than a power
spectrum-like representation in a spatially organized map
(such maps do exist, but the relationship between time and
place representations is a long and intricate discussion).
If one looks at the distribution of interspike intervals
across the entire auditory nerve (50,000 fibres in cat,
30,000 in humans), the most common interval corresponds to
the pitch that would be heard by human listeners. This
observation holds for a very, very wide range of complex
stimuli and levels. Licklider's 1951 neural architecture,
with delay lines and coincidence detectors easily extracts
the pitch using such information, and as far as I can see
the best pitch detectors available right now are those that
use his principle. (As I understand it, the best speech
front-ends are similarly those which use simulated
interspike interval auditory nerve representations...).
Here is an example where preserving the temporal structure
of the stimulus allows one to extract stimulus qualities in
a much simpler way than by "pixelizing the stimulus" via an
array of specialized detectors and then trying to reconstruct
the global structure.
I have much, much less experience on the motor side than any
of you, but I can imagine temporally-patterned inputs influencing
the recruitment of various muscles to perform a task,
such that a multidimensional signal is sent to many different
muscles and they each respond in their own way. A rudimentary
example of this is the crayfish claw opener motoneuron,
which has one axonal branch which innervates one muscle which
controls the degree to which the claw is open and another branch
which innervates another muscle which controls the amount of
force applied. The axon branches have
different membrane recovery kinetics such that spikes are
propagated down the respective branches contingent upon their
recent history (e.g. interspike interval= 1st order history)
Different spike trains having different
interspike interval distributions can activate one or the other
branch independently by altering the mix of intervals in the
inputs. I can imagine how such an organization might support
"motor synergies" -- one then does not have to send specific
signals to each of a large number of cooperating muscles; one
changes the mix of input periodicities (or higher order
temporal patterns) in order to modify the output.*
The situation might be likened to the problem of gait control --
if you have systems that are already organized in such a way
that the coordination of the parts is the natural result,
then the problem of getting that system to behave in a desired
way (faster, slower relative to some reference) is drastically
simplified (fewer parameters need be specified to the subsystem
if it is appropriately organized). This notion is, I think,
well within in the general spirit of Powers' hierarchy of
control loops (although must the structure be a hierarchy?).
Again, none of this changes the basic nature of the feedback
loops, but instead of the various signals being scalars,
they are multi-dimensional and the steering is done in a
higher dimensional space.
It seems to me that the apparent complexity of
sensory-motor tasks is reduced
or eliminated once we discover the perceptual
variable(s) that are being
controlled in these tasks-- the seeming
complexity being a result of looking
only at the side-effects of the disturbance
resisting actions that control perception(s).
I certainly agree that extracting the relevant perceptual
variables being controlled does enormously simplify the
problem, but I am not sure that the appropriate variable(s)
can always be isolated without some consideration of what
their neural representation might be. In effect, I think
it is the neurally-implemented perceptual variables that
matter if one wants to explain the behavior of some biological
organism.
I wish I could be clearer and more explicit about what temporal
coding might mean on the motor side (any thoughts out there?),
but I'm still working out the basic conceptual framework,
how it all might hang together.
Peter Cariani
eplunix!peter@eddie.mit.edu
*A very provocative paper in this context is
Abeles et al (1993) Spatiotemporal firing patterns in the
frontal cortex of behaving monkeys. J. Neurophysiology
70(4) October, 1993, pp. 1629-1638.