Neural impulse timing and frequency

CSGnet Archive CSG1997
1

[Martin Taylor 970926 10:30]

Bill Powers (970926.0419 MDT)]

The most realistic model of a neuron is a
frequency-to-chemical-concentration-to-frequency converter.

I think a lot of neurophysiologists would have considerable problems with
this statement. The relative timings of impulses are extremely important,
at least in respect of synaptic "learning". Two papers in _Science_ this
spring are especially relevant. H. Markram et al., Science, 275, 213 (1997),
showed that if a postsynaptic action potential (AP) closely follows an
excitatory postsynaptic potential (EPSP) in coupled neocortical pyramidal
neurons, the EPSP amplitude increases ("Hebbian" learning (2)), whereas
if the timing relation is reversed, the EPSP amplitude decreases
("anti-Hebbian" learning).

De Ruyter van Steveninck et al., Science 275, 1805, (1977), showed that
the timing patterns of pulse trains (in a motion-sensitive neuron of the
fly's visual system) is highly reproducible for naturalistically moving
visual patterns.

Both papers argue that it is the timing of the individual neural impulse
that matters, rather than the frequency of impulses.

I have to confess to a personal interest in this matter, since 24 years
ago I wrote a paper (M. M. Taylor, South African J. Psychol., 3, 23,
(1973)) in which I argued that if the timing of neural impulses
was consistent (as the van Steveninck paper says it is) and if both
Hebbian and anti-Hebbian learning occurred (as the Markram paper says
it does), then lateral inhibition among neurons at the same level (as
we would call it in HPCT) would give rise to an efficient neural
representation of input patterns. Specifically, it would generate a
near-equivalent to a principal-components representation of the set of
inputs to which the level was exposed.

A principal components representation is the maximally efficient
representation of a _set_ of patterns, not of any one pattern. Within the
HPCT context, if the perceptual functions at any level consisted not of
templates for individual possible percepts (as much of the discussion
presupposes) but of orthogonal feature representations, perceptual
control would be just as easy as with single-wire representations. It
would have the added advantage of requiring no new learning for the
control of novel perceptions built from features already developed through
reorganization.

That's all by the way, extra to the point that the "most realistic"
model of neural functioning apparently takes into account the precise
timings of incoming and output impulses.

Martin

2

[From Bill Powers (970926.1035 MDT)]

Martin Taylor 970926 10:30]--

The most realistic model of a neuron is a
frequency-to-chemical-concentration-to-frequency converter.

I think a lot of neurophysiologists would have considerable problems with
this statement.

Yes, I know.

The relative timings of impulses are extremely important,
at least in respect of synaptic "learning". Two papers in _Science_ this
spring are especially relevant. H. Markram et al., Science, 275, 213
(1997), showed that if a postsynaptic action potential (AP) closely

follows >an excitatory postsynaptic potential (EPSP) in coupled neocortical
pyramidal

neurons, the EPSP amplitude increases ("Hebbian" learning (2)), whereas
if the timing relation is reversed, the EPSP amplitude decreases
("anti-Hebbian" learning).

An interesting finding, but the experiment was done with single impulses.
The timing relationship can also be seen as phase relationships in a
frequency domain. And anyway, who says this change is "learning?" Does
learning involve only increases in synaptic weights?

De Ruyter van Steveninck et al., Science 275, 1805, (1977), showed that
the timing patterns of pulse trains (in a motion-sensitive neuron of the
fly's visual system) is highly reproducible for naturalistically moving
visual patterns.

But all you have to do is plot 1/interval, and you will have a frequency
pattern that will also be highly reproducible. This is nothing more than a
change of variable, substituting 1/f for interval and vice versa. The
angular echolocation in a bat's perceptions can be seen as depending on
small delay differences, or small phase shifts -- whichever strikes your
fancy. A delay between one impulse and another can also be see as a phase
shift between modulated impulse trains.

One reason I prefer the frequency domain is that frequency measures seem a
more natural representation of quantities that range from zero to some
maximum value. When the amount of an input goes to zero, the frequency of
the signal representing it goes to zero, but the interval goes to infinity.
It's awkward to propose that a signal that is subjectively increasing is
actually decreasing in magnitude, and that when the subjective impression
goes to zero, the signal is actually infinite, or at least at its maximum
value.

Both papers argue that it is the timing of the individual neural impulse
that matters, rather than the frequency of impulses.

I don't see how they can tell the difference, since the one measure is just
the inverse of the other. I'm sure it is more convenient to speak in terms
of intervals when the experiments employ single impulses, but in a more
natural situation with trains of impulses, it really doesn't matter,
mathematically, which representation you use.

I have to confess to a personal interest in this matter, since 24 years
ago I wrote a paper (M. M. Taylor, South African J. Psychol., 3, 23,
(1973)) in which I argued that if the timing of neural impulses
was consistent (as the van Steveninck paper says it is) and if both
Hebbian and anti-Hebbian learning occurred (as the Markram paper says
it does), then lateral inhibition among neurons at the same level (as
we would call it in HPCT) would give rise to an efficient neural
representation of input patterns. Specifically, it would generate a
near-equivalent to a principal-components representation of the set of
inputs to which the level was exposed.

Sounds very powerful. How did the computer simulation work out?

Best,

Bill P.

3

[Martin Taylor 970927 16:00]

Bill Powers (970926.1035 MDT)]

The relative timings of impulses are extremely important,
at least in respect of synaptic "learning". Two papers in _Science_ this
spring are especially relevant. H. Markram et al., Science, 275, 213
(1997), showed that if a postsynaptic action potential (AP) closely

follows >an excitatory postsynaptic potential (EPSP) in coupled neocortical
pyramidal

neurons, the EPSP amplitude increases ("Hebbian" learning (2)), whereas
if the timing relation is reversed, the EPSP amplitude decreases
("anti-Hebbian" learning).

Does
learning involve only increases in synaptic weights?

No. See the passage you quoted above.

De Ruyter van Steveninck et al., Science 275, 1805, (1977), showed that
the timing patterns of pulse trains (in a motion-sensitive neuron of the
fly's visual system) is highly reproducible for naturalistically moving
visual patterns.

But all you have to do is plot 1/interval, and you will have a frequency
pattern that will also be highly reproducible.

The point of the experiment was that the movement patterns were _ir_regular.
A constant-velocity motion produces irregular and non-reproducible impulse
intervals.

You can't really talk about "frequency" except over a substantial time
period, technically an infinite time-period. But we do talk about short-
time "frequency" often enough that we forget this point. However, when
you get to the situation where you have interval sequences such as
short-short-medium-short-long... that come out pretty much the same
when the same pattern of movement occurs again, frequency seems not to
be a very useful concept.

This is nothing more than a

change of variable, substituting 1/f for interval and vice versa.

Are you saying that the forgoing sequence could equally be seen in "frequency"
as high-high-medium-high-low? I think that would be quite a perversion of
the notion of "frequency", even worse than the approximation implied by
a "short-time" Fourier transform.

One reason I prefer the frequency domain is that frequency measures seem a
more natural representation of quantities that range from zero to some
maximum value.

I would have thought that any measure that was constrained to be positive
would be equally appropriate--or even a measure without that constraint.
The problem with frequency and its various approximations is that is always
represents an average over some extended time period. That's fine so long
as the thing represented stays more or less constant during the averaging
time, but it's not so fine when the thing represented changes quickly.

When the amount of an input goes to zero, the frequency of
the signal representing it goes to zero, but the interval goes to infinity.

Well, that's not true of a lot of neurons, as you pointed out to someone
the other day. When the input goes to zero, the neuron fires at some
"resting" rate.

It's awkward to propose that a signal that is subjectively increasing is
actually decreasing in magnitude, and that when the subjective impression
goes to zero, the signal is actually infinite, or at least at its maximum
value.

There are lots of ways to deal with this awkwardness, which clearly exists
for the analyst who wants to use words to describe what is going on. For
example, one could propose that the "effect" of an impulse is provided
by a leaky integrator, so that for closely spaced impulses it grows
fast to some saturation value while for more widely spaced impulses it
drops back substantially between them. But that would be a convenience,
saying nothing about what actually happens in the system being described,
for which there are continuous changes in electrical potentials and
chemical concentrations--no frequencies, no impulse intervals.

Both papers argue that it is the timing of the individual neural impulse
that matters, rather than the frequency of impulses.

I don't see how they can tell the difference, since the one measure is just
the inverse of the other.

Only when the intervals are reasonably equal over some extended period of
time.

I think frequency is a perfectly reasonable surrogate for the signal
that is controlled in a control loop, provided we are talking in gross
terms about the properties of the loop--bandwidth, gain, and so forth.
It's only if we were to get into fine details of behaviour that the
difference might matter. And, I think, when we talk about learning and
reorganization.

···

-------------------

I have to confess to a personal interest in this matter, since 24 years
ago I wrote a paper (M. M. Taylor, South African J. Psychol., 3, 23,
(1973)) in which I argued that if the timing of neural impulses
was consistent (as the van Steveninck paper says it is) and if both
Hebbian and anti-Hebbian learning occurred (as the Markram paper says
it does), then lateral inhibition among neurons at the same level (as
we would call it in HPCT) would give rise to an efficient neural
representation of input patterns. Specifically, it would generate a
near-equivalent to a principal-components representation of the set of
inputs to which the level was exposed.

Sounds very powerful. How did the computer simulation work out?

That question is sensible to ask these days. It would not have been so at
the time. It might even be feasible to try it now. However, the argument
was quite straightforward, and I would be rather surprised if it didn't
work out. As in most such simulations, the big problem would be in defining
an environment suitable for testing the system. Simulating the environment
is often 90% of the work, as I'm sure you have often discovered.

Martin

4

[From Bill Powers (970927.1518 MDT)]

Martin Taylor 970927 16:00--

De Ruyter van Steveninck et al., Science 275, 1805, (1977), showed that
the timing patterns of pulse trains (in a motion-sensitive neuron of the
fly's visual system) is highly reproducible for naturalistically moving
visual patterns.

But all you have to do is plot 1/interval, and you will have a frequency
pattern that will also be highly reproducible.

The point of the experiment was that the movement patterns were _ir_regular.
A constant-velocity motion produces irregular and non-reproducible impulse
intervals.

Fine. Then constant-velocity movement will also produce irregular and
non-reproducible frequencies of firing.

You can't really talk about "frequency" except over a substantial time
period, technically an infinite time-period.

I don't know where you got that rule. I've never heard of it.

But we do talk about short-
time "frequency" often enough that we forget this point.

Yes, indeed we do. Perhaps it deserves to be forgotten.

However, when
you get to the situation where you have interval sequences such as
short-short-medium-short-long... that come out pretty much the same
when the same pattern of movement occurs again, frequency seems not to
be a very useful concept.

This is nothing more than a

change of variable, substituting 1/f for interval and vice versa.

Are you saying that the forgoing sequence could equally be seen in
"frequency" as high-high-medium-high-low? I think that would be quite a
perversion of the notion of "frequency", even worse than the approximation
implied by a "short-time" Fourier transform.

I agree; that would be pretty hard to represent plausibly as a change in
frequency. However, if this pattern were repetitive, representing it in
frequency terms would be perfectly OK. Somehow I think those "naturalistic
movements" must have resulted in repetitive patterns, not just a brief
burst of five impulses.

One reason I prefer the frequency domain is that frequency measures seem a
more natural representation of quantities that range from zero to some
maximum value.

I would have thought that any measure that was constrained to be positive
would be equally appropriate--or even a measure without that constraint.

Not for correlating with subjective experiences, or in psychophysics.

The problem with frequency and its various approximations is that it always
represents an average over some extended time period. That's fine so long
as the thing represented stays more or less constant during the averaging
time, but it's not so fine when the thing represented changes quickly.

I don't know where you get this "extended period of time." Pulse frequency
modulation is a perfectly good way to transmit a varying signal, with the
upper frequency limit being set by the Nyquist criterion -- one cycle for
every two impulses.

When the amount of an input goes to zero, the frequency of
the signal representing it goes to zero, but the interval goes to infinity.

Well, that's not true of a lot of neurons, as you pointed out to someone
the other day. When the input goes to zero, the neuron fires at some
"resting" rate.

Didn't I add (or some maximum interval?) I thought I did, if not here then
elsewhere. This is true of _some_ neurons. Can we agree that when the
amount of input goes to zero, the pulse interval representing it goes to a
very big number?

It's awkward to propose that a signal that is subjectively increasing is
actually decreasing in magnitude, and that when the subjective impression
goes to zero, the signal is actually infinite, or at least at its maximum
value.

There are lots of ways to deal with this awkwardness, which clearly exists
for the analyst who wants to use words to describe what is going on.

You're not going to catch me with that one. My reply is ( ).

For
example, one could propose that the "effect" of an impulse is provided
by a leaky integrator, so that for closely spaced impulses it grows
fast to some saturation value while for more widely spaced impulses it
drops back substantially between them. But that would be a convenience,
saying nothing about what actually happens in the system being described,
for which there are continuous changes in electrical potentials and
chemical concentrations--no frequencies, no impulse intervals.

Yes. The mean value of the leaky integrator output would be maximum when
the interval is least. Of course it might take several cycles to determine
the mean value...

True, there are neither frequencies nor intervals in the real system. There
are no voltages or potentials or chemical concentrations, either. All of
those things are conventions for measuring and describing phenomena. So what?

Both papers argue that it is the timing of the individual neural impulse
that matters, rather than the frequency of impulses.

I don't see how they can tell the difference, since the one measure is just
the inverse of the other.

Only when the intervals are reasonably equal over some extended period of
time.

Why not say that if pulse 2 comes x milliseconds after pulse 1, the
equivalent frequency is 1000/x Hz? That's an "extended period of time."

I think frequency is a perfectly reasonable surrogate for the signal
that is controlled in a control loop, provided we are talking in gross
terms about the properties of the loop--bandwidth, gain, and so forth.
It's only if we were to get into fine details of behaviour that the
difference might matter. And, I think, when we talk about learning and
reorganization.

Well, that's a relief. I don't have to think about it.

-------------------

I have to confess to a personal interest in this matter, since 24 years
ago I wrote a paper ...

Sounds very powerful. How did the computer simulation work out?

That question is sensible to ask these days. It would not have been so at
the time.

Twenty-four years ago was 1973. I was doing computer simulations at that
time. So were lots of other people.

It might even be feasible to try it now. However, the argument
was quite straightforward, and I would be rather surprised if it didn't
work out.

Might be a good idea to try it. That would be more persuasive than just
saying that you find your own idea convincing.

Best,

Bill P.