[From Peter Cariani (960510, 10 AM EST)]
[From Rick Marken (960509.1110)] [re pitch models]
I would feel more confidence in this assessment
if I saw an actual working
model of a pitch control system based on temporal coding of pitch.
Could you write up an example program to do this?
Ciao, Rick,
I normally do my analysis on spike trains from recorded auditory
nerve fibers, but analogous operations can be done on the
signals themselves. To simplify things, remove any DC components
in the signals, by subtracting their mean value.
The first strategy is an autocorrelation-based approach:
1. Compute the short-term autocorrelation of the signal by taking the
signal X(t) and multiplying it by itself X(t-delay), adding each
term as you go to corresponding delay term in the the
autocorrelation function AC(delay). Increment a counter for each
delay term every time you do this, and at the end divide
each delay term in the unnormalized autocorrelation function by its
corresponding counter to get the normalized autocorrelation function
(or 'unbiased short-time autocorrelation function'). You only need
to compute delays in the range that you're interested in, e.g.
0-15 msec covers all periodicities from 66 Hz up to the Nyquist
limit (half the sampling rate of your signal). Set all near-zero
delay terms to 0, e.g. 0-0.5 msec. The highest peak in this
autocorrelation function corresponds to the low pitch that is heard,
which for a harmonic complex tone will be the fundamental period. If
you want to match by fundamental, zero everything out except for the
highest peak(s), which correspond to 1/F0 and its multiples (2/F0,
3/F0, etc.). If you want to do general stimulus matching, don't
zero out anything (but you may match on common partials).
This operation is neurally roughly equivalent to
a) the population interspike interval distribution at the level of
the auditory nerve, for stimulus periodicities < 4-5 kHz and/or
b) the exitation pattern for a (Lidklider-like) array of
periodicity detectors.
2. Store this autocorrelation function in memory.
3. Get another incoming signal Y(t), and compute its short-term
autocorrelation function in the same way.
4. Compute the cross-correlation function of the two autocorrelation
functions, ACX(t)*ACY(t-delay) where delay ranges from -15 msec
(or max delay) to + 15 msec. The peak in this (cross-correlation of
autocorrelations) function will tell you how close the two pitches
are and which way to tune the system. If it is at 0 delay,
you are there. A maximum at a positive delay means you tune
F0 of the incoming sound downwards (which will
move the major peak in its AC function upwards, thereby reducing the
difference).
One could do this also with a simulated auditory nerve (cochlear
band-pass functions, hair cell models, auditory nerve fiber spike
generators and all of that, see Meddis & Hewitt, Lyon & Slaney's
papers listed below for details on these approaches),
but I think the essential operations are captured here.
The summed autocorrelations of a set of frequency
channels (after band-pass filtering) is formally equivalent to the
autocorrelation of the unfiltered signal.
Cross-correlation approach.
Here one can simply take the reference signal and store it in
memory. Get the second (test) signal. Compute a (normalized)
cross-correlation function for the two signals, CC(delay) =
Sum[X(t)*Y(t-delay)], finding the best delay (to take out any
absolute relative differences in onset times). For these
purposes, the "best delay" is the temporal alignment between
the two signals that yields the highest CC peak. Align the two
signals using this delay, and take the cross-correlation term
between them using this alignment, i.e. CC(t) = X(t)*Y'(t), where
Y'(t) is the aligned (delayed) Y(t). If the two signals have the
same F0, then all subsequent major peaks spaced at intervals
of 1/F0 (i.e. 2/F0, 3/F0, 4/F0) will be the same height as the
highest peak (now positioned at 0 delay). If their F0's are
slightly different, their heights will gradually decline as
time goes on (such that the slope of this decline gives you a
magnitude of the pitch error). If you take the autocorrelation
of the zero-delay cross-correlation (CC(t)), this can be compared
with the autocorrelation of the reference signal and adjusted
in the same manner as above, using off-zero terms.
The neural interpretation of this is that one has a temporal
memory trace (X(t) that one correlates with an incoming signal
Y(t). The autocorrelation of X(t) is the all-order interspike interval
distribution of X(t) and the autocorrelation of X(t)*Y'(t) is the
interspike interval distribution of the spike trains coming out
of the coincidence detector. Differences in intervals (the error
terms) can be computed easily using sets of tapped delay lines
with (slightly) different conduction times. Then the channel in
which one sees the maximum coindidence tells you the size and
direction of the error.
I hope this helps. I'm not sure when I'll get around to putting this
down in code (probably MATLAB). Despite all appearances, I'm really
swamped with other work..... I have pieces of it from our spike
train analysis, but that just entails analyzing which peaks are
highest in spike autocorrelation histograms. If you're interested
in trying to write a program, I could help you with the details.
References:
Lyon, R., and Shamma, S. Auditory representations of timbre
and pitch. In: Auditory Computation, edited by H. Hawkins,
T. McMullin, A.N. Popper, and R.R. Fay, New York:
Springer Verlag, 1995, p. 517.
Meddis, R., and Hewitt, M.J. Virtual pitch and phase sensitivity
of a computer model of the auditory periphery.
I. Pitch identification. II. Phase sensitivity.
J. Acoust. Soc. Am. 89: 2866-2894, 1991b.
Slaney, M., and Lyon, R.F. On the importance of time -
a temporal representation of sound.
In: Visual Representations of Speech Signals,
edited by M. Cooke, S. Beet, and M. Crawford,
New York: John Wiley, 1993, p. 95-118.
Ghitza, O. Auditory nerve representation as a basis
for speech processing. In: Advances in Speech Signal Processing,
edited by S. Furui and M.M. Sondhi, New York: Marcel Dekker,
1992, p. 453-485. Oded Ghitza isn't interested in pitch per se,
but he has a simple means of generating simulated spike trains
from signals, and his front end does very well at representing
all sorts of speech contrasts.
Good luck,
Peter Cariani