On Psychophysics

[Tom_BOURBON3]

Tom Bourbon [941005.1106]

[Martin Taylor 941004 18:30] Re: Mixed up confusion
Rick Marken (941003.2330)

Martin Taylor (941003)

. . .

Martin:

With THAT reference, then the relationship between o and d becomes less
problematic. And when the experiment deals with the limits on a subject's
ability, as in a detection study, there is very little the subject can do
to generate data that would be consistent with greater abilities than he/she
actually has. The data can be taken at the very least as providing a
lower bound on the subject's ability to discriminate (or whatever). Even
if the subject was not controlling the "satisfy experimenter" perception,
but was doing the task for quite other personal purposes, the results still
provide a lower bound on the subject's ability. And usually those bounds
are pretty close to the performance that would be expected of an appropriately
constrained mathematically ideal observer, so they probably represent
something very near the subject's actual ability.

A comment, Martin, on "limits of abiliy." (For now, I will abstain from a
comment on "the ideal observer.") Even in a (signal?) detection study, we
measure _performance_, not _ability_. We know that the participant can "do
at least this well," but we do not know the limits ofeither performance or
ability. For example (an example I am certain you know well, Martin), a
participant might be caught up in a detection study as part of the
examinations for military conscription. A person who intends to avoid
conscription might perform so as to create the impression of poor ability
(poor sensitivity) by pretending not to perceive signals that are in fact
perceived.

As you suggested, it is another matter again for a person to feign
sensitivity (ability) greater than the sensitivity we assume exists; but
here, too, there are limits on _our_ ability to measure the person's limits.
Using either the methods from classical psychophysics (CP), or those from
signal detection theory (SDT), what we see in our data is an association (in
our perceptions as experimenters) between "stimuli" we present and responses
the participant "makes." The results are always stated in terms of "some
measure of responses vs some measured dimension of the stimulus." In CP,
for example, we might report "the stimulus intensity at which, on .5 of the
trials when that particular intensity occurred, the person said, "yes, I
hear-see-taste-feel-etc it."" In SDT, we might report, "at each of several
ratios of signal power to noise power, the probability that the person
correctly identified which of two observation intervals contained the
signal." The report is _always_ some measure of R as a function of S --
-- always R = f(S).

My quarrel here is not with the S-R nature of psychophysics. Instead, I am
interested in one of the underlying assumptions in SDT: the idea that
discrimination (a measure of performance, not of absolute ability) does not
occur around an absolute limit or threshold, above which there is perception
(subjects say "yes") and below which no perception (subjects say "no"), but
that so long as there is _any_ physical difference between signal events (S)
and noise events (N), a detector will discriminate between the two classes
of events. Non-zero differences between S and N will always yield non-zero
measures of discrimination, assuming (and it is a very big assumption) a
large enough number of trials when the person is required to make
discriminations. Often it is impractical for an experimenter to put this
idea to the test: not enough money or research time, subjects will revolt,
even if discrimination is non-zero it is so low that for _all practical
purposes_ it is zero, and so on. In the end, the experimenter accepts a
measured "limit" on the participant's "ability" that also clearly reflects
the experimenter's reference levels for perceptions such as humane treatment of
subjects, fiscal responsibility as a grant holder, practicality, and the
like. I think that, whether we use the methods from CP or from SDT,
psychophysics always shows R = f(S), and perceptual control by subjects and
experimenters alike _always_ figures into the results. There is no royal
psychophysical road that lets us directly measure the sensitivity of isolated
parts of perceptual control systems.

As for your examples of the games subjects can play (listening to relays
instead of watching lights, dismantling the equipment instead of playing the
desired but unstated game, and so on), they brought back fond, and not-so
fond, memories of days in the psychophysics labs. Some of the stories
probably cannot be told even on a liberal forum like the internet.

Later,

Tom

[_Martin_Taylor]

[Martin Taylor 951006 1515]

Remarkable--now I'm getting messages posted as recently as yesterday! And
that despite the lack of our primary Internet connection.

Tom Bourbon [941005.1106]

Martin Taylor 941004 18:30

the results still
provide a lower bound on the subject's ability. And usually those bounds
are pretty close to the performance that would be expected of an appropriately
constrained mathematically ideal observer, so they probably represent
something very near the subject's actual ability.

A comment, Martin, on "limits of abiliy." (For now, I will abstain from a
comment on "the ideal observer.") Even in a (signal?) detection study, we
measure _performance_, not _ability_. We know that the participant can "do
at least this well," but we do not know the limits ofeither performance or
ability.

"At least this well" is a lower bound on ability. The appropriate
ideal observer is an upper bound on the ability of ANY observer, including
the experimental subject. When those two measures are with 4dB, as is often
the case with a trained listener, the actual ability is pretty closely
constrained.

... Instead, I am
interested in one of the underlying assumptions in SDT: the idea that
discrimination (a measure of performance, not of absolute ability) does not
occur around an absolute limit or threshold, above which there is perception
(subjects say "yes") and below which no perception (subjects say "no"), but
that so long as there is _any_ physical difference between signal events (S)
and noise events (N), a detector will discriminate between the two classes
of events. Non-zero differences between S and N will always yield non-zero
measures of discrimination, assuming (and it is a very big assumption) a
large enough number of trials when the person is required to make
discriminations.

Yes, this is so, but I'm not at all clear where you perceive a problem.

In the end, the experimenter accepts a
measured "limit" on the participant's "ability" that also clearly reflects
the experimenter's reference levels for perceptions such as humane treatment of
subjects, fiscal responsibility as a grant holder, practicality, and the
like.

You take this to follow from the preceding quote, but to me it is a
non-sequitur. If I want to know the subject's ability to detect, say,
a 500 Hz tone in a rectangular band of noise presented in the other ear,
I don't do it at a level where the subject gets 55% or even 65% in a two
alternative forced choice. And I don't do it at a signal level where the
subject gets 99% or 99.5% correct. Both situations make it necessary to
run lots of trials. I would do it in the region 70% to 85%, where the
number of trials required to get a good estimate of the subject's ability
is in a moderately flat minimum. And I would use an efficient adaptive
procedure, which reduces one kind of bias that makes a subject appear
less able than they actually are (see below).

I think that, whether we use the methods from CP or from SDT,
psychophysics always shows R = f(S), and perceptual control by subjects and
experimenters alike _always_ figures into the results.

Yes, it must. There's no way around it. The problem of good experimental
technique is to induce the "right" reference signals in the subjects, so
that they use maximum gain in the control systems that support their
perception of satisfying the experimenter (or perhaps of gaining monetary
rewards for good performance), and so that the ONLY way they can control
that perception is to do well on the task set by the experimenter. It is
often in the word "ONLY" that poor experimental technique fails.

There is no royal
psychophysical road that lets us directly measure the sensitivity of isolated
parts of perceptual control systems.

Right. Sensitivity is always a statistical question. Two "stimulus"
presentations that are as nearly identical as the experimenter can make them
(identical waveforms at the eardrum, for example) will often result in
different responses. Does this mean that the sensitivity was different
on the two occasions? One could treat it so, and say always that the
sub-system was more sensitive on those occasions when the subject gave
the "right" answer than on those occasions when the answer was "wrong."

I prefer to think of the sensitivity as a property of the system that
changes more slowly, and that is independent of the signals applied to
the system. The "ability" that I talked about above and in previous
postings refers to the maximum sensitivity of which that sub-system is
capable, and I have demonstrated (J. Acoustical Soc America 1983, 74,
1367-74) that the actual ability is normally somewhat greater than is
actually measured in a forced choice experiment that uses fixed signal
levels during a run. In other words, the standard technique underestimates
the lower bound on the subject's ability.

If you allow the subject to control the level by means of the correctness
of the responses, performance is better than if the subject's responses
have no effect on subsequent stimuli--but we are still dealing with
statistical estimates of sensitivity, unlike a Bekesy audiometer, where
the subject tries to adjust the signal to "just audible."

The real problems with interpreting psychophysical data come, it seems to
me, in situations where there is no measurable or theoretical bound on one
side of the subject's performance (or even on both sides). Then you are
stuck with the problem of determining the extent to which the subject was
an effective co-conspirator. You do this both by observing the subject's
demeanour and by designing the experiment so that the data can be cross-checked
(a term from ice-hockey) for consistency. But you can never be sure.

···

======================

Avery Andrews 941005.1753

The expected result (if you're a linguist) is that subjects would
resist disturbances that merged or otherwise distorted phonemes,
but not ones that didn't (for example, slowing down the overall
speech-rate, but who knows?)

The background notion sounds very like John Ohala's theory of linguistic
drift and divergence (I have a reprint somewhere, but I seem to have
mislaid it). And the experiment might be used as a test of his ideas.

Simplified, as I understand it, Ohala argues that similar perceived sounds
can be generated by different articulator positions or trajectories. So
some versions of a fricative (f) generated by blowing air through a narrow
gap between the lower teeth and upper lip sound much like some versions
of a fricative (th) generated by blowing air through a small gap between
the tongue-tip and the upper teeth; or in some contexts an "l" might sound
quite like an "n" or even an "o" (my examples). It is quite possible for
a child to learn to use the "wrong" articulation for a sound in some contexts,
even though some variants of that articulation may sound appreciably different
from extreme variants of the "right" articulation, and might not be used in
other contexts. But anyone learning from that child grown to adulthood
would be hearing (and controlling for hearing in their own production)
the range of sounds produced by the "wrong" articulation. And thereby
hangs a "sound shift" and a dialect variant.

Any results from your experiment could, it seems to me, be used as a direct
test of Ohala's ideas, inasmuch as the sounds most liable to a "sound shift"
would be those less strongly controlled by an experimental subject. Ohala
discusses stable and labile kinds of sounds, and the experiment could well
choose to apply the Test to very stable and to very labile sounds (I won't
say "phonemes"). It would be ideal if subjects could be chosen from
speakers of languages from different language families.

Martin

[Tom_BOURBON3]

Tom Bourbon [941010.1245]

[Martin Taylor 951006 1515]

Replied to my post:

Tom Bourbon [941005.1106]

Martin Taylor 941004 18:30

Martin, you support one of the points I was making, when you say:

. . . If I want to know the subject's ability to detect, say,
a 500 Hz tone in a rectangular band of noise presented in the other ear,
I don't do it at a level where the subject gets 55% or even 65% in a two
alternative forced choice. And I don't do it at a signal level where the
subject gets 99% or 99.5% correct. Both situations make it necessary to
run lots of trials. I would do it in the region 70% to 85%, where the
number of trials required to get a good estimate of the subject's ability
is in a moderately flat minimum. And I would use an efficient adaptive
procedure, which reduces one kind of bias that makes a subject appear
less able than they actually are (see below).

I would call this a _certain_ measure of the subject's performance and at
best an _assumed_ measure of the subject's ability. That aside, you have
made very clear many of your own criteria (reference perceptions) that you
adopt during a psychophysical experiment. You have references for the kind
of task, the particular level of performance you will accept, why you
prefer that level rather than others, and so on. The experimenter
functions as a control system, "adjusting" various aspects of the
experiment until the perceived psychophysical data match the desired data.

Tom:

I think that, whether we use the methods from CP or from SDT,
psychophysics always shows R = f(S), and perceptual control by subjects and
experimenters alike _always_ figures into the results.

Martin:

Yes, it must. There's no way around it. The problem of good experimental
technique is to induce the "right" reference signals in the subjects, so
that they use maximum gain in the control systems that support their
perception of satisfying the experimenter (or perhaps of gaining monetary
rewards for good performance), and so that the ONLY way they can control
that perception is to do well on the task set by the experimenter. It is
often in the word "ONLY" that poor experimental technique fails.

It is certainly true that problems can arise when experimenters attempt to
"induce reference signals" in subjects, by various coercive means (giving
or withholding personal approval, academic credit, money, leave time, other
privileges). In these respects, psychophysical studies resemble the
coercive, manipulative procedures in "the experimental analysis of
behavior," otherwise known as classical and operant conditioning. When
psychophysical procedures turn coercive, the experimenter confronts
"problems" that are less a matter of "poor experimental technique" than of
not understanding the nature of the subject. The subject is a control
system, but is treated as though he or she were a lineal input-output
device. To be sure, psychophysical procedures used in that manner can
produce data, often highly reliable and systematic data, but they tell us
nothing _direct_ about the phenomenon of control or about the subject as a
control system.

Later,

Tom