[From Bill Powers (2009.05.09.0445 MDT)]
David Goldstein (2009.05.08.06:26 EDT) –
BP earlier:…If you had said
that studies of this kind are highly likely to predict the results of
psychotherapy accurately for every person
needing help, say with a 90% probability of correct prediction for
each case, I would be impressed.
DG: You don’t understand. Medication has been viewed as the only way
to
help a person with the diagnosis of Schizophrenia. However,
medication
does not help a person acquire the practical skills of everyday living
which
the person has failed to acquire. Adding some other interventions which
specifically teach these skills adds some benefit, at least in some
cases.
BP: What did your paragraph have to do with my paragraph that you
cited? Mine was about predicting results of psychotherapy, yours
was about medication and teaching skills.
DG:In your case, you seem to be
willing to undergo the rehabilitation therapy for improving your
breathing functioning. Have you been given a 90% probablity that it will
be successful? If not, why are you doing it? If yes, have you seen the
data?
BP: I had enquired of two doctors whether exercise would help. Both of
them said no, COPD is a progressive disease and will not get better.
Finally I was referred to a specialist, who recommended the
rehabilitation approach, which involves exercise. I think the exercise
will help me – I did better than I thought I would. There was even a
little biofeedback: the woman overseeing my efforts on the machine
propped up a pulse oximeter so I could see it, and told me not to let the
reading get below 90. That was a big help.
I never said I had to have 90% reliability of a method before I would try
it. When nothing else is available you use what’s there, if it doesn’t
send you in the wrong direction. I’d say the same about being treated by
a witch doctor. In fact I have a distinct feeling of being treated by
witch doctors. So I don’t expect too much.
DG: Richard Kennaway’s paper is
very interesting and persuasive, as far as it goes. However, it does not
address the possibility of predicting a variable from a set of
variables, as in multiple, step-wise regression. This is what most
psychological studies of the correlational type do. Each subsequent
variable entered into the regression equation adds something that the
previous variable did not. This applies to the case of predicting end of
first grade reading achievement from IQ and other variables (for example
measures of phonemic awareness).
BP: If that is the rationale behind most psychological studies, I think
you may have put your finger on what is wrong with them. While what you
say sounds plausible, it looks to me like a total fallacy.
You sent me a list of 16 items which are used to judge the presence of
schizophrenia. Each item was a general term followed by a set of other
terms that elaborated on the category. These items struck me as a random
collection of informal terms referring to bad things people could
feel and do. Each one could arise for any number of different reasons.
Some sets of conditions seemed to go naturally with other sets (anxiety
and tension) but others seemed unlikely to go together (grandiosity and
blunted affect). As a basis for identifying a person’s underlying
problems, they looked useless to me.
Yet they can be used, and that is the problem. They can be used to create
totally artificial categories that have no existence except in the
context of a particular way of analyzing data. A good example of this is
found in the pdf file you sent me titled "Neurobiology of Smooth
Pursuit Eye Movement Deficits
in Schizophrenia: An fMRI Study" (to which Gary Cziko also referred,
and is in Science for May 1). In this paper 28 subjects performed
a pursuit eye movement task, with 14 being classed as schizophrenic and
14 being “healthy.” The problem can be seen in this plot of the
results:
In the left panel, 5 schizophrenic subjects (red) showed
performance in the range of the normal subjects’ performance, 8 did not.
In the right figure, 6 schizophrenics showed normal behavior, 7 did not,
2 just barely (there seems to be one missing schizophrenic).
Yet read the caption.“Subjects with schizophrenia had significantly
lower gain,” it says. It does not mention that almost half of the
schizophrenic subjects performed just like the normals. In fact,
schizophrenics who who showed higher scores than almost all the normals
in the left panel are referred to as having lower scores than the
normals, because the group mean for the schizophrenics is lower than the
group mean for the normals.
Something mysterious has gone wrong with the thinking here. Somehow a
schizophrenic who does better than most of the normals is affected by
other schizophrenics who do worse, so that his superiority is erased by
the fact that he has been classed as schizophrenic. He has been
“clustered” with his fellow-schizophrenics, and from now on
will be judged not by his actual score, but by the average score of the
group with whom he is now irrevocably associated. “Schizophrenics
have lower gain” has become a fact, in exactly the same way it once
became a fact that “Jews are pushy.” This is the sort of fact I
simply reject; it is a pseudo-fact. Psychology, it often seems to me,
consists almost entirely of pseudo-facts like this.
What the data for schizophrenics leads me to speculate is that we have
here a group of 13 people with different problems, selected because to
some psychologist they all looked sort of similar in their behavior. But
each person had something different wrong with him, and these various
problems had different indirect effects on the gain or other measures
that would be observed in a visual pursuit task. Other than that, the
problems had nothing else in common. However, it is possible to calculate
the mean gain of the pursuit model over all subjects, and while
this mean has no relevance to any of them, it characterises the
group.
The basic error here is to assume that because the mean is calculated by
using all members of the group, it is a measure of each of them. Because
a functional cluster is defined by all the people who lie within its
boundaries, its characteristics pertain to each individual in the
cluster. And so on for all the cases in which multiple measures are used
to refine some group characteristic, a characteristic which is then
turned around to be used as if it applies to each individual in the
group.
DG: I don’t think that you have
not given the study “Neurobiology of Smooth Pursuit Eye Movement
Deficits in Schizophrenia: An fMRI Study” your
usual thoughtful consideration. I will come back to it in another post,
maybe after you have had a second look.
BP: Was this time around more like what you wanted? (he asked
slyly).
I think the moral is clear. We have to treat individual cases
individually. Creating groups, analyzing the groups, and then applying
the result back to the individuals is not only cumbersome and
time-consuming, it is an offense against reason. The only time it is
justified is when the groups have been selected so wisely that
within-group measures are closely similar, and differ radically from
between-group measures – in other words, when the differences are
obvious to the naked eye and statistical analysis is not needed to see
them.
I think it is possible that the best approach to schizophrenia would be
to abolish the category and simply take each patient as a new person to
be investigated as if no other person had ever been observed. That is the
basic theme of the method of levels: every session stands alone, every
person stands alone. We do not see people through the filter of what we
supposedly know about “people.” We do not look for similarities
between cases.
We do not diagnose. We follow the trail where it leads and hope to be
surprised.
I’m reminded of the days, at least a decade ago, when I was pleading for
people to try MOL all by itself, rather than tacking it onto some
existing approach. Tim Carey committed to that first; you, David, did
that later with some patients. Now I would like to see what happens with
a straight MOL approach to schizophrenics, preferable without the
knowledge of the therapist that they have been so classified. How else
can we find out if anything else is required?
Best,
Bill P.