[From Bill Powers (2007.01.01.0540 MST)]
Very hard to get the fingers to type 2007. Happy new year to
all.
Martin Taylor 2006.12.31.22.57 –
I think you might enjoy the
attached, from the current issue of American Scientist.
Thanks. Most interesting. One of the remarkable features of this paper is
that it shows how to improve treatments, on the average, without knowing
ANYTHING AT ALL about what is wrong with the patient. By examining
symptoms, one can determine risk factors for subgroups, and then use the
results of clinical trials within subgroups to determine treatments. This
gives results that are clearly better, on the average, than using just
one measure of past outcomes for everyone.
Phil Runkel has pointed out that subdividing a population has its own
risk factor, in that this strategy reduces the numbers within each
subgroup and increases the standard deviation of any measure. He calls
this “fine slicing.”
There is a spectrum of fine slicing with the whole group at one end and a
single individual at the other end. Statistical analysis does no good at
all with single individuals – however any judgment about an individual
is stated on the basis of group statistics, it inevitably includes terms
like “in the long run” or “on the average” or
“typically” which refer to the whole population, not one
person. The probably of a single event is zero if it doesn’t happen, and
1 if it does happen, and you don’t know which is going to be the case if
there is to be only a single event (this patient’s experience of
the treatment).
Obviously, the larger the clinical study is, the smaller is the average
effect of a treatment that can be shown to be significant. But by the
same token, the larger the clinical study needed to achieve significance
becomes, the less significant must the measured effect in any given size
of subgroup. The Am Sci paper shows that there is a tradeoff, with
improvements being achievable when the subgroups are based on estimated
risks. But there must be some number of subgroups at which the increasing
uncertainty of measurement offsets the gain due to segregating people
into a larger number of risk groups. And way over at one end of the
spectrum, where the doctor must decide on the treatment of a single
individual and the individual must decide whether to allow it, even the
improved forecasting does that individual no good at all (except perhaps
to make the person feel better about the prognosis, which of course is a
population prediction).
My complaint is not that we shouldn’t do studies like these and use
statistics to try to improve treatments. When that is all we have, it
would be foolish not to do what is possible. My complaint is that until
quite recently, that statistical approach to treatment has been
essentially all we do have, most particularly at the level of patient
treatment as opposed to research. A very large part of the
“education” of a general practitioner comes from the salesmen
representing drug companies, the “detail men” (If women insist
on being included in this dubious group, I will amend the wording, but I
have yet to hear a feminist complaining about phrases like “the evil
men do.”). And the drug companies, by and large, rely far more on
statistics than on science.
I see hints here and there that a new movement is afoot, called (I think)
“systems biology.” I’ve been advocating this for years, but
most of the money spent on research into cures has been of the old
statistical kind, with very little support for trying to understand how
the damned thing works. Systems biology, as I understand the hints, is
aimed at dealing with all the variables that are simultaneously
interacting in a living system, instead of looking for simplistic causal
relationships between one variable and a whole complex of symptoms.
The systems approach is what we need to deal with people at the specimen
level. It’s what we need in order to say what is wrong with someone who
exhibits the symptoms we carelessly label with words like
“schizophrenia” or “measles” or
“headaches.” It’s what we need to start recognizing that
side-effects result from real and important changes in variables in the
whole system that are caused by the treatment. You can’t affect just one
thing in a system.
We have to get beyond the statistical approach before we can start trying
to say what is wrong with a person who exhibits certain symptoms.
“Depression” is not a name for a disorder, it’s a name for a
set of symptoms that arises from some as yet unknown underlying
malfunction. The same holds true for most of the other names we give to
maladies. Because putative “sciences of life” have relied so
heavily on statistics, what they think they know about the human system
is at a terribly superficial level, like the level of the cargo cultists
trying to bring prosperity back by cutting crude landing strips through
the jungle. Treating symptoms is what we do before we have science. The
basic premise of statistics is that the future will (for no reason at
all) be like the past, which is no way to get anywhere.
Best,
Bill P.