Statistical hypothesis testing

[From Bruce Abbott (951023.1045 EST)]

Phil Runkel (22 Oct 1995 17:30) --

This is Phil Runkel replying to Bruce Abbott's posting of 6 October 95.

Hi, Phil, glad to hear from you. I was beginning to think that you were no
longer even "lurking" on CSG-L.

       Bruce, you say (I paraphrase) that you see nothing improper in
conducting a study of groups or averages even though you would prefer to
study individuals--that the findings might help you to understand the
physiological mechanisms. Then you say that you disagree with me. But I
think I agree with you more than you agree with me. On page 15 of my
"Casting Nets," 17 lines from the bottom, I said, "The method of relative
frequencies ... can suggest standards to be tested for existence by the
method of specimens." Your study does not tell you how alpha-MST does
what it looks aas if it does, nor does it tell you what will happen every
time, but I agree with you that it encourages you to look more deeply.

Well then, perhaps I agree with you after all, or perhaps if there is a
disagreement, it is in another area. It was my impression that you hold
inferential hypothesis testing in experiments to depend on random sampling
from well-specified populations, whereas I am claiming that conclusions
about the probable effectiveness of an independent variable in a given
experiment need depend only on random assignment to treatments. Random
sampling, I would assert, only helps to assess the probable existence of the
relationship in a particular population. However, under either viewpoint
the conclusion you offer about the usefulness of the procedure as a
screening device still holds.

To me, your choice is not a matter of propriety, but a matter of
strategy, which in turn is a matter of purpose.

Yes, exactly so. If your research is focused on a specific problem (such as
identifying the neurological substrate involved in attachment and separation
distress, with an eye toward identifying biochemical interventions for the
treatment of certain disorders), such a screening process can provide
valuable information. If your purpose is to elucidate the fundamental
control structures in chicks, such information is likely to prove useful at
some stage in the process. If I were to conduct such research, I would want
to become familiar with these findings for the guidance they might provide.