Poll Results; Statistics

[from Gary Cziko 930411.2210 GMT]

Here are the results on the my poll on whether the perceptual signal
contains useful information for a control system about the disturbance.

800 NO INFORMATION

300 YES INFORMATION

100 MAYBE INFORMATION

100 TOO EARLY INFORMATION

100 CAN'T ANSWER, DON'T KNOW WHAT 'INFORMATION ABOUT' IS

100 WHAT'S IT TO ME?

Although the results look decisive for NO INFORMATION, a chi-square test
using just the YES and NO categories yields a chi-square of 2.273 which is
not significant (at alpha = .05; df = 1). Therefore, the matter is still
unsettled. Further discussion may now proceed.

Note. The results of the poll are reported in hundredths of votes in order
to make the results initially appear more impressive than they actually
are.

P.S. I just realized something interesting in doing this chi-square test.
I did the test using the number of votes, not hundredths of votes. If I
had used the votes multiplied by 100, the results would have been
statistically significant with a whopping chi-square of 186.36. I never
realized before that the chi-square test was sensitive to how the results
are counted. So you can always get a statistically signficant chi-square
by reporting the results in fractions of frequencies. This is a decided
advantage over other inferential statistical tests for which in order to
get a statistically signficant finding you have to have a large enough
sample (which takes much more work than just multiplying your frequencies
by 10 or 100 or 1,000). In either case, however, it's nice to know that
you can always get statistically significant results (exciting findings
like rejecting the null hypothesis that some population correlation
coefficient is not zero) if you try hard enough (probably another reason
why "statistical" psychology and social science is better than PCT for
getting published and getting tenure).

ยทยทยท

==============================================================

Greg Williams (930410) boldly flirts with the possibility of having to
fetch the CSGnet archive files on his own by stating:

You're beating a dead unicorn. I have always agreed that to study
mass phenomena, mass statistics are appropriate.

No, I was beating (on?) a live Gary Cziko (to whom I sent the original
post, directly), whose comment

I agree. It's called the STATISTICAL method, or the METHOD OF RELATIVE
FREQUENCIES. Pretty useless as real science goes (althought the
behavioral sciences seem to like it alot.

implied to me that HE (NOT YOU, unless you're ghost-writing Gary's
direct posts to me!) thinks that mass statistics are inappropriate to
"real science." Maybe Gary actually agrees with you (and me, too) that
"to study mass phenomena, mass statistics are appropriate." I suppose he
does, even though he didn't answer my questions about his beliefs in this
regard. At any rate, if he does agree with us, it would appear that he
doesn't count studying mass phenomena as "real science."

Greg, my remarks on statistics were sent directly to you and not intended
for dissemination to CSGnet. They WERE intended to be somewhat facetious
and humorous in the spirit of other private exchanges we have had.

Yes, I most certainly agree that inferential statistics are useful for
doing research about populations, but only when one has a random sample
from that population (or can provide convincing evidence that the sample is
representative of the population of interest, even if not random)

But while I am happy to admit that inferential statistics can be usefully
employed by scientists interested in populations, I cannot recall ever
having come across a study in psychology or education psychology which has
used inferential statistics based on (a) random sample(s) of some
population(s) of theoretical or practical interest. All the studies I have
seen use SAMPLES OF CONVENIENCE (e.g., all nonabsent Boredom University
students enrolled in Psych 100 on a given day) and then generalize the
results to populations such as "fourth graders" or "people" or "men" or
"women"--just like the inane chi-square test I computed above for the poll
results!

Indeed, Joel Judd did his doctoral dissertation on reviewing several of the
most influential studies done on second language acquisition, and he
couldn't find a single useful study among them (if I remember correctly).
No random samples. Statements about how individuals learn languages when
not a single individual was investigated as a specimen.

Perhaps you can point out to me some psychological studies (ed psych would
even be better) which have used inferential statistics properly in this
respect--I would love to have some good examples to show to the students in
my intro statistics class to show how inferential statistics can be
properly used in ed. psych. Maybe such use of statistics can provide
answers to the types of questions I'm interested in (e.g., how learning in
school takes place) and I've been looking in the wrong journals.--Gary

------------------------------------------------------------------
Gary Cziko Telephone: 217-333-8527
Educational Psychology FAX: 217-244-7620
University of Illinois E-mail: g-cziko@uiuc.edu
1310 S. Sixth Street Radio: N9MJZ
210 Education Building
Champaign, Illinois 61820-6990
USA
------------------------------------------------------------------

From Tom Bourbon [930412.1513 CDT]

Gary Cziko (930411.2028) (I am still catching up after the holiday):
What a nice post on statistics! Your chi-square analysis of the poll
results reminds me of many analyses I have seen in print, in which
significance hinges on the proper transformation of the data. It is always
great fun to do a little computation and discover that a cell with p = 0.12
actually represents one person out of a sample of eight.

And your remarks about the Method of Specimens and the Method of Relative
Freqs (Freaks?) ring true. They are in line with the points Phil Runkel
made in his delightful and decisive book on the subject -- and with many
posts by you, Joel Judd, Bill Powers, Rick Marken, and a small host of others on
this net -- I even got in a few. The problem is not (we say once more) with
group statistics per se, but with their misuse and abuse in behavioral and
social science. Rarely -- hardly ever -- are they used properly in those
fields. Hence, the myriad conclusions, "models," and theories that people
derive from them are at least suspect, and in most cases are not worth the
trees cut down to make the paper on which they are printed. Period.

Of course, most misusers of statistics defend their practices on computational
grounds, asserting that they did indeed do the proper computations. In the
discussions I have had on this point, with guilty practitioners, they have no
inkling that the problems are more fundamental than the mere mechanics of
computation -- that their whole concept of research, behavior, sampling,
data and the like is open to challenge and to rejection.

Keep plugging away, Gary.