[From Bruce Abbott (980531.1955 EST)]
Rick Marken (980530.1220) --
Bruce Abbott (980530.1005 EST)
Actually Rick is doing a fine job of demonstrating a problem
with theory testing that arises when one looks only for evidence
that will _confirm_ a theory, something called a "confirmatory
This would, indeed, be a bad approach to theory testing; it implies
that disconfirmatory evidence is ignored. What you call a
"confirmatory strategy" is what I call "scientific fraud". When
you test a psychological theory you obtain evidence by observing
behavior in carefully contrived situations. You can't know in
advance whether or not what you observe will "confirm" (be consistent
with) or disconfirm (be inconsistent with) the predictions of the
theory. The evidence you get is the evidence you get; you're not
supposed to contrive (control) the results of your tests. So it
is impossible (in principle if not in practice) to do science using
either a "confirmatory" or a "disconfiormatory" strategy. The
only scientific startegy (Oded notwithstanding) is testing to
determine whether or not the empirical implications (predictions)
of a theory match what is actually observed.
Let's try this again; I'm not talking about ignoring contrary evidence
(which would constitute very bad science indeed). To practice a
confirmatory research stragegy, you make a prediction, based on theory or
hypothesis, as to what _should be observed_ under particular circumstances
if the theory or hypothesis is correct. You then set up the circumstances
(or wait for them to occur naturally if they are not subject to your
control) and determine whether _or not_ the results _confirm_ the
prediction. For example, I have a rule in mind which produces a series of
numbers, and it is your job to determine the rule. You are allowed to ask
whether certain numbers are in the series. As a first guess, you theorize
that the rule is "even numbers." You ask whether "2" is in the series, and
I answer "yes." "Ah," you think, "this looks promising." You ask whether
"4" is in the series, and I answer "yes." This continues for 6, 8, 10, 12,
14, 16, 18, 20, and so on; in each case I answer "yes." Your theory has
been confirmed with each observation, so you now confidently assert that the
rule is "even numbers." I answer, "no."
The strategy you followed was a confirmatory strategy. Note that, although
it never happened, if I had said "no" to any of your numbers, you would have
rejected your hypothesis.
In addition to this confirmatory strategy, you should have used a
disconfirmatory stragegy. This would involve asking questions whose answers
in the positive would _disconfirm_ your theory. For example, if your
hypothesis was "even numbers," then no odd number should produce a "yes"
response. You ask whether 3 is in the series, and I answer "yes." This
cannot be the case under your hypothesis, so the hypothesis is disconfirmed.
To follow a disconfirmatory strategy, you look for data that should _not_ be
observed under the theory. PCT, for example, might be tested under a
disconfirmational strategy by observing whether the ability to emit a series
of well-learned movements (e.g., playing a tune on a piano keyboard) is lost
when tactile, kinesthetic, auditory, and visual feedback are blocked during
the task. Under PCT, such feedback is crucial, so one would not expect the
person to be able to perform the movements as learned; the observation that
the person can do so would disconfirm the theory.
This is all explained in my book, _Research Design and Methods: A Process
Approach_, which represents a shameless attempt by greedy capitalists
(Mayfield Publishing, my coauthor Ken Bordens, and myself), to present at an
introductory college level all facets of the research process, from
searching the literature to publishing the results (and maybe even receive
some financial reward for having done so in a competent way).
available from Mayfield Publishing at
I think it would be _very_ instructive for everyone to go to
Me, too. There you will find a nice chapter outline to inform you of the
range of topics covered in the text, which range from how to get a research
idea through how to ethically treat human participants and animal subjects,
to how distortions in the published literature are introduced by current
research publication practices such as the use of statistical significance
testing and the p < .05 criterion.
However, psychological research has shown that people tend to
accept what fits into their current schema but ignore, reject,
or distort information that does not. Thus Rick is likely to
ignore or rationalize away this disconfirmatory evidence, rather
than allow it to affect his view of me.
This is a good example of how you cling to the idea that there
is merit in the data and methods of conventional psychology.
Here you are applying a group result ("people tend to accept what
fits into their current schema", which probably means that, say,
70% of the people in a sample accepted what fit their current
schema) to an individual (saying that I have this "tendency").
If 70% of Americans are overweight, this does not mean that you will have a
tendency to be overweight. It means that if you are an American, the
chances are better than even that you are overweight, given that I have no
other information about you personally.
If fact there are good reasons why most people are reluctant to accept
information that does not seems to "fit" with what they believe they already
know. For one, they've usually been looking for (and found) a lot of
confirmatory evidence which gives them confidence in their current views.
For another, changing these beliefs may require abandoning cherished
assumptions and may lead to a period of considerable uncertainty concerning
what to believe or not to believe -- a very unconfortable state for most.
Or they may simply not want to do the cognitive work involved in sorting
things out. It's easier, all things considered, to find a way to reject the
offending information and retain the old beliefs, or failing that, to
reinterpret the information so that it appears to "fit" after all.
Interestingly, your responses to my post indicate that the generalization
does indeed apply to you, personally, but then again, I could simply be
trying to put them in a context that will support my prior belief.