The Test: objectivity?

[From Bill Powers (930414.1430 MDT)]

Bill Silvert (930414) --

Thanks, Bill. From now on I will provide a file according to your
specs (including observing upper and lower case).


Greg Williams (930413-2) --

My point (same as when I challenged Bill) is that The Test is
the OBJECTIVE way to determine motivation, but it is not always
easy to apply or without problems.

The Test, like all other methods for discovering things about
systems, is based on subjective interpretations and impressions.
If you're looking for an OBJECTIVE test, you are doomed to
disappointment. Objective tests conduct and interpret themselves.
No human being will ever know their outcome.

What _do_ we require of tests in order to accept them as yielding
scientific data?

First, we expect them to be done according to a communicable
principle in which all the criteria and methods are publicly
known, so in principle any person could apply the test in the
same way.

Second, we expect the data produced during the test to be
observable by anyone.

Third, we expect there to be a consensus on what the data were
and how they are to be interpreted. That is, anyone "versed in
the art" should be able to derive from a record of the test and
its results the same description of what happened.

Fourth, we expect that the reasoning by which the test is said to
support or disprove some hypothesis is public and describable, so
that anyone who can understand the method of reasoning can apply
it and come up with the same conclusion.

All of these criteria, in any science, require that people
communicate with each other about what they are doing, how they
see things, how they reason, and by what means they draw
conclusions. There is no objective standard for any of these
things -- only what people can agree on. There is no way for any
person to prove to another that these understandings are in fact
the same in all the people -- that what one person sees the meter
to be displaying is the same as what another person sees. All
that can be demonstrated is that these individual, private,
subjective understandings do not conflict when they are used in
connection with some commonly-observed part of the environment.
Given the conditions of human consciousness, perception, and
thought, that is as near as we will ever get to Objective Truth.

When you and I were participating in the Test, we could not, by
definition, satisfy all the criteria above. You were the sole
judge of whether the challenge had been met. Therefore it didn't
matter whether there was a consensus between you and me, or among
all those observing the proceedings. The requirement for public
criteria publicly applied couldn't be met, as the criteria
existed solely in your head, and you didn't say what they were.

I simply did the best I could until I was satisfied that I had
identified a number of controlled variables. Then, by presenting
my method and my observations to you -- publicly -- and by laying
out my reasoning so you or anyone else could see it, I tried to
get you to pronounce the challenge either met or lost.

I now see your statement

I admitted to Bill that his predictions last week (ca.?) were
reasonable in terms of my subjective impressions of my

... as indicating some consensus between you and me at least on
the outcome of the Test. You never commented on whether my
evidence or my reasoning were acceptable to you. As one of my
conclusions was that you did not want your private controlled
variables to be known, I have ceased to ask about them (I'm not
much interested in gotchas, and almost always respect people's
desires in this regard, whether stated or implied).

If we took a vote on this, asking if people felt that they
understood the evidence, the method, and the reasoning that led
to the conclusions, we might come nearer to a scientific
consensus. But the rules of the challenge forbade that, and still
do. Such a vote would be interesting, not because it would prove
anything but because I would like to ask dissenters what there
was lacking in the procedure, so they remain unconvinced. My
goal, as always, is perfect correlations and the absence of
exceptions to the rule. If the correlations shown by a vote are
not perfect, if there are exceptions to the consensus, then we
might be able to learn from them how to improve the method and
eliminate the anomalies. From my point of view, a vote of 80 aye
and 20 nay would indicate some serious problems with the method
that could not be ignored just because a majority approved.

As you say, applying the Test can be difficult, particularly if a
person does not want a controlled variable to be identified
publicly. One thing that would make that difficult for me would
be a reluctance to pry out of someone an inner desire that was
considered private. I can imagine circumstances where I would try
to discover the controlled variable anyway, but there would have
to be some very good reason for doing so. And even then, the
difficulty I would forsee would not be discovering what the
controlled variable actually is; it would be in making any
practical use of my discovery, either in getting the victim to
admit that I am right, or in getting others to believe it.

It is really very difficult to conceal a controlled variable if
one is actually controlling it. The only practical way is to stop
controlling it, or to make the means of control so indirect and
devious that the connection is hard to see. And even then, one
can hardly conceal ALL controlled variables; to do so would make
behaving impossible.

"Motivation," by the way, is too ambiguous a term to be of much
use in a scientific discussion (although I'm sure you didn't mean
it in any precise sense). I could discover your motivation for
typing your challenge by arranging for your computer screen to
show letters different from the ones you hit on the keyboard. I
would find that you would take steps to correct that problem, so
I would know that you are motivated to make the letters on the
screen match the ones you type. The only way to conceal that
motivation would be to let the garbage keep appearing on the
screen as long as I was watching, and try desperately to figure
out the new correspondences.

But this wouldn't tell me that you are motivated to make those
letters appear on the screen as a way to send messages, or that
you want to send messages as a way of telling others what you
think or finding out what they think, and so on up to the level
of why you are doing this in Gravel Switch, KY with the wife and
children you have.

Science doesn't give anyone automatic permission to find out
things about people that they don't want known -- even if it may
provide methods for doing so.

Bill P.