Methods N' Such

[Abbott_Bruce_Indiana]

[From Bruce Abbott (950104.1215 EST)]

I'm a little pressed for time at the moment, so I'll just comment briefly on a
couple of subjects.

1. Traditional Research Methods Versus PCT Methods

     Rick: please tell me how you plan to study the properties of human
     memory using the Test. Your results should allow you to explain, for
     example, how it is that you are able to recognize a familiar face.

2. Program ThreeCV

     Bill: Keep in mind that I agree that the properties of control systems
     are best analyzed in light of the control system model. What I am
     saying is that there are many questions about human and animal behavior
     that can be researched quite effectively using other methods. I see the
     control systems analysis as a subset of "IV-DV" methods, not as an
     alternative to them. Each has its proper place. Yet, once again I have
     been invited to play the role of the traditionalist and once again I
     have been given the "once you understand PCT" speech. It's like saying
     "to take apart an engine, you need a variety of tools," and then being
     "corrected" by being told that "once you understand carburetors, you'll
     realize that the only appropriate tool for dismantling them is the
     screwdriver."

     I've run the program; what is it you wish the "traditional analysis" to
     analyze? Knowing nothing about control systems, negative feedback, and
     the folly of trying to analyze such systems in unidirectional cause-
     effect terms, and having run the program, I would treat mouse movement
     as the _independent_ variable and the observed changes in screen display
     as the _dependent_ variable. I would seek to determine the underlying
     rules by which cursor movement translates into those changes in screen
     display (line angle, rectangle size, rectangle shape). This means that
     I need to move the mouse to various positions and then repeatedly sample
     and record the screen variables at each position. The collected data
     would be plotted against cursor position and subjected to a regression
     analysis (after appropriate transformations of the data, if necessary).
     The analysis would reveal, to a fair approximation, the relationship
     between cursor movement and screen variables, and would indicate the
     degree to which those random disturbances were reducing the ability to
     predict screen variable values (percentage of variance accounted for
     would be lowered in proportion to the relative magnitude of the
     disturbance effect).

     But I'll bet this is not what you had in mind. (;->

3. Statistical Analysis

     Bill, your comments about the misuse of statistical analysis in
     psychology are well taken, but you have made some errors in your
     analysis of the probabilities and in their interpretation. You might
     want to ask Rick, an expert in such matters, to explain the problems to
     you. When he's done that, I'll respond.

Regards,

Bruce

[_Martin_Taylor]

[Martin Taylor 950104 12:40]

Bruce Abbott (950104.1215 EST)

Butting in, treading where angels fear to rush, etc...

2. Program ThreeCV

... I've run the program; what is it you wish the "traditional analysis" to
   analyze?
... I would treat mouse movement
    as the _independent_ variable and the observed changes in screen display
    as the _dependent_ variable. I would seek to determine the underlying
    rules by which cursor movement translates into those changes in screen
    display (line angle, rectangle size, rectangle shape). This means that
    I need to move the mouse to various positions and then repeatedly sample
    and record the screen variables at each position. The collected data
    would be plotted against cursor position and subjected to a regression
    analysis (after appropriate transformations of the data, if necessary).
    The analysis would reveal, to a fair approximation, the relationship
    between cursor movement and screen variables, and would indicate the
    degree to which those random disturbances were reducing the ability to
    predict screen variable values

Yep, and what would you find, if the subject happened to be controlling for
a constant rectangular area? (If that was one of the three things being
disturbed--I forget.)

What you'd WANT to find would probably be some connection between the
mouse movements and something about the rectangle, at least more so than
between the mouse movements and aspects of the other two displays. But
that's exactly what you would NOT find, if the subject's control was good.
The rectangle would not change, while the mouse was moving all over the
place, but the OTHER displays would show changes that correlated, poorly
but significantly, with the mouse movements.

If I've intuited correctly, the naive analyst would say that the subject
was "doing" something to the other two displays, but not to the rectangle.
Of course, in a sense, this is right. But it is not right from the point
of view of analyzing what the subject is intending to "do." It is precisely
the LACK of correlation that shows the subject to be controlling the
rectangle display as opposed to one of the others.

But lack of correlation is not in itself enough. The "Test" analyst must
at least be able to determine that the rectangle display WOULD HAVE moved
if the subject hadn't acted against the disturbance. So the analyst, but
not the subject, must be able to perceive either the disturbing variable
or some surrogate for it.

Rick has made a very nice demo of this effect. Some numbers move around
the screen in a random walk fashion. Movements of the mouse affect all of
them simultaneously. The subject selects one of the numbers, and using
the mouse, causes it to follow a track of the subject's choosing. The
track may be as random as the subject wants, provided that the subject
chooses the track in advance and is careful to make the chosen number follow
the chosen track. If the subject does this, the computer can determine
which number the subject has chosen to manipulate, by finding which one
has the least correlation between the mouse movement and the number's
track. But if the subject just manipulates one of the numbers with no
clear preconception of where the number should go, the computer fails.
It is the subject's intention, and the follow-through on that intention,
that matters, not the mouse movements. Mouse movements correlate well
with what the subject is NOT "doing" on the screen.

Mathematically, the mouse movement effects on the number's position are
antiphase to the disturbance effect, so that the result is decorrelated
from either, whereas for the unselected numbers the mouse movements are
orthogonal to their disturbances, yielding a sum that is correlated with
both the disturbance and the mouse movement (each acting as noise when the
other is analyzed).

These demonstrations seem to me to illustrate very well why the PCT-core
is sceptical about the "facts" of correlational analysis.

Martin

[Tom_BOURBON3]

Tom Bourbon [950104.1330]

[From Bruce Abbott (950104.1215 EST)]

1. Traditional Research Methods Versus PCT Methods

    Rick: please tell me how you plan to study the properties of human
    memory using the Test. Your results should allow you to explain, for
    example, how it is that you are able to recognize a familiar face.

But(t)insky Bourbon, that's me.

Cows say quack, quack.

The figure at the end of the second line back was a (

2. Program ThreeCV

    Bill: Keep in mind that I agree that the properties of control systems
    are best analyzed in light of the control system model. What I am
    saying is that there are many questions about human and animal behavior
    that can be researched quite effectively using other methods. I see the
    control systems analysis as a subset of "IV-DV" methods, not as an
    alternative to them.

Could you say a little more about the idea that PCT analysis is a subset of
IV-DV methods?
. . .

    I've run the program; what is it you wish the "traditional analysis" to
    analyze? Knowing nothing about control systems, negative feedback, and
    the folly of trying to analyze such systems in unidirectional cause-
    effect terms, and having run the program, I would treat mouse movement
    as the _independent_ variable and the observed changes in screen display
    as the _dependent_ variable.

That would make you a very unusual novice at studying control. Something
very different usually happens in similar studies where a person controls
the position of one of several independently disturbed cursors (yuk!
tracking!), and inexperienced observers, including most psychologists,
watch. Most observers believe the person moves the control device in
response to movements of the *uncontrolled* cursors, whose positions often
correlate +.4 to +.7 or so with positions of the control device. Rarely --
practically never -- does a novice even guess that the seemingly random
positions of the control device have anything to do with the relatively
unchanging position of the controlled cursor: correlations between the
device and the position of the controlled cursor hover around 0.0.

  I would seek to determine the underlying
    rules by which cursor movement translates into those changes in screen
    display (line angle, rectangle size, rectangle shape). This means that
    I need to move the mouse to various positions and then repeatedly sample
    and record the screen variables at each position.

Good traditional thinking. You play your reluctantly accepted role very
well. Of course, the results of your mouse movements will be determined by
the movements and by the random disturbances that act on each object.

You seem to do a good "traditional" job describing the experiment in lineal
terms. The traditionalist might overlook the fact that, if the person
continuously controls perceptions of one of the objects against the effects
of a disturbance, there is no sequence: move, sample, move, sample, etc..
The sampling occurs "on the fly," during continuous movements by the person.

    The collected data
    would be plotted against cursor position and subjected to a regression
    analysis (after appropriate transformations of the data, if necessary).
    The analysis would reveal, to a fair approximation, the relationship
    between cursor movement and screen variables, and would indicate the
    degree to which those random disturbances were reducing the ability to
    predict screen variable values (percentage of variance accounted for
    would be lowered in proportion to the relative magnitude of the
    disturbance effect).

And the good traditionalist would get to work eliminating all of those
"extraneous" and "uncontrolled" variables, until the prediction of
property-of-object by position-of-control-device was nearly perfect. But
that would mean the world had been turned into a disturbance-free place.
And the traditionalist, if true to the breed, probably would miss the
beautiful point that the person had controlled the chosen object in spite
of the disturbances. People don't achieve good control only when the
environment is made free of disturbances (although that *can* help); they
usually control well in a disturbance-filled world.

Now that you have stepped out of your self-imposed role-play as a
traditionalist, how many real traditionalists do you believe would interpret
the demonstration the way you did?

3. Statistical Analysis

    Bill, your comments about the misuse of statistical analysis in
    psychology are well taken, but you have made some errors in your
    analysis of the probabilities and in their interpretation. You might
    want to ask Rick, an expert in such matters, to explain the problems to
    you. When he's done that, I'll respond.

Wow! I'll But(t)outsky of that one.

Later,

tom