Bait and Switch

[From Bruce Abbott (950104.2050 EST)]

Rick Marken (950104.1445)

Bruce Abbott (950104.1215 EST)

Rick: please tell me how you plan to study the properties of human memory
using the Test. Your results should allow you to explain, for example, how
it is that you are able to recognize a familiar face.

I presume that this means that you don't want to discuss the problems of
using conventional IV-DV methodology to study living control systems?

Actually no, I don't. I want to discuss the problems of using control-system
methodology to study aspects of human and animal capabilities for which that
methodology is inappropriate. In this way I hope to bolster my argument that
traditional methods have their place in psychology. I seem to recall your
saying that they have no such place. What you have done is to RESPOND as if I
had said that conventional methods are adequate for studying the phenomena of
control, which I never claimed. You are now out to prove this statement I
never made wrong. It's a great politician's trick, to avoid answering the
question asked by acting as though a DIFFERENT question had been asked, and
answering that one.

Ah well. I'll try to answer your not obviously germane question as best as I
can.

As you should now understand, it is germane to the issue I raised (but not to
the one you wish to address).

"The Test" and variations thereof are used to study control phenomena.
Memory is probably a control phenomenon with a hefty dose of control of
"imagination" (see B:CP for technical definition). I would probably have to
use some variant of "The Test" to study the controlling that goes on when one
recognizes a face as familiar or not. I can't tell you exactly how I would
use "The Test" to study this phenomenon until I had a pretty good model of
the phenomenon to be tested. Of course, if memory is not a control
phenomenon then use of "The Test" would, indeed, be unnecessary.

Let's assume for the sake of argument that the aspects of memory I wish to
study are not control phenomena. "The Test" is unnecessary. What now? How
do I go about discovering the underlying mechanism by which I am able to
recognize a face? Do I throw up my hands in despair, or do I reach for one of
those other methods described in my text?

Since I answered your memory question, could you please answer my
(implicit) "Los Tres Bandidios" question: After doing the "Los Tres
Bandidios" demo is there any reason why research psychologists should not
immediately abandon the conventional IV- DV approach to understanding
behavior?

You haven't answered my question; you've cleverly avoided it. You first.

Bill, your comments about the misuse of statistical analysis in psychology
are well taken, but you have made some errors in your analysis of the
probabilities and in their interpretation. You might want to ask Rick, an
expert in such matters, to explain the problems to you. When he's done
that, I'll respond.

Gee. It looked OK to me.

I'm done. Your turn:-)

I don't believe you made a good-faith effort. How about summarizing the
statistical argument Bill made so I'll know you actually read and understood
it? I know you've had better statistical training than your response would
indicate; I think you're just playing dumb. Your turn. >:->

Tom Bourbon [950104.1330]

    Rick: please tell me how you plan to study the properties of human
    memory using the Test. Your results should allow you to explain, for
    example, how it is that you are able to recognize a familiar face.

But(t)insky Bourbon, that's me. :wink:

Cows say quack, quack.

I'm afraid I missed the significance of your reply...please clarify?

The figure at the end of the second line back was a (:frowning:

No it wasn't.

Could you say a little more about the idea that PCT analysis is a subset of
IV-DV methods?

I defer to Bill (950102.1825 MST):

I agree that Rick is overgeneralizing. We use IV-DV analysis when using
the Test for the controlled variable, as you pointed out.

Me:

    I've run the program; what is it you wish the "traditional analysis" to
    analyze? Knowing nothing about control systems, negative feedback, and
    the folly of trying to analyze such systems in unidirectional cause-
    effect terms, and having run the program, I would treat mouse movement
    as the _independent_ variable and the observed changes in screen
    display as the _dependent_ variable.

That would make you a very unusual novice at studying control. Something
very different usually happens in similar studies where a person controls
the position of one of several independently disturbed cursors (yuk!
tracking!), and inexperienced observers, including most psychologists,
watch.

Well, maybe I am a very unusual novice at studying control. But that's an
empirical question. Tell you what--I'll show the demo to several of my
colleagues (have them serve as participants) and ask them how they would go
about analyzing what is going on.

    I would seek to determine the underlying
    rules by which cursor movement translates into those changes in screen
    display (line angle, rectangle size, rectangle shape). This means that
    I need to move the mouse to various positions and then repeatedly
    sample and record the screen variables at each position.

Good traditional thinking. You play your reluctantly accepted role very
well. Of course, the results of your mouse movements will be determined by
the movements and by the random disturbances that act on each object.

Yes.....

And the good traditionalist would get to work eliminating all of those
"extraneous" and "uncontrolled" variables, until the prediction of
property-of-object by position-of-control-device was nearly perfect. But
that would mean the world had been turned into a disturbance-free place.
And the traditionalist, if true to the breed, probably would miss the
beautiful point that the person had controlled the chosen object in spite
of the disturbances. People don't achieve good control only when the
environment is made free of disturbances (although that *can* help); they
usually control well in a disturbance-filled world.

Yes, and once those disturbances were removed, there would be one of those
much-prized near-perfect fits with correspondingly high correlations in the
+.99 range. And Rick said these methods were useless...

Having gotten this far, I would now examine my performance on this task as a
function of those pesky disturbances. Which way do I move the mouse as the
screen stimuli change and I attempt to keep them unchanged, and by how much?
Would others less familiar with control theory than I am try this? I don't
know, but I wouldn't be so sure they wouldn't. After all, it's just another
simple application of traditional research methods....

But enough of this. As I said before, I agree that a proper analysis of such
control systems requires using the appropriate (control system) model. What I
do not agree with is the idea that traditional methods are categorically
inappropriate when used to research human and animal behavioral capabilities.

Tom Bourbon [950104.0950]

A quick question, Bruce. Have you read Phil Runkel's book, _Casting Nets
and Testing Specimens: Two Grand Methods of Psychology_?

No, but I plan to. Sounds interesting...

Another quick (double) question. When you signed on to csg-l, you said you
were planning a new edition of your book on methods -- on all of the right
tools. Have you decided to include anything on PCT? If so, have you
decided how you will present the PCT material, relative to all of the
traditional tools? If so, how? (I guess this was a quick *triple*
question.)

I'd like to, but I'm not sure as yet how to proceed. Much of it would belong
in the chapter on single-subject research. How would you go about describing
"the" PCT approach to data-collection and analysis? Keep in mind that the
book is about how to conduct psychological research (methods-based), not about
specific areas of research (content-based). I do not get into specific detail
on the how to do psychophysical research, or human memory research, or
research on "learned helplessness," or research in other specific areas.

Bill (and Rick?) already replied to your portrayal of "the Test." Enough
said on that.

If you are not going to add to the discussion, why bring it up? Bill agreed
with me, Rick didn't. What's your opinion?

. . . . . I am
putting together a book about some of the horrors that are wrought when
behavioral and life scientists abuse their experimental designs, their
statistics, and their participants. Examples of the practice you describe
here are among the easiest abuses to find in the *contemporary* literature.
I have a drawer full of them.

How about some references? I haven't seen a case in recent times in which the
actual results were omitted in favor of a statistical table, but maybe I
haven't been looking in the right places. However, I agree that there is
plenty of abuse going on, but remember, "statistics don't kill research,
researchers kill research." So vote "no" against the Brady Anti-statistics
Bill!"

Me:

there are many, many questions about how humans and other animals function
for which these stock-and-trade methods work well, and I strongly disagree
with Rick's assertion that what traditional research methods texts teach
must be seen as total nonsense after one adopts PCT. It is one thing to
argue that the wrong tool has been chosen for the job at hand, but quite
another to argue that the same tool must be used on every job. Even "living
control systems" offer plenty of "jobs" for which those other tools are well
suited.

Tom:

Yes, *if* we want to know which proportions of a particular species of
living control systems will do X, Y or Z, under conditions A, B, or C, then
group methods are the *only* way to go. But we had better not, on pain of
ranting by Marken and Bourbon, say we learned anything specific about any
individual, or about individuals in general -- remember the aggregated
curves and keep them holy. If we want to know about individuals, or about a
particular individual, we must study individuals, no two ways about it. :wink:

Hey, you're talkin' to an "n-of-one" kind-a guy! In general, I agree with
you. But there are some questions that can be addressed in no other way.
Some processes are irreversible; they can be observed only once in a given
participant. In those cases the aggregate methods may be the only game in
town.

Regards,

Bruce

Tom Bourbon [950105.0851]

[From Bruce Abbott (950104.2050 EST)]

. . .

Tom Bourbon [950104.1330]

Bruce had said:

    Rick: please tell me how you plan to study the properties of human
    memory using the Test. Your results should allow you to explain, for
    example, how it is that you are able to recognize a familiar face.

Cows say quack, quack.

I'm afraid I missed the significance of your reply...please clarify?

Dogs say "meow."

The figure at the end of the second line back was a (:frowning:

No it wasn't.

Are you sure, now that the original second line back has been deleted?

I'll reply to the remainder of your post a little later.

Tom

Tom Bourbon [950105.1158]

[From Bruce Abbott (950104.2050 EST)]

. . .

Tom Bourbon [950104.1330]

Could you say a little more about the idea that PCT analysis is a subset of
IV-DV methods?

I defer to Bill (950102.1825 MST):

And to think. You are the guy who jumped on me when I deferred to Bill and
Rick in a topic. :slight_smile: I really would like to see your ideas on the subject.

Bruce:

    I've run the program; what is it you wish the "traditional analysis" to
    analyze? Knowing nothing about control systems, negative feedback, and
    the folly of trying to analyze such systems in unidirectional cause-
    effect terms, and having run the program, I would treat mouse movement
    as the _independent_ variable and the observed changes in screen
    display as the _dependent_ variable.

Tom:

That would make you a very unusual novice at studying control. Something
very different usually happens in similar studies where a person controls
the position of one of several independently disturbed cursors (yuk!
tracking!), and inexperienced observers, including most psychologists,
watch.

Bruce:

Well, maybe I am a very unusual novice at studying control. But that's an
empirical question. Tell you what--I'll show the demo to several of my
colleagues (have them serve as participants) and ask them how they would go
about analyzing what is going on.

Great! I have suggestions for a few simple modifications in your procedure
that might produce some interesting results. Present the demo to them, one
at a time, and ask them not to discuss it with one another until after all
have seen it. Before you ask them how they would analyze what is going on,
ask them a simpler question: What is going on? Let them do that sacred
"first step" described in every intro psychology text, and most texts on
methods in psychology: Observe and describe behavior. In books like those,
analysis, explanation and the control of behavior all come later in the
ceremony.

. . .

Tom:

And the good traditionalist would get to work eliminating all of those
"extraneous" and "uncontrolled" variables, until the prediction of
property-of-object by position-of-control-device was nearly perfect. But
that would mean the world had been turned into a disturbance-free place.
And the traditionalist, if true to the breed, probably would miss the
beautiful point that the person had controlled the chosen object in spite
of the disturbances. People don't achieve good control only when the
environment is made free of disturbances (although that *can* help); they
usually control well in a disturbance-filled world.

Bruce:

Yes, and once those disturbances were removed, there would be one of those
much-prized near-perfect fits with correspondingly high correlations in the
+.99 range. And Rick said these methods were useless...

Rick is right. We (you and I) seem to be talking about different things.
The idea in PCT research is not to create environmental conditions where
the correlation between actions and their results is 1.0. That is trivially
easy. In an undisturbed pursuit tracking task the correlation between
positions of the control device and the cursor is *aways* +1.0, not a
mere +.99.

                     c = h + d

  cursor position = handle position + disturbance

and in this case, d = 0. The correlations between c and t, and h and t,
will be identical and their value depends on how well the person tracks.

Make d nonzero, and things change. Up to a pretty wild disturbance, the
correlation between c and t will be close to what it was in the
undisturbed condition; the correlation between handle and target will be
low; the correlation between h and c will be low -- over a series of runs
on such a task, this correlation approaches zero.

The +1.0 correlation between h and c, in undisturbed tracking, isn't the
one that gets a few of us all excited (and leaves the rest of the world
absolutely cold). What we love are the correlations of +.99+ between the
person's handle positions and those of a PCT model with its gain adjusted to
a best approximation of the person's gain, when the person is modeled as a
single PCT control loop. Now *that* is someting to get excited about. Like
Rick, I know traditional methods and traditional theories in behavioral
science can't produce results like that.

But there is more. After we get the +.99+ correlations between model and
person in the undisturbed task, we let the model do another run, where a
random disturbance acts on the cursor:

     c = h + d

and d is nonzero and random. The model produces its 1800 predicted handle
positions, one at a time, and the program adds each of them to the
corresponding time-indexed value of the disturbance to produce 1800
predicted cursor positions. Then the person does the run, with the same
disturbance active. The result? Magnificent! Whether the person does the
predicted run 1 few minutes later, one year later, or five years later,
the correlation between the 1800 pairs of predicted and actual handle
positons is +.99+ -- in the series of runs I am thinking about, the
correlations were >= +.996, across a span of a few minutes to five years.
When the results for the person and model are plotted on the same
coordinates, they are virtually indistinguishable. I'd show you, but the
paper keeps getting rejected -- I guess traditionalists already know these
things.

That's what we get excited about. Those are the much-prized, near-perfect
correlations that mean sometning to us. Who cares if r = +1.0, when
c = h + d and d = 0? If that's all it takes to excite a traditional
behavioral scientist, life must be bleak. :wink:

To a simple-minded guy like me, another nice thing about these high
model-person correlations is that they hold up, not only across years, but
also across a range of disturbance-induced "difficulty" that sometimes
results in rather poor control by the person -- under a condition that is
difficult for the person, the model also controls rather poorly and the
correlation between person and model remains >= +.99. Exciting, but some
of us are just sort of kinky.

Having gotten this far, I would now examine my performance on this task as a
function of those pesky disturbances.

Ah, yes. And those pesky disturbances would keep changing -- a different
one every time you did a run. That should make life interesting for the
traditionalist. Pretty soon, the experimenter might get the idea that
the particiant's movements are random.

Which way do I move the mouse as the
screen stimuli change and I attempt to keep them unchanged, and by how much?

Good. You are still role-playing the traditional S-R psychologist, looking
for the stimuli of which actions are a function. How long would you (in
that role) need to look before you decided there was no identifiable screen
stimulus of which behavior is a function? (Incidentally, have you read
"Models and their worlds," an obscure -- no kidding -- paper by Bill and
me? If not, I'll send you a copy. It says a little about this topic.)

Would others less familiar with control theory than I am try this? I don't
know, but I wouldn't be so sure they wouldn't. After all, it's just another
simple application of traditional research methods....

I'm sure that's what most of them would think. I'm certain of it. Just
another simple failure, by those who use traditional methods, to even
identify the phenomenon at hand, much less begin to analyze it and explain
it. :slight_smile:

But enough of this. As I said before, I agree that a proper analysis of such
control systems requires using the appropriate (control system) model.

Fine. We agree that to explain control, you must use the a control model,
of which the PCT model is one. But Rick and I are talking about using the
proper experimental methods, first. Before you analyze a behavioral
phenomenon, like control, you must observe it -- you must recognize that it
exists, in general, and that it occurred during the time when you observed
your participant. It's only when you have evidence that control occurred
that you need the PCT model.

What I
do not agree with is the idea that traditional methods are categorically
inappropriate when used to research human and animal behavioral capabilities.

Categorically? We never said that. But I contend that the assertion is
true for a majority of published behavioral research.

Tom Bourbon [950104.0950]

A quick question, Bruce. Have you read Phil Runkel's book, _Casting Nets
and Testing Specimens: Two Grand Methods of Psychology_?

No, but I plan to. Sounds interesting...

Do! Phil writes eloquently about many topics that obviously interest you.
He draws clear and insightful distinctions between issues that pertain to
research methods and designs, on the one hand, and to theories and models
of behavior, on the other.

Another quick (double) question. When you signed on to csg-l, you said you
were planning a new edition of your book on methods -- on all of the right
tools. Have you decided to include anything on PCT? If so, have you
decided how you will present the PCT material, relative to all of the
traditional tools? If so, how? (I guess this was a quick *triple*
question.)

I'd like to, but I'm not sure as yet how to proceed. Much of it would belong
in the chapter on single-subject research. How would you go about describing
"the" PCT approach to data-collection and analysis? Keep in mind that the
book is about how to conduct psychological research (methods-based), not about
specific areas of research (content-based). I do not get into specific detail
on the how to do psychophysical research, or human memory research, or
research on "learned helplessness," or research in other specific areas.

All the more reason for me to believe Phil and you should meet, even if only
through the medium of the printed page.

Bruce:

. . . However, I agree that there is
plenty of abuse going on, but remember, "statistics don't kill research,
researchers kill research." So vote "no" against the Brady Anti-statistics
Bill!"

You make my point, and, as a target shooter, I like the analogy. I have no
objection to statistics (guns). I object strongly to people who misuse
statistics (guns) in ways that harm innocent people. In my warped value
system (higher-level reference perceptions), that places many murders and
many behavioral scientists in the same category and I believe we should deal
harshly with them all. :wink:

Tom:

Yes, *if* we want to know which proportions of a particular species of
living control systems will do X, Y or Z, under conditions A, B, or C, then
group methods are the *only* way to go. But we had better not, on pain of
ranting by Marken and Bourbon, say we learned anything specific about any
individual, or about individuals in general -- remember the aggregated
curves and keep them holy. If we want to know about individuals, or about a
particular individual, we must study individuals, no two ways about it. :wink:

Bruce:

Hey, you're talkin' to an "n-of-one" kind-a guy! In general, I agree with
you. But there are some questions that can be addressed in no other way.
Some processes are irreversible; they can be observed only once in a given
participant. In those cases the aggregate methods may be the only game in
town.

Hey, right on, brother! Like, how many people wearing green coats went
into that corner drug store and bought pain relievers last Saturday, man?

ยทยทยท

==============

Rick: please tell me how you plan to study the properties of human memory
using the Test. Your results should allow you to explain, for example, how
it is that you are able to recognize a familiar face.

President Clinton has a large purple mark, in the shape of Arkansas, on his
forehead.

Chickens say, "moo."

It never rains in southern California.

==================

Later,

Tom