A new methods article

CSGnet Archive CSG2001
1

I came across an article entitled: Estimating and Testing Mediation and Moderation in Within-Subject Designs. It appeared in Psychological Methods, June 2001 Vol 6, No.2 115-134. The authors are Charles M. Judd, David A. Kenny and Gary H. McClelland. I learned about this article from the APA Monitor which heralded it as a big break-through in methodolgy.

My question: Does this new methodology approach have anything to offer to PCT research?

What do I think? I am not sure. I know that in depth studies of individual cases is consistent with PCT. That is, within-subject designs is preferred over between subject designs.

I know that the attempt to find out what is going on within a person is consistent with PCT. Therefore, the attempt to decide whether an independent variable is a mediator or a moderator is a good idea.

If we take the basic pursuit tracking task and ask some questions: (a) What variables make a difference in how well a person controls in the pursuit tracking task? (b) What does this tell us about the person? The above approach would seem to apply.

What do you all think?

David M. Goldstein, Ph.D.

2

[From Bill Powers (2001.09.20.1423 MDT)]

David Goldstein (2001.08.20) --

>I came across an article entitled: Estimating and Testing Mediation and
Moderation in Within-Subject Designs. It appeared in Psychological Methods,
June 2001 Vol 6, No.2 115-134. The authors are Charles M. Judd, David A.
Kenny and Gary H. McClelland. I learned about this article from the APA
Monitor which heralded it as a big break-through in methodolgy.

My question: Does this new methodology approach have anything to offer to

PCT research?

How about telling us what it is? Not many of us have access of
"Psychological Methods" and other such literature.

Best,

Bill P.

3

[from Jeff Vancouver (2001.08.20.1650 EST)]

David,

I saw the same reference and have sent one of my students to get the paper
(i.e., I have not read it yet). However, from reading the description in
the Monitor, I got they impression that what makes the paper so
break-through it that is provides an easy way to do what some are doing
using more sophisticated analysis procedures (e.g., HLM). I am one of those
people who use the more sophisticated procedures and have found it helpful
in dealing with PCT issues (I used it in the first and third studies I
presented at CSG in St. Louis). Nonetheless, it has its limitations and
indeed, one of the reasons I sent my student for the paper is we are
currently working on a paper that outlines the limitations and talking about
other ways of testing for mediation.

Jeff

···

-----Original Message-----
From: Control Systems Group Network (CSGnet)
[mailto:CSGNET@listserv.uiuc.edu]On Behalf Of David Goldstein
Sent: Monday, August 20, 2001 2:05 PM
To: CSGNET@listserv.uiuc.edu
Subject: A new methods article

I came across an article entitled: Estimating and Testing
Mediation and Moderation in Within-Subject Designs. It
appeared in Psychological Methods, June 2001 Vol 6, No.2
115-134. The authors are Charles M. Judd, David A. Kenny and
Gary H. McClelland. I learned about this article from the APA
Monitor which heralded it as a big break-through in methodolgy.

My question: Does this new methodology approach have anything
to offer to PCT research?

What do I think? I am not sure. I know that in depth studies
of individual cases is consistent with PCT. That is,
within-subject designs is preferred over between subject designs.

I know that the attempt to find out what is going on within a
person is consistent with PCT. Therefore, the attempt to
decide whether an independent variable is a mediator or a
moderator is a good idea.

If we take the basic pursuit tracking task and ask some
questions: (a) What variables make a difference in how well a
person controls in the pursuit tracking task? (b) What does
this tell us about the person? The above approach would seem to apply.

What do you all think?

David M. Goldstein, Ph.D.

4

David M. Goldstein (2001.08.21.1241pm)

Bill Powers and Jeff Vancouver responded to my message. Actually, Jeff seems
to already be using a more sophisticated version of the method in the
article. However, I will give an explanation a try, by means of an example.
Jeff can probably tell us more.

Suppose that we want to find out if "attention/concentration" makes a
difference in how well a person controls in a pursuit tracking task. The
treatment effect is the ease/hardness of the task. Is
attention/concentration a mediator or a moderator?

As a mediator, attention/concentration causes the treatment effect, by
definition. As a moderator, attention/concentration affects the magnitude of
the effect.

Suppose that a person is doing a pursuit tracking task. Let us say that a
person does the task 10 times for 2 minutes each time. We pick a disturbance
function randomly for each trial. This varies the ease/hardness of the task.
We divide the distribution of disturbance functions in half, one-half above
and one-half below the median. This defines two treatment conditions: easy,
below the median and hard, above the median.

This experiment is repeated for 25 subjects.

The outcome measure is the stability statistic for each trial. For each
person we look at the difference score for the Hard-Easy task.

The person is hooked up to an EEG machine that measures his EEG at the FZ
10/20 location. For each trial, we record the theta/beta ratio. This ratio
is often thought of as a measure of concentration/attention. The theta/beta
ratio is the variable we want to test if it is a mediator or moderator.

So, for 25 subjects we have three variables: (1) Outcome Difference score
for the Hard-Easy Task, (2) Theta/beta ratio for Hard - Theta/beta ratio for
Easy, (3) Theta/beta ratio for Hard + Theta/beta ratio for Easy.

We do multiple regression statistical analysis of variable (1) against
variables (2) and variables (3).

" To asess both mediation and moderation due to a concomitant variable that
varies between treatment conditions, one regresses the Y difference on both
the X sum and the X difference. Assuming that there is an overall treatment
effect on X and that the X difference predicts the Y difference, mediation
of the treatment effect in Y by X is indicated (assuming that X and Y are
scaled to have a positive relationship and the treatment effects in Y and X
are in the same direction). If the X sum is predictive of the Y difference,
then X also serves as a moderator of the treatment effect and, equivalently,
X relates to Y differently in the two treatment conditions. Finally, if the
X sum has been centered, then the intercept in this regression equation will
equal the the magnitude of the residual treatment difference in Y, over and
above mediation due to X and the mean of value of X. In other words, it will
equal the portion of the mean treatment effectg that is not mediated through
X."

This is probably as good as I understand it right now.

Attention/concentration could be a moderator and/or a mediator factor or
neither.

In the pursuit tracking task, if a person stops trying to control,
attention/concentration is focused off the task. In the sense that
concentration/attention must be present for any degree of control to be
present, it should turn out to be mediator. If a person is simultaneously
attending/concentrating on other stuff, performance will not be as good.
Attention/concentration should therefore turn out to be a moderator as well.

5

[From Rick Marken (2001.08.23.0930)]

David Goldstein said:

I came across an article entitled: Estimating and Testing Mediation
and Moderation in Within-Subject Designs. It appeared in Psychological
Methods, June 2001 Vol 6, No.2 115-134. The authors are Charles M. Judd,
David A. Kenny and Gary H. McClelland. I learned about this article
from the APA Monitor which heralded it as a big break-through in
methodolgy.

Gee. I published a paper in Psychological Methods some time ago but the
Monitor failed to hail it as a big breakthrough in methodology. And my
paper actually _did_ describe a big breakthrough in methodology. Go figure.

My question: Does this new methodology approach have anything to offer
to PCT research?

Based on the abstract and your [David M. Goldstein (2001.08.21.1241pm)]
description of the methodology, my answer would be "no". This is
standard group based research methodology using two within subjects
covariates: x1+x2 and x1-x2 where x1 and x2 are measures of variable x
for the same subject in two different conditions, 1 and 2. The results
tell you nothing about the nature of the effect of these covariates on
each individual subject. Moreover, the methodology is based on the wrong
model of behavior--the general linear model-- so even if it were applied
to individual cases (as it often is in the study of perception and
psychophysics) it would still give you misleading results (due to the
behavioral illusion).

But it is a clever idea in the context of the conventional model of
behavior. Maybe that's why the folks at the Monitor got so excited.

Always great to hear from you, David.

Best regards

Rick

···

--
Richard S. Marken, Ph.D.
MindReadings.com
10459 Holman Ave
Los Angeles, CA 90024
Tel: 310-474-0313
E-mail: marken@mindreadings.com

6

[From Bill Powers (2001.08.23.1023 MDT)]
[Attachment: MM.exe, a bitmap graphic]

David M. Goldstein (2001.08.21.1241pm)

Bill Powers and Jeff Vancouver responded to my message. Actually, Jeff seems
to already be using a more sophisticated version of the method in the
article. However, I will give an explanation a try, by means of an example.
Jeff can probably tell us more.

Suppose that we want to find out if "attention/concentration" makes a
difference in how well a person controls in a pursuit tracking task. The
treatment effect is the ease/hardness of the task. Is
attention/concentration a mediator or a moderator?

As a mediator, attention/concentration causes the treatment effect, by
definition. As a moderator, attention/concentration affects the magnitude of
the effect.

I'm not clear about what you mean here. "Mediate," according to my
understanding and my dictionary, means "Acting by an intermediate cause or
instrument; not direct." If I poke a balloon with a pin, my act is the
im-mediate (not mediated) cause of the balloon's popping. If I poke you
with the pin, and you jump and accidentally pop the balloon you're holding,
my effect on the balloon is _mediated_ by you. So if
attention/concentration is a mediator, it must come between some cause and
its ultimate effect. The mediator is the immediate cause of the effect, but
there is a prior cause, not mentioned, that makes the mediator act. Would
the "mediated" proposal be that the cause (whatever it is) acts on
attention/concentration, which in turn produces the final effect?

I can understand "moderator" as you describe it, although you still don't
indicate the cause of the effect. As I would see it, a moderater acts to
alter the connection between a cause and an effect, while a mediator _is_
the connection between the cause and the effect.

I'm further confused by your reference to a "treatment effect." You say
that the "ease/hardness" of the task is the treatment effect. But what is
it the effect OF? I would think of difficulty level as an independent
variable, not an "effect" in the sense of resulting from some
experimentally-observed cause. The experimenter simply sets the difficulty
level prior to each experiment.

At any rate, the attached diagram shows my understanding of the
"mediation-moderation" difference -- is it correct? MM.exe is a
self-extracting zipped file which will produce file called mediate.bmp. Use
the paint or paintbrush program to look at it, or just double-click on it

Best,

Bill P.

Mm.exe (50 Bytes)

···

from My Computer or Explorer.

7

from David M. Goldstein (2001.24.8.841am)

Thanks to Rick Marken (2001.08.23.0930) and Bill Powers (2001.08.23.1023
MDT) for responding to my email about a new method.

Rick said that he doesn't think the new method will contribute anything to
PCT research.

He says: "The results tell you nothing about the nature of the effect of
these covariates on each individual subject. Moreover, the methodology is
based on the wrong model of behavior--the general linear model-- so even if
it were applied to individual cases (as it often is in the study of
perception and psychophysics) it would still give you misleading results
(due to the behavioral illusion)."

I kind of agree with this, but have to think about it more.

Bill asks some good questions designed to clarify the meaning the terms
mediator and moderator. I am going to have to read the article a lot more
carefully to answer Bill's questions.

In one part of the article the authors say: "These are alternative reasons
why the performance difference is found, one might plausibly hypothesize
different mediating mechanisms for this effect. " So, with mediator
variables, we are talking about mechanism.

The easy/hard variable is an independent variable. Is
attention/concentration one of the mediating variables for any difference in
performance (control as measured by the stability statistic) between the
easy and hard conditions? Maybe. I seem to remember that Bill's model
analyzes performance in the task with two parameters. One is a delay
parameter in the input function. The other is an integration parmeter
associated with the output function. I can see that the easy/hard difference
might challenge a person differently. The hard tasks are a greater
challenge. In order to perform this task well, a person might have to
concentrate/pay attention more than in the easy task. The person can alter
how hard he/she is concentrating/paying attention. This shows up as a change
in the theta/beta ratio.

Without knowing Bill's model of the task, knowing that
attention/concentration is a mediating variable doesn't help a lot. In
Bill's model of the task, the two parameters are the mediating variables. If
we found out that the theta/beta ratio related strongly to one or both of
these parameters, then we have found out something new about the person. We
have found out how the person may be able to manipulate the parameters.