Testing for the universal error curve

CSGnet Archive CSG1998
1

[From Bruce Abbott (980609.1715 EST)]

Rick Marken (980609.1420) --

i.kurtzer (980604.1330)

Not saying anything [about the universal error curve] until data
comes is the most intelligent thing to say.

Bruce Abbott (980604.1525 EST) --

How would you go about collecting data bearing on this issue?

I'd test 30 subjects in a three-level between subjects tracking task
with 10 subjects in each condition. The maximum size of the
disturbance would be varied across conditions: low, medium and high.
The dependent variable is the amount of time the subject stays in
the tracking task before quitting voluntarily (up to a maximum of
10 days;-)). Subjects would be carefully instructed about how they
should try to stay at the task as long as possible, for the good of
science. The universal error curve theory predicts that there will
be a significant effect of maximum disturbance on quitting; the
greater the maximum, the less time until quitting. ;-))

O.K., you've had your fun. It was a serious question; it deserves a serious
answer. Isaac said he didn't know, which is fair enough. Do you?

But there is a lot of anecdotal evidence for it.

It's going to take more than anecdotes to make a _scientific_ case for it.

I know there are other possible explanations for this phenomenon
but one is certainly the "universal error curve" (I think "universal"
just means that this error curve applies to all control systems in
the hierarchy).

If there are other possible explanations, then any anecdotal support for the
"universal error curve" will likely support those other explanations, too.
Any worthwhile experimental test would have to be constructed so as to rule
out one or more alternatives. How do you propose to do that?

By the way, your "support" for the "universal error curve" consists of the
sorts of observations the "universal error curve" was invented to explain.
Thus it is not too suprising that they "support" it.

Regards,

Bruce

2

[From Rick Marken (980609.1730)]

Me:

I'd test 30 subjects in a three-level between subjects tracking task
with 10 subjects in each condition...The universal error curve theory
predicts that there will be a significant effect of maximum
disturbance on quitting; the greater the maximum, the less time
until quitting. ;-))

Bruce Abbott (980609.1715 EST) --

O.K., you've had your fun. It was a serious question; it deserves
a serious answer.

What's wrong with my answer? It's an experimental design that I
learned from your methods textbook. I'm using a between, rather
than a within, subject design because I think a subject's exposure
to one condition may influence his/her performance in another. I
would, of course, control (hold constant) all variables that can
possibly be controlled; and I will eliminate any possibility of
systematic effects of uncontrollable variables by randomly
assigning subjects to conditions. The universal error curve
predicts that subjects will quit performing (controlling) when
the error becomes too large. This is most likely to happen in
the "large" maximum disturbance conditions. I would, of course,
have to do pilot tests to determine an appropriate the range of
maximum disturbance values to use in the experiment.

Just basic experimental psychology. What's the problem?

Best

Rick

···

--
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken/

3

[From Bruce Abbott (980610.1000 EST)]

Rick Marken (980609.1730) --

Bruce Abbott (980609.1715 EST)

O.K., you've had your fun. It was a serious question; it deserves
a serious answer.

What's wrong with my answer? It's an experimental design that I
learned from your methods textbook.

I seem to recall that you wrote your own methods textbook; surely you knew
all about this kind of experimental design before looking at my book. But
O.K., I'll take you at your word (for now): this is a serious answer from you.

I'm using a between, rather
than a within, subject design because I think a subject's exposure
to one condition may influence his/her performance in another.

Please explain. On what basis do you expect carryover to invalidate the
results?

I
would, of course, control (hold constant) all variables that can
possibly be controlled; and I will eliminate any possibility of
systematic effects of uncontrollable variables by randomly
assigning subjects to conditions.

As I'm sure you know, random assignment to treatments does not "eliminate
any possiblity of systematic effects of uncontrolled variables"; the chances
are good that random assignment will make these systematic effects small,
but the procedure does not guarantee that this will happen.

The universal error curve
predicts that subjects will quit performing (controlling) when
the error becomes too large.

Error in what? You told your participants that they "should try to stay at
the task as long as possible, for the good of science." Why should they
quit just because they can't get the cursor to stay over the target? Given
your instructions, they will most likely try to control something like
"pleasing the experimenter," or "doing good for science," and they could
continue to do this 'till the cows come home,' if the time being consumed
does not create errors in other CVs (like attending classes, or having
lunch, for example).

Furthermore, whatever effect would be predicted under the "universal error
curve" hypothesis would also be predicted by all other hypotheses under
which participants would quit when their efforts to control failed for too
long, so the proposed experiment has no power to discriminate among them.
The best that your experiment can hope to accomplish (and I seriously doubt
that it would do even that, for the reasons given above) is demonstrate the
phenomenon; it cannot help us to decide which explanation among several
plausible ones is correct.

I'm sure you would prefer to use a single-subject design to answer this
question (as would I), so you must have a very good reason for choosing a
group design instead. Apparently, you must believe that increasing the size
of the disturbance over time until the participant is no longer able to
control either would not result in the participant's giving up, or that
carryover from the weaker-disturbance values would confound the results. Or
maybe you have some other reason. The one I currently favor is that you
either cannot or do not wish to attempt a serious answer and wish to deflect
attention from this fact by engaging me in a debate about group-based designs.

Regards,

Bruce

4

[From Rick Marken (980610.1010)]

Bruce Abbott (980610.1000 EST)--

On what basis do you expect carryover to invalidate the
results?

I suspect that a subject who has already had success in the small
disturbance condition might be more inclined to "stick it out" in
the large disturbance condition than a subject who starts in the
large disturbance condition. But I don't know. Maybe a within
subjects design could work.

As I'm sure you know, random assignment to treatments does not
"eliminate any possiblity of systematic effects of uncontrolled
variables";

Right. Should have said random assignment "reduces the possibility..."

Me:

The universal error curve predicts that subjects will quit
performing (controlling) when the error becomes too large.

You.

Error in what?

In the subjects -- theoretical error.

You told your participants that they "should try to stay at
the task as long as possible, for the good of science." Why
should they quit just because they can't get the cursor to stay
over the target?

When the error reaches a certain point, further increases in error
lead to a _reduction_ in output. So when the error become large
enough the subject should just stop generating output.

Given your instructions, they will most likely try to control
something like "pleasing the experimenter," or "doing good for
science," and they could continue to do this 'till the cows come
home,'

According to my understanding of the universal error curve, when
the error becomes large enough the output goes to zero no matter
what. If you have a better suggestion about what the instructions
should be then please let me know.

Furthermore, whatever effect would be predicted under the
"universal error curve" hypothesis would also be predicted by
all other hypotheses under which participants would quit when
their efforts to control failed for too long, so the proposed
experiment has no power to discriminate among them.

I agree that this is a problem. The subjects could be quitting
because a higher level system changes the reference for the
tracking control system to zero. But at least this experiment
is a first step; the experiment is designed to get data on how
subjects behave when controlling at different levels of difficulty.
Once we see the results of this experiments we can design other
experiments that more precisely distinguish the alternative theories.
Isn't this the right way to do experimental psychology?

The best that your experiment can hope to accomplish (and I
seriously doubt that it would do even that, for the reasons
given above) is demonstrate the phenomenon;

Yes. Can you suggest a better experimental way to demonstrate the
phenomenon?

I'm sure you would prefer to use a single-subject design to
answer this question (as would I), so you must have a very
good reason for choosing a group design instead.

Yes. I assume that different subjects have different points at
which they go into the "no output" regime. By testing many
subjects in each condition I hope to average out these between
subject differences to see the true effect of the independent
variable (disturbance size) on the dependent variable (time until
quitting).

Apparently, you must believe that increasing the size of the
disturbance over time until the participant is no longer able to
control either would not result in the participant's giving up, or
that carryover from the weaker-disturbance values would confound
the results. Or maybe you have some other reason. The one I
currently favor is that you either cannot or do not wish to attempt
a serious answer and wish to deflect attention from this fact by
engaging me in a debate about group-based designs.

I am trying to design an experiment to test the idea that there
is a non-linear relationship between error and output (the
universal error curve) like this:

output | *
        > * *
        > * *
        > * *
      0 |* * * *

···

------------------------------------------
         0
                     error

Why not just suggest better ways to test this theory (using
single subjects if you like) instead of just criticizing my
design? How would you test the universal error curve theory?

Best

Rick
--
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken/