What about The Test

[From Rick Marken (951103.0800)]

Bruce Abbott (951103.0800)--

How is it that when Bill presents his hypotheses-to-be-tested that's fine,
but when I do the same, then it becomes a big problem for you? Enquiring
minds want to know.

Because I have never seen you test any of your hypotheses using The Test. I
suspect that up until now you have not even seen your explanations of
conventional data as hypotheses about controlled variables. When you start
tesing for controlled varables, I'll stop kvetching.

No such causal structure is necessarily assumed by any of these methods.

Your methods text says that we should hold extraneous variables constant in
an experiment in order to be sure that any relationship that might exists
between the independent variable (IV) and the dependent variable (IV), is
_causal_. This is the presumed advantage of experimental over correlational
methods: experimental methods let you determine whether or not there is a
_causal_ relationship between IV and DV; correlational methods don't.
Correlation does not imply causality, remember.

Experimental psychologists use the experimental method described in your
textbook to determine whether there is a causal relationship between IVs and
DVs. They do this becuase such causal relationships presumably tell them
something about the nature of the organism that mediates these relationships.
The experimental method described in your text is based on the assumption
that

DV = f (IV) (1)

where f() is assumed to be a characteristic of the organism (a threshold, an
integrator, a memory buffer, a processing lag, etc).

In a control system, however,

DV = -g(IV) (2)

where g() is the inverse of the environmental (feedback) function that
relates output (DV) to controlled (perceptual) variable (CV) -- the
possibility that there is a CV is _never_ even considered in the
experimental methods described in your text.

Furthermore, the fact that behavior is control-system action in no way
precludes causal relationships among observed variables... You seem to
think that the existence of control loops has banished causal
relationships.

Not at all. I think conventional methods assume that causal relationships
are arranged in a line (open loop); in a control system, the causal
relationships are arranged in a circle. The nature of the apparent causal
relationships that can be _seen_ in the behavior of a closed loop system
(disturbance-output relationships) are not what they seem (compare equation
(1) to equation 2).

So of course I believe in the existence of causal relationships between
variables. Perhaps it would be clearer if I said that conventional methods
assume _lineal_ rather than circular causal relationships. When there is a
circular, stable negative-feedback causal relationship between the
organism and its environment, ID-DV relationships reflect the nature of the
feedback function, not the organism. The experimental method used to study
such system must be oriented toward determined the CVs in the control loop.

By the way, this stuff is all delt with quite clearly in Powers' (1978)
"Spadewaok" paper in Psychological Review. You should know this already.

To suggest that these functions [perceptual input functions, memory,
and so on] cannot be studied by ordinary experimental methods is ludicrous.

You can use these methods to study these "functions" but what you find out
about these presumed functions will be misleading -- very misleading (compare
equations 1 and 2 above; also, try Gary Cziko's "conventional
experiment" version of the rubber band demo).

Well, well, so I CAN use one of the methods described in my text--the
EXPERIMENTAL method. I'll manipulate one variable (the disturbance)
while holding attempting to hold extraneous variables constant... and then
observe whether my participant is able to compensate for these disturbances
by making opposing movements of the mouse.

How would you know that there is "compensation" for your "disturbance"? All
you see is a relationship between two variables, IV and DV-- and usually a
noisy one at that. It's like observing that the garage door opens every time
a car drives up and closes when it goes in. Is the garage door "compensating"
for the "disturbance" of the car? In fact, we know that nothing is controlled
in this case; the garage door system is a lineal causal mechanism; but, if it
were a control system (controlling, perhaps, for having a car "inside") we
never know it from this kind of research.

You can only tell if there is "compensation" for a disturbance (and not
just a response to it) if you are monitoring a hypothetical controlled
variable. The experimental method you describe leaves out the crucial step of
identifying and monitoring the state of a hypothetical controlled variable.
This may seem like a small omission -- but if you are dealing with a
purposeful (closed-loop) system, you've left out the whole enchilada.

It is also interesting to note that the need to hold extraneous variables
constant is not part of The Test in the same way as it is part of
conventional research. In conventional research you hold extraneous
variables constant because you want to be sure that any effect of the IV on
the DV is, indeed, the result of the IV; you hold extraneous variable
constant in order to isolate the "true cause" of behavior (another indication
that these methods assume a lineal causal model of behavior). But you are not
looking for causal relationships when you do The Test -- you are looking for
controlled variables. In this case, what are considered "extraneous
variables" and how you deal with them is quite different than in the lineal
causal experiment.

For example, suppose that you are trying to determine the variable controlled
by a possible control system. Your first hypothesis might be that the system
is controlling the noise level at point X in a room. So you set up a noise
level meter at point X and start varying noise sources that should affect the
noise level at point X if there is no control. There is no need to control
for extraneous variables yet; the only extraneous variables of interest are
other sources of noise; if noise is controlled by the system under study,
all noise effects on the noise level at point X (including extraneous ones)
will be canceled and you will see that the noise measure varies far less
than expected. If noise is not controlled, extraneous noise just adds to the
variance on the noise measure at X; the hypothetical controlled variable
varies "more" than expected and you would correctly reject noise as a
controlled variable and move on; the extraneous noise does not affect your
conclusion about the variable under control.

Extraneous variables become a concern once you have identified a variable
that seems to be under control. If you find that its not noise but heat at
point X that seems to be under control, then you want to be sure that the
stability of this variable is not caused by some extraneous variable -- such
as coincidental changes in temperature outside the room that might have
compensated for your disturbances. This is why part of The Test involves
identifying the means the system uses to affect and sense the controlled
varable; you want to be sure that stability of a variable results from the
control actions of the systems, not by coincidental variations in
"extraneous" variables. Extraneous variables (in the Test) are variables,
other than the outputs of the control system (sych as the glue holding the
marble) that produce stability of a hypothetical controlled variable.

Lookie here--I vary the disturbance, and my participant varies the mouse
position: cause-effect! I thought you said the methods I describe are
USELESS for studying control

What have you learned about control from doing this experiment? I contend
that you have learned nothing at all. You were not monitoring the effect of
your disturbance on a possible controlled variable so you have no idea _what_
variable is being controlled or even _whether_ a variable is being
controlled. The cause-effect relationship that you are calling "disturbance
resistance" may simply be an open-loop response to the stimulus. When I move
my racquet into the path of an oncoming racquetball, the ball abruptly
changes direction;. Lookie, I vary the IV (position of racquet) and the ball
varies its position (DV): disturbance resistance! Yeah. Right.

My book is about research methods that can be applied when one is asking
certain general kinds of questions (like do variables A and B tend to vary
together, or what proportion of the population believes in astrology),
whether in psychology, education, medicine, or any other area in which they
would apply. It is not about testing a specific theory of behavior.

The research methods described in your book do not consider the possibility
that the system under study might be organised around _control_ of an input
variable. Therefore, these methods cannot reveal whether the system under
study is organized as a lineal causal or as a control system.

We've been through all this many times before and I don't expect you to be
convinced this time; it's not easy to revise references for principles (like
the principles involved in doing scientific research). I say this stuff
mostly for the sake of lurkers who might be in a better position to revise
their ideas about behavioral science methodology or who do not yet have
intensely defended ideas about how to study behavior.

I think it is important for those who want to learn PCT to know that PCT is
not just an alternative theory of the data that has already been obtained
using conventional methodology. PCT requires a change in perspective
regarding the nature of behavior itself (behavior is a control, not a lineal
causal, process) and how to go about understanding it. It's a difficult
change for people who already have a career in behavioral research; it may
be an earier change (or no change at all) for neophyte behavioral scientists.
Those people are my target audience.

Best

Rick