# The Test in practice, Why 99%

[From Rick Marken (920930.1100)]

Martin Taylor (920929 16:00) --

Since I am at the moment being asked to think of PCT-driven
experiments in planning and decisions-making tasks, I may be able to satisfy
condition (2) in that context.

I am starting to do something similar with satellite control tasks. I don't
know how it's going to work out but here's my thinking:

1) identify what seem to be the (perceptual) variables that need to be
controlled in the task. For example, in a satellite contact one variable that
must be controlled is "degree of connectivity". This is a complicated
variable whose value depends on all kinds of things; antenna availability,
location, status of processing equipment, etc. Of course, during the
pass this variable should be in a state very close to "connected".

2) identify factors that influence the states of these variables; distinguish
external (disturbance) factors from operator produced factors.

3) test the definitions of the variables in 1 by doing the test; I hope
we will be able to run simulations where, say, the operator is asked to
set up for a pass and will have various ways of influencing connectivity --
while (simulated) external factors also affect connectivity (antennae
go down, demuxes become unavailable, etc).

4) try to map variables in terms of being means or ends relative to each
other. This can be done by asking experts "how do you do this; why do
you do that" where doing is expressed in terms of the perceptual variables
identified in 3.

This is just talking through the ol' hat at the moment; if the group I'm
working with makes any progress I will keep you posted.

As you see from the above, "inferential statistics" is not what I am talking
about. What I am talking about would be more along the lines of parameter
estimation for the perceptual functions, gain estimates with variance, and
the like.

Then we are in furious agreement. I love it.

Dennis Delprato (920929)--

Recent discussion of The Test reminds me this could be the
subject of a great laboratory exercise.

Yes.

Any progress on the "Goal-Seeking with Random Consequences
of Responses" lab, Rick?

Nope. My kid took my computer so I can only work on it at work -- but that
would be wrong (to quote that great moral philosopher, R. M. Nixon).

But I think I would like to do the following: I would like to write
the lab in Basic. I would suggest that the lab be a study of
"reinforcement" and discriminative stimulus theory. Have them collect
data from a couple subjects; then have them try to explain the
results. They could choose from several models in the computer. Then
they could compare the behavior of the model to that of the subject.
The goal of the lab is to help the student understand that the concept
of reinforcement imples that there is a controlled perceptual variable --
in this case, what is controlled is the relative position of the moving
dot and one of the stationary dots on the screen.

I think it could be fun -- especially if you have a condition which
lets the experimenter manipulate the direction of the movement of the
dot after each press; once the experimenter has discovered the
controlled variable, he/she can then "control" the subject's bar
pressing rate by making the dot tend to move away from the target
dot after each press. So you can show that this kind of "control by rein-
forcement" is a consequence of the disturbance resistance characteristics
of a control system.

Greg Williams (920930 - 2) --

I believe that the highly precise predictions which have been achieved by some
PCT tracking models are due to the condition of "good" control being
satisfied in the experiments to which the predictions are applied.

ANY negative-feedback model with a reasonable
frequency response would predict tracking close to the target, yielding highly
precise predictions. But this is misleading, because the important question
(if you already believe in PCT) is how to choose BETWEEN different negative-
feedback models, and the only way to do this WITH HIGH PRECISION is by looking
at transient (temporarily "poor") control.

I agree. It is somewhat disingenuous of me to posture about the 99% accuracy
predictions when, in fact, they are virtually guaranteed by the fact that
we are dealing with situations where there is clearly control and we have
a damn good notion of the controlled variable. I do think that the improvements
in prediction that I spoke of are good indications of an improved
description of the controlled variable. But I'll try to lighten up the
bragging about 99% prediction. The reason I have harped on it is because I
want PCT to avoid succumbing to the statistical "cop out" which, to me, means
that you do a study (in our case, test for a controlled variable, say), find
noisy results (the subject resists the disturbance on x % of the trials)
and INSTEAD OF TRYING TO GET A BETTER DEFINITION OF THE CONTROLLED VARIABLE,
you apply a statistical test and say "the effect of the disturbance was
significant so I conclude that I have found a controlled variable". I want
PCT to avoid this approach in favor of working hard (and it will be hard --
doing the test for the controlled variable correctly will be a lot harder than
doing an ANOVA correctly; the test can be done "cook book" -- at least not
yet) to improve one's results (with 99% accuracy as the goal) by trying to
find improved definitions of the controlled variable and improved ways
of testing it. I hope someone knows how to do this; I sure don't.

Best regards

Rick

ยทยทยท

**************************************************************

Richard S. Marken USMail: 10459 Holman Ave
The Aerospace Corporation Los Angeles, CA 90024
E-mail: marken@aero.org
(310) 336-6214 (day)
(310) 474-0313 (evening)