Experiment with controlling uncertainty

[From Bill Powers (940603.1015 MDT)]

Martin Taylor (940603.0930 MDT) --

Don't confuse the uncertainty used by the analyst with the
uncertainty currently under discussion when we look into Paul
Revere's head. If Paul is to perceive uncertainty, and to
control that perception, he has to have an ECS whose input
function provides a signal that is the level of uncertainty in
some other perceptual signal about its CEV. That "uncertainty-
control" ECS cannot use at all the same inputs as the analyst
would, and it probably will not get the same results the
analyst would.

We're making some progress. You have now defined subjective
uncertainty in a way that allows applying the Test. I'm
uncomfortable with drawing conclusions about what goes on in Paul
Revere's head, because you can put anything there, in imagination,
that will make the argument come out the way you want.

The easiest way to apply the Test is to set up a control task (e.g.,
every tracking task is an application of the Test). So we need first
a control system in which there is an uncertain connection between
its CEV and the perceptual signal, and of course a means of
affecting the perceptual signal via action on the CEV. Then we need
another control system which senses both the CEV and the perceptual
signal of the first system and perceives the uncertainty in the
relationship, and has some means of altering the uncertainty (which
can easily be provided by a program).

You're going to have to help out here. Suppose the CEV is the actual
relationship between a cursor and a target, and the perceptual
signal is the perceived relationship (you can substitute any other
task). We know how to model the person in this task. But I'm having
some trouble imagining how to set up the second task so the person
can monitor the CEV as well as the perceptual signal in order to
perceive the irregularity in the relationship. And come to think of
it, I'm having a problem with seeing how to introduce an uncertainty
between the CEV and the perceptual signal, so the CEV has a known
value but the perceptual signal is an uncertain representation of
it.

We have to find a way of setting up a CEV so we can know what it
"really" is, but in such a way that the participant in the
experiment gets only an uncertain perception of it. Then we have to
think of a second way for the same person to perceive the CEV, this
time accurately perceiving its actual state, and also being aware of
the uncertain perception of it.

This seems to create a logical problem. We have an uncertain
perception but a known CEV. This implies that we can define the CEV
objectively, and introduce an uncertainty such that the perceptual
signal could represent an apparent value of the CEV that is
different from the actual value. But in that case, wouldn't the Test
give the wrong definition of the CEV? When the person claims to be
maintaining a constant value of the perception, the actual CEV would
be varying. But the Test can use only the actual CEV as the basis
for judging resistance to disturbance and computing the reference
level. If we used that as the basis for defining the CEV, we would
have to conclude that the reference signal is varying.

I'm stuck on that one for the moment. Perhaps you have some
suggestions.

The second problem seems just as difficult. Somehow we have to
provide the person with a second way of perceiving the same CEV,
only this time without uncertainty. This implies a perceptual
function just like the one used in the first control system, but
free of uncertainty in its representation of the CEV. We can't use a
copy of the perceptual signal from the first system (i.e., the
perception of the CEV that the person is already getting through the
input function of the first control system), because that is the
signal we want to compare with a second signal representing the true
value of the CEV. We must duplicate the form of the function in the
first control system, but this time making sure that there is no
uncertainty in it. In experimental terms, we must provide the person
with a second way of perceiving the same CEV which will give the
person accurate information about its state.

But I confess that I can't see why, if the person had a way of
perceiving the true state of the CEV, the person wouldn't simply use
that input function for controlling it in the first place. How could
we arrange the experiment so the person would use an uncertain way
of perceiving the CEV in controlling it, and would use a certain way
of perceiving it only as a basis for adjusting the uncertainty
(using means we provide) in the first way of perceiving it? It seems
to me that in saying that the person is given some way of perceiving
the true state of the CEV, we have made it unlikely that the
experiment could be done at all.

Over to you.

···

--------------------------------------------------------------
Best,

Bill P.

[Martin Taylor 940603 14:00]

Bill Powers (940603.1015 MDT)
:Rick Marken (940603.0830)

We're making some progress. You have now defined subjective
uncertainty in a way that allows applying the Test.
...
The easiest way to apply the Test is to set up a control task (e.g.,
every tracking task is an application of the Test). So we need first
a control system in which there is an uncertain connection between
its CEV and the perceptual signal, and of course a means of
affecting the perceptual signal via action on the CEV.

:I thought of a "control of uncertainty" study. A signal (S) light appears
:(say every second) on the computer screnn and then another light (A) appears
:below it and to the right of left with some probability. Eg.
: S
: A1 A2
:
:So after each S, either A1 or A2 appears. The uncertainty is determined by
:the probaility of A1, p(A1), which is 1-p(A2). So if p(A1)=1.0 then there is
:complete certainty that A1 appears after S; intermediate values of p(A1)
:result in intermediate values of uncertainty about whether A1 or A2 will
:appear after each S. Max uncertainty is when p(A1) = .5. The subject controls
:the uncertainty by moving the mouse (m) which changes the value of p(A1). A
:distrubance (d) is also applied to p(A1) so p(A1) = d + m (with the
:constraint that p(A1) be in the range from 0 to 1.0).
:
:Would this qualify as a control if "uncertainty" experiment?

Do you two guys ever talk to one another?

I think that we can put these ideas together into a neat experiment, but I
can't in any practical sense follow up on Rick's (apparently nauseating):

:How about sicking a research associate on this?

The experiment I see as based on these ideas is a simple compensatory
tracking experiment, in which the subject has to keep a cursor in its
reference position. Quite ordinary, except that the subject doesn't see
the target directly. Instead, what happens is that the subject sees
Rick's display, and the proportion of A1 events is some function of
the placement of the cursor relative to the target. Rick proposes changing
the p(A1) by moving the mouse. That's exactly what would happen in my variant,
but the "real-world" variable through which the mouse affects the probability
would be the position of the cursor, visible to an outside observer but
not to the subject. (Just as the position of the British Army might be
visible to an outside observer, but not to Revere).

Here's a variant that is analogous to Paul Revere and the disguised British
infiltrators. The subject has the same display as before, and p(A1) is
again controlled by a function of the cursor position. But this time,
the subject has to make discrete choices. The function p(A1) is zero
for cursor locations to the left of position X and 1.0 for positions to
the right of position Y. Between X and Y, p(A1) is linearly proportional
to the distance of the cursor from X. The cursor starts each "trial" at
the centre position, and the subject's job is to keep it there. But
the mechanism is different from before. There is no mouse, but only two
buttons. The cursor moves slowly and at a constant rate from the centre
toward X or toward Y. If the subject pushes the button that opposes the
motion, the cursor reverses direction, but if the wrong button is pushed,
it accelerates away from centre and an "error" is recorded. In an extreme
version of this, pushing the correct button terminates the trial and begins
a new one with the cursor at centre moving in a random direction, whereas
pushing the wrong button sends it to one or other extreme in the direction
it was going, and the subject has to push the correct button as soom as
possible to get it back to centre.

The first experiment above allows for the subject to maintain any desired
reference probability, in that there could be another pair of lights
controlled by the same probability function to set the reference position
of the cursor relative to the target (same as Tom choosing a reference of
1 cm above the target for his tracking study). The second experiment
deals with the accumulation of evidence, because "bad things" happen only
if the subject gets it wrong or delays too long in making a choice. The
rate of the lights flashing can be adjusted to provide an experimental
limitation on the possible rate at which the subject could possibly
accumulate information. This is directly analogous to visual perception
in fog or dim illumination, conditions in which the rate of accumulation
of visual information is reduced from normal.

I should think Rick could code both of these in HyperCard without too
much difficulty, if he wants. I'm not likely to, being up to the ears
in alligator swamp.

Martin