arrested development; threecv1 stuff

[From Bill Powers (950122.1550 MST)]

Rick Marken (950122.1115) --

Officer, I'm not as think as you drunk I am. I merely changed
definitions in the middle of the stream. Yes, the basic fact is that c =
h + d no matter how you look at it. That's the way we write the
programs.

Let's simplify. Suppose you see a target moving in a nice triangular
waveform, and you see the cursor moving in the same waveform but with a
10-pixel displacement: simple pursuit tracking. The question is, is this
person controlling the cursor to make it maintain a 10-pixel
displacement, or a 0-pixel displacement? If the former, then we would
conclude that there is probably no disturbance being applied to the
cursor; if the latter, that there is a rather large constant
disturbance.

Ah, you say, but the handle position will tell us which is which. If the
handle is far to one side, then there must be a disturbance. BUT,
officer, that assumes that whoever wrote this program wrote it the same
way we write ours -- with a centered handle providing zero effect on the
cursor, and a disturbance with a measure of zero having zero effect on
the cursor. In fact, approaching this situation observationally, without
knowing all the underlying machinery, we simply can't tell what is
happening. Or rather, many different scenarios would work just as well.
That, at least, is my suspicion. I suspect that in order to characterize
the control system, we must do experiments with a known environmental
situation.

So my basic question remains. Is there an experimental procedure that
will give the correct analysis to an infinitely wise person who does not
know how we wrote the program, what the person's intention is, or what
the disturbance is? I will stipulate linear functions.

···

-----------------------------------------------------------------------
Bruce Abbott (950122.1340 EST) --

     The main problem, if that's what it is, has to do with the initial
     values of k and delta in the FITMODEL functiton:

Yes, you've explained the blowup nicely, and no, k should not be allowed
to go negative (that produces positive feedback as you discovered). I'm
glad you're doing all this debugging, by the way, even though it does
make me look a bit slapdash. The best cure is to put limits on k so it
stays above some implausibly low level but still positive, and stays
less than 1.0. The latter problem, by the way, is an artifact of the
digital modeling of an analog situation. In a true analog computer that
problem would not arise. Since the biggest k values I have seen are less
than 0.2, with a dt of 1/60 sec., I didn't safeguard against that one.
In the threecv program I didn't use ANY safeguards.

Of course, that's why I separated FITMODEL from RUNMODEL. Then I forgot
to use RUNMODEL. Typical.

There may be better ways of automating the analysis, or equally good
ways that don't risk blowing up. I used the successive approximation
method because it's fast and seems to give reproducible results. Perhaps
you can think of a statistical way to derive k that does it all in one
pass of the data, which would be lovely and still faster. However, one
has to be careful that the results are reproducible.

The disadvantage of my method is that when you near the minimum squared
error, there may be more than one minimum -- the "local minimum"
problem. I haven't looked for a better way to get the parameters because
so far the parameters have been good enough to predict performance with
sufficient accuracy for our humble purposes. But there may be both a
better and quicker way, and if you can find one I will happily adopt it
(as will Martin Taylor).

     Another small bug is that the final display of the model versus
     handle positions calls FITMODEL instead of MODELRUN. As a result,
     the whole fitting process repeats, and any user-entered alternative
     k value gets replaced with the fitted value. Calling MODELRUN
     instead at this point just runs the model with the current k value,
     which is, I think, what was intended. I've modified the code to do
     this.

And so will I as soon as I finish here. Good find, as they say in NASA.
Of course, that's why I separated FITMODEL from RUNMODEL. Then I forgot
to use RUNMODEL. Typical. Now you see how I can program so fast.

     Just prior to reading your post I had been thinking about the
     situation in which the disturbance is added to the target position
     rather than to the cursor position.

This is what we call "simple pursuit tracking." If you then add a second
disturbance (to the cursor), this is "disturbed pursuit tracking." In
both cases, the controlled variable is cursor minus target, and in the
latter case the effective disturbance is the difference between the two
disturbing waveforms.

     This [pursuit tracking] would seem to be a situation far easier to
     analyze using a conventional regression approach than the
     compensatory tracking task.

What you would end up with is a high correlation between cursor and
handle position, and between handle position and target position. So it
would seem that the conventional S-R approach would work. This was the
first case handled in Models and Their Worlds, in which the S-R model
apparently works.

     Now consider the situation in which separate disturbances are being
     applied BOTH to the cursors AND to the targets. Is there any way
     to separate the effects of the two disturbances on the controlled
     cursor-target system? My guess is that there isn't, unless you
     have a measurable external target and can assume that the target
     position IS the subject's reference.

This was the third case in Models and their Worlds. The S-R model fails
this one, of course.

Now comes the payoff of assuming that cursor minus target is the
controlled variable. The disturbance of the target and the disturbance
of the cursor can be combined into a single equivalent disturbance, Dc -
Dt. The controlled variable remains near zero, and the handle position
is essentially equal and opposite to Dc - Dt. So the model works just as
well as ever to predict handle positions. Try it -- I think you can
squeeze in three more disturbances without any problem. Just remember
that in the model, the controlled perception is p[i] = c[i] - t[i], and
the disturbance is Dc[i] - Dt[i]. The reference level is the average of
c[i] - t[i].

     On the tracking task involving a disturbance applied to the target
     rather than to the cursor, the disturbance to the target is visible
     regardless of handle movements, thus posing an easier (if still
     misleading) problem for the analyst. For either task, what is easy
     to miss is the important thing--that control acts to minimize the
     error between target (assuming target = reference) and cursor,
     whether the target is stationary or moving.

You got it. For the S-R interpretation, the critical test comes when the
disturbance of the cursor is introduced. Then it no longer works to say
that the handle position is simply a response to the target position.
-------------------------
     Once you have identified the correlation between disturbance and
     handle position, you would seem to have demonstrated a simple case
     of cause-and-effect, so I can understand why they would respond in
     this way. Nothing earth-shaking here: when the cursor goes that
     way, you move the mouse in the opposite direction. So what? How
     do you respond to that? Get out your coin and rubber bands?

Ah, but this isn't right: the cursor movements are not "the disturbance"
in the compensatory tracking case. That's the point where you bring in
the correlation between cursor movements and handle movements, showing
that they _don't_ correlate. You can't simply attribute the handle
movements to the cursor movements (not even first derivatives).

It's a bit hard to get across the idea that "a disturbance" is not the
same as "a perturbation of the cursor position." The disturbance," or as
I like to call it when trying hard to maintain this distinction, "the
disturbing variable," varies, in threecv1, over a range from +80 to -80
in a certain waveform. When you plot this disturbance against the
cursor, you end up with Lower Michigan. The cursor waveform bears
essentially no relationship to the disturbing variable's waveform _or_
to the handle's waveform..

If you really want to cause some head-scratching, show this diagram:

                      0.05 0.03
        disturbance <------> cursor <----------> handle position
            > >
             <---------------------------------------->
                              -0.996

.. or whatever values you get. Remind the head-scratcher that the only
variables the subject can SEE on the screen are the cursor and
(constant) target positions. So all that the subject can know about the
disturbance has to come via the effects of the disturbance on the
cursor. The handle position correlates almost perfectly with the
disturbance, yet the only means of getting information about the
disturbance, the cursor position, correlates only 0.03 or whatever
(normally less than 0.2) with the handle position.

I showed this to a old psychology professor of mine at Northwestern
University, and after he ascertained that I was really claiming that
these were actual findings, he turned purple, shouted that he wasn't
going to listen to any more of this nonsense, and left.

     What about the following demo? You have your colleagues watch
     someone tracking a moving target around the screen. The mouse
     controls a small white circle; the target is a same-size red
     circle. Your colleagues are watching the screen while wearing red
     goggles, which renders the target circle invisible to them (but not
     to the participant, who is not wearing goggles). All they see is
     the white circle dancing around the screen. At the end of the task
     the screen clears and the colleagues are allowed to remove their
     goggles. You then ask them to explain the participant's cursor-
     moving behavior.

Oh, lovely! Dag Forsell has a somewhat simpler, but not nearly as
elegant, demonstration. He does the rubber-band experiment against a
sheet of newsprint, having the participant hold a dry-marker against the
paper with the same hand that holds the rubber band. Then, after putting
in a variety of disturbances while the participant holds the knot over a
dot, Dag asks the audience "Now, supposing someone walked through the
door right now and was told that this squiggly line represented this
person's behavior for the last 30 seconds. What would the answer be if I
asked what this person here has been doing?"

Incidentally, the answer to people who say the participant is simply
mirroring the experimenters moves is to pass the rubber band through a
small hole in a large piece of cardboard which obscures the
experimenter's end of the rubber bands from the participant's view.
Makes no difference.

     After I've had a chance to "clean them up," I'll post the revised
     THREECV1 and PCT analysis programs. Perhaps others will want to
     get in on the fun. At this point, I've gotten five of my colleagues
     to run through the task and have given them a copy of their own
     data to analyze as they see fit.

Great. Rick now has Turbo P 7, and will be able to join in. If any
others want to, please feel free (Chuck Tucker, my offer to talk you
through setting up your TP-7 still holds -- call me while you're in
front of your computer). Tell them, Bruce: there is absolutely no
substitute for seeing these phenomena "live." After you get some results
from your five professors, there may be others who would like to use
your programs to extend this research project!

I'm thinking of presenting the threecv experiment in another disguise.
Imagine a triangle on the screen which is varied in size, orientation,
and position. The mouse affects all three equally; each is disturbed by
a different disturbance. The participant is asked to use the mouse to
keep one "aspect" of the triangle constant. This will obviously work. It
illustrates how controlling something at one level requires controlling
its attributes at a lower level, and how you need one degree of freedom
of action to control each attribute. What we speak of as "controlling
the triangle" has to be broken down into specific one-dimensional
aspects of the thing to be controlled. And of course with only one
output degree of freedom, we can control only one attribute at a time,
while the others remain uncontrolled.
---------------------------------------------------------------------
Best,

Bill P.

Tom Bourbon [950123.1051]

It looks like those of us in the US who live west of the Mississippi
are the ones experiencing trouble with csg-l. Either that, or it's all
a plot to keep Marken and me off the air. :wink:

Avery Andrews is also having trouble. I'm not sure how to characterize his
location relative to the Mississippi. :slight_smile:

[From Bill Powers (950122.1550 MST)]

. . .

-----------------------------------------------------------------------
Bruce Abbott (950122.1340 EST) --

. . .

Bill P:

I'm thinking of presenting the threecv experiment in another disguise.
Imagine a triangle on the screen which is varied in size, orientation,
and position. The mouse affects all three equally; each is disturbed by
a different disturbance. The participant is asked to use the mouse to
keep one "aspect" of the triangle constant. This will obviously work. It
illustrates how controlling something at one level requires controlling
its attributes at a lower level, and how you need one degree of freedom
of action to control each attribute. What we speak of as "controlling
the triangle" has to be broken down into specific one-dimensional
aspects of the thing to be controlled. And of course with only one
output degree of freedom, we can control only one attribute at a time,
while the others remain uncontrolled.

That would be a good modification, Bill. You would certainly be able to see
whenever the person switched from one controlled attribute to another.

We can alse set up a variation on this kind of demo so that X-axis movement
of the mouse affects one attribute of the object, Y-axis another, and a
button-press switches one of the mouse axes to affect the third attribute.
Then you can see which one or two of the attributes the person controls at
any given time and the switch from one to another via the button is obvious.

Then again, if someone has a computer equipped with both a mouse and a game
port, we can set up a 4-df object (I guess we could go to 6 df) and the
person can simultaneously control all four. A few years ago, one of my
thesis students, Wade Harman, used a setup like that. The target was an
arrowhead:
                 .
                . .
               . .
              . .
             . .
            . . .
           . . . .
          . . . .
         . . . .
        .. ..

It simultaneously changed randomly in four attributes: X and Y positions of
the tip, Z-axis change in the lengths of the sides, and displacement of the
tip in degrees of azimuth (rotation). A participant made a similar arrowhead
track the target in all four df. The two axes of the mouse affected two df;
the two axes of a joystick, the other two df. Everybody could do it after a
little practice -- the amount of practice varied from person to person --
and four independent PCt models running simultaneously matched their
performance with correlations ranging from .91 to .99. (By the way, Wade
gave a nice interpretation of instances when person-model correlations are
low. His discussion is relevant to the present thread on csg-l about data
from the sleep study reported by Martin Taylor.)

During the practice runs for some participants, an interesting phenomenon
shows up. Sometimes a person will quickly discover how to simultaneously
control two or three attributes, but have difficulty controlling the other
one(s). It is not unusual for the person to ignore the controllable
attributes and concentrate on the other one(s) until "under control," then
resume controlling the original ones along with the new one(s). Ray
Pavloski and his associates reported a similar phenomenon during the
reorganization phase in an experiment where people simultaneously controlled
two markedly different variables -- cursor-target relationship, and
numerical sums. Ray and his colleagues called the phenomenon "task
switching." By any name, it seems to reflect reorganization in a hierarchy
that is tinkering with its own parameters while it learns how to control
multiple dfs. There is lots of room for some good research projects on
that subject. Oh, to have students once again!

Later,

Tom