[From Bill Powers (980327.1721 MST)]
Martin Taylor 980327 17:15 --
Excellent! Thank you for doing this. If these data are typical, it does
look as if noise is the major reason for the low correlation between
d and qi.
I am glad to concur.
I tried, using Rick's Demo 1 (Nature of control). It's very jumpy on my
machine, but I got control ratios (Rick calls it "Stability") averaging
6.6 (range 4.7 to 8.3) over 10 runs (1/CR, which you plotted) = 0.152.
The stability factor is 1 - variance(qi)/variance(qi*), where var(qi) is
the value with control present and var(qi*) is the value without it
present. So this is not the same as what you call the control ratio. A
stability factor of 5 would correspond to a 1/CR of about 1/sqrt(6) or 0.408.
This means I could control under those somewhat difficult conditions
rather better than you could control using your local program.
The disturbance in Rick's demo is quite slow, but the delay (using my
machine at least) is about 0.25 seconds, enough to be very distracting for me.
When you get your C program running, I recommend that you do a series of
practice runs until your performance levels out. The data mean little until
learning is complete. You can adjust the timing of the iteration loop by
using a dummy "for" statement: for (t = 1; t < 50000; ++t); for example.
I have been doing as you suggested. I've taken to my home machine the
source code for the sleep study model fitting, and half the data (71
Mbytes so far--and I can't afford much more disk space), and will test
out those data. If the results look like yours, I think we should be
able to develop estimates of the amount of internal noise in the control
those poor people were doing. It would be interesting if we could, and
if the noise level turned out to be related to their sleepiness or to
the drug.
As you know, I am critical of the quality of the sleep data. But how you
use it is up to you.
Given what you said earlier about the human being better fitted by a model
with a leaky integrator, do you think I should modify the model we used
in the sleep fits to change it from a perfect integrator output function
to a leaky one with the leak rate as a fitting parameter?
The leak rate will make a difference only for the very slowest
disturbances, slower than any we used in the sleep study. However, there
would be no harm in adding leak rate as a third parameter to be determined.
To get meaningful measures, however, you have to be very careful about the
uniformity of experimental conditions. The viewing distance should always
be the same; the grip on the mouse and the configuration of the arm should
be constant; the body should be supported in the same position on each run.
Adding a third parameter is what Phil Runkel calls "fine slicing." Your
data have to be uniform enough to support adding a new dimension of
variation to the model. If there are variations in the physical setup,
enough noise can be added to smear out the results of changing three
parameters to the point where the "best fit" range of values is so broad
that it's useless.
I gather from your messages in this thread that
nowadays you do use the leak as a parameter. Should I? How much improvement
do you find that it gives?
I can't answer that quantitatively, but I can say that I never got any
noticeable improvement in the fit of the model to tracking data by
adjusting the leak rate. I tried it just to be able to say I tried it, for
the sake of completeness. Of course I haven't used very slow disturbances
-- when control is too easy, it's hard to tell the actual performance from
ideal performance, which means that the model isn't going to be very
sensitive to small parameter changes, nor will it reveal differences in
parameters between different participants. You may or may not find that
leak rate is a useful parameter.
...don't expect these
results any time soon--I still have to learn how to use CodeWarrior to
write C code on the Mac!
Side note to Rick -- have you looked into that program? I'd be willing to
convert some of my procedures and functions to C if that would help you
write your program in C for the Mac. We have always short-changed people
with Macs, mainly because I don't have one.
Best,
Bill P.