# DS

**URL:** <http://discourse.iapct.org/t/ds/3297>\
**Category:** CSG1998\
**Created:** [March 27, 1998, 6:06pm UTC](http://discourse.iapct.org/t/ds/3297 "1998-03-27T18:06:51Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bill\_Powers1](http://discourse.iapct.org/letter_avatar_proxy/v4/letter/b/8e7dd6/32.png) [@Bill\_Powers1](http://discourse.iapct.org/u/Bill_Powers1)\
**Post date:** [March 27, 1998, 6:06pm UTC](http://discourse.iapct.org/t/ds/3297/1 "1998-03-27T18:06:51Z")

</div>

[From Bill Powers (980327.1053 MST)]

Martin Taylor (various) --

OK, I changed the program to include a printout of sigma(qi)/sigma(d),  
which is available from the correlation program after each run. Here are  
the results from another set of three runs with the disturbance set at very  
fast, medium, and slow:  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Disturbance  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Very Fast Medium Slow

d vs qo -0.134 -0.875 -0.985  
d vs qi 0.516 0.130 0.012  
qi vs qo 0.764 0.363 0.159  
sigma (dist) 121.822 121.137 120.006  
sigma (qi) 78.767 67.298 22.711  
CR 0.647 0.556 0.189

The correlations seem to agree more or less with those from the first set  
of runs. As you can see, the "control ratio", sigma(qi)/sigma(dist), has  
nothing to do with any of the correlations.

So now if you want to rescue your prediction (assuming that my data and/or  
analysis is biased), you're STILL going to have to run the experiment  
yourself.

Best,

Bill P.

---

<div class="post-metadata">

**Author:** ![Martin\_Taylor4](http://discourse.iapct.org/letter_avatar_proxy/v4/letter/m/71c47a/32.png) [@Martin\_Taylor4](http://discourse.iapct.org/u/Martin_Taylor4)\
**Post date:** [March 27, 1998, 10:42pm UTC](http://discourse.iapct.org/t/ds/3297/2 "1998-03-27T22:42:14Z")

</div>

[Martin Taylor 980327 17:15]

> Bill Powers (980327.1053 MST)]
> 
> Martin Taylor (various) --
> 
> OK, I changed the program to include a printout of sigma(qi)/sigma(d),  
> which is available from the correlation program after each run. Here are  
> the results from another set of three runs with the disturbance set at very  
> fast, medium, and slow:  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Disturbance  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Very Fast Medium Slow
> 
> d vs qo -0.134 -0.875 -0.985  
> d vs qi 0.516 0.130 0.012  
> qi vs qo 0.764 0.363 0.159  
> sigma (dist) 121.822 121.137 120.006  
> sigma (qi) 78.767 67.298 22.711  
> CR 0.647 0.556 0.189
> 
> The correlations seem to agree more or less with those from the first set  
> of runs. As you can see, the "control ratio", sigma(qi)/sigma(dist), has  
> nothing to do with any of the correlations.

Excellent! Thank you for doing this. If these data are typical, it does  
look as if noise is the major reason for the low correlation between  
d and qi.

> So now if you want to rescue your prediction (assuming that my data and/or  
> analysis is biased), you're STILL going to have to run the experiment  
> yourself.

I tried, using Rick's Demo 1 (Nature of control). It's very jumpy on my  
machine, but I got control ratios (Rick calls it "Stability") averaging  
6.6 (range 4.7 to 8.3) over 10 runs (1/CR, which you plotted) = 0.152.  
This means I could control under those somewhat difficult conditions  
rather better than you could control using your local program. The  
d vs qi correlation I got averaged .094 (about 62% of 1/CR), but over  
the 10 runs it varied from -.025 to 0.338, so that estimate isn't  
very stable.

I have been doing as you suggested. I've taken to my home machine the  
source code for the sleep study model fitting, and half the data (71  
Mbytes so far--and I can't afford much more disk space), and will test  
out those data. If the results look like yours, I think we should be  
able to develop estimates of the amount of internal noise in the control  
those poor people were doing. It would be interesting if we could, and  
if the noise level turned out to be related to their sleepiness or to  
the drug.

Given what you said earlier about the human being better fitted by a model  
with a leaky integrator, do you think I should modify the model we used  
in the sleep fits to change it from a perfect integrator output function  
to a leaky one with the leak rate as a fitting parameter? You never  
suggested this modification before, when we were looking for ways to  
get better fits, but I gather from your messages in this thread that  
nowadays you do use the leak as a parameter. Should I? How much improvement  
do you find that it gives?

As a side effect of redoing these fits, I ought to be able to determine  
the d vs qi correlation both for the model (with a perfect integrator  
output function) and for the human. Adding noise into the best fit model  
then should be able to reproduce the d vs qi correlation that is found  
in the experiments. Since I know of no obvious way to make a prediction  
of the d vs qi correlation in a loop with transport lag (such as we always  
have when fitting human data), the simulation seems to be the only way  
to do it. It may be, however, that the correlation data are not stable  
enough to allow good fitting.

I'll be out of town Sunday through Friday, and I haven't yet got past  
March 19 in the existing backlog of messages. Also I have a lot of  
preparation to do for the meetings I will be in, so don't expect these  
results any time soon--I still have to learn how to use CodeWarrior to  
write C code on the Mac!

Martin

---

<div class="post-metadata">

**Author:** ![Bill\_Powers1](http://discourse.iapct.org/letter_avatar_proxy/v4/letter/b/8e7dd6/32.png) [@Bill\_Powers1](http://discourse.iapct.org/u/Bill_Powers1)\
**Post date:** [March 28, 1998, 12:57am UTC](http://discourse.iapct.org/t/ds/3297/3 "1998-03-28T00:57:25Z")

</div>

[From Bill Powers (980327.1721 MST)]

Martin Taylor 980327 17:15 --

> Excellent! Thank you for doing this. If these data are typical, it does  
> look as if noise is the major reason for the low correlation between  
> d and qi.

I am glad to concur.

> I tried, using Rick's Demo 1 (Nature of control). It's very jumpy on my  
> machine, but I got control ratios (Rick calls it "Stability") averaging  
> 6.6 (range 4.7 to 8.3) over 10 runs (1/CR, which you plotted) = 0.152.

The stability factor is 1 - variance(qi)/variance(qi\*), where var(qi) is  
the value with control present and var(qi\*) is the value without it  
present. So this is not the same as what you call the control ratio. A  
stability factor of 5 would correspond to a 1/CR of about 1/sqrt(6) or 0.408.

> This means I could control under those somewhat difficult conditions  
> rather better than you could control using your local program.

The disturbance in Rick's demo is quite slow, but the delay (using my  
machine at least) is about 0.25 seconds, enough to be very distracting for me.

When you get your C program running, I recommend that you do a series of  
practice runs until your performance levels out. The data mean little until  
learning is complete. You can adjust the timing of the iteration loop by  
using a dummy "for" statement: for (t = 1; t \< 50000; ++t); for example.

> I have been doing as you suggested. I've taken to my home machine the  
> source code for the sleep study model fitting, and half the data (71  
> Mbytes so far--and I can't afford much more disk space), and will test  
> out those data. If the results look like yours, I think we should be  
> able to develop estimates of the amount of internal noise in the control  
> those poor people were doing. It would be interesting if we could, and  
> if the noise level turned out to be related to their sleepiness or to  
> the drug.

As you know, I am critical of the quality of the sleep data. But how you  
use it is up to you.

> Given what you said earlier about the human being better fitted by a model  
> with a leaky integrator, do you think I should modify the model we used  
> in the sleep fits to change it from a perfect integrator output function  
> to a leaky one with the leak rate as a fitting parameter?

The leak rate will make a difference only for the very slowest  
disturbances, slower than any we used in the sleep study. However, there  
would be no harm in adding leak rate as a third parameter to be determined.  
To get meaningful measures, however, you have to be very careful about the  
uniformity of experimental conditions. The viewing distance should always  
be the same; the grip on the mouse and the configuration of the arm should  
be constant; the body should be supported in the same position on each run.  
Adding a third parameter is what Phil Runkel calls "fine slicing." Your  
data have to be uniform enough to support adding a new dimension of  
variation to the model. If there are variations in the physical setup,  
enough noise can be added to smear out the results of changing three  
parameters to the point where the "best fit" range of values is so broad  
that it's useless.

> I gather from your messages in this thread that  
> nowadays you do use the leak as a parameter. Should I? How much improvement  
> do you find that it gives?

I can't answer that quantitatively, but I can say that I never got any  
noticeable improvement in the fit of the model to tracking data by  
adjusting the leak rate. I tried it just to be able to say I tried it, for  
the sake of completeness. Of course I haven't used very slow disturbances  
-- when control is too easy, it's hard to tell the actual performance from  
ideal performance, which means that the model isn't going to be very  
sensitive to small parameter changes, nor will it reveal differences in  
parameters between different participants. You may or may not find that  
leak rate is a useful parameter.

> ...don't expect these  
> results any time soon--I still have to learn how to use CodeWarrior to  
> write C code on the Mac!

Side note to Rick -- have you looked into that program? I'd be willing to  
convert some of my procedures and functions to C if that would help you  
write your program in C for the Mac. We have always short-changed people  
with Macs, mainly because I don't have one.

Best,

Bill P.

---

<div class="post-metadata">

**Author:** ![Lise\_Marken](http://discourse.iapct.org/letter_avatar_proxy/v4/letter/l/34f0e0/32.png) [@Lise\_Marken](http://discourse.iapct.org/u/Lise_Marken)\
**Post date:** [March 28, 1998, 2:12am UTC](http://discourse.iapct.org/t/ds/3297/4 "1998-03-28T02:12:01Z")

</div>

[From Rick Marken (980327.1800)]

Bill Powers (980327.1721 MST)]

> The stability factor is 1 - variance(qi)/variance(qi\*), where  
> var(qi) is the value with control present and var(qi\*) is the  
> value without it present.

In my demos, the stability factor is sqrt([(var(d)+var(o)]/var(qi))  
which is (by your definitions) sqrt(variance (qi\*)/variance(qi)).  
So the bigger the value of my stability measure, the better the  
control (measured in standard deviations from 1.0, which is  
no control at all).

> Side note to Rick -- have you looked into that program? I'd be  
> willing to convert some of my procedures and functions to C if  
> that would help you write your program in C for the Mac. We have  
> always short-changed people with Macs, mainly because I don't  
> have one.

I get awfully nice results with the Java demos using my 100 MH  
Power PC Mac; stability factors in the 18+ range. I bet most  
Mac users who would care about even trying these demos are at  
least at this level of Mac. People who want to stick with Mac  
will surely start going to the G3 anyway. I've tried my demos  
on the G3 and they're smooth as silk. I don't think it's really  
worth the effort to convert stuff to C on the Mac because  
I don't think there are enough people on primative Macs who  
would be interested in trying the demos. It just doesn't pencil  
out for me. It's a lot of work, there's not much of an audience  
on the kind of Mac that would make it worth doing and very few  
people learn anything from these demos anyway. No, I think I'll  
just continue to develop these fultile demos in a medium I like --  
the web.

Best

Rick

> **···**
>
> --
> 
> Richard S. Marken Phone or Fax: 310 474-0313  
> Life Learning Associates e-mail: rmarken@earthlink.net  
> [http://home.earthlink.net/~rmarken/](http://home.earthlink.net/~rmarken/)

---

<div class="post-metadata">

**Author:** ![Bill\_Powers1](http://discourse.iapct.org/letter_avatar_proxy/v4/letter/b/8e7dd6/32.png) [@Bill\_Powers1](http://discourse.iapct.org/u/Bill_Powers1)\
**Post date:** [March 28, 1998, 12:10pm UTC](http://discourse.iapct.org/t/ds/3297/5 "1998-03-28T12:10:21Z")

</div>

[From Bill Powers (980328.0507 MST)]

Rick Marken (980327.1800)]

> In my demos, the stability factor is sqrt([(var(d)+var(o)]/var(qi))  
> which is (by your definitions) sqrt(variance (qi\*)/variance(qi)).  
> So the bigger the value of my stability measure, the better the  
> control (measured in standard deviations from 1.0, which is  
> no control at all).

Right, sorry about that. I got it upside down.

Best,

Bill P.