Correlations

[From Bill Powers (980319.0632 MST)]

Here is the program (below) for doing a compensatory tracking run at three
levels of difficulty. I'm attaching the source files for Units that are
used, as well as the main program, Track.pas

The program constructs three disturbance tables with slowing factors of 3,
10, and 30 to produce fast, medium, and slow changes in the disturbance. My
results for these disturbances are

                       fast medium slow
  disturbance vs qo: -0.076 -0.892 -0.981
  disturbance vs qi: 0.578 0.181 0.053
  qi vs qo: 0.754 0.282 0.142

I should have done each of these trials many times, but my hand cramps up
so I did just one each. Somebody younger can do the multiple trials.

For the fastest disturbance I was barely able to control at all. The
slowest disturbance case was pretty easy. The filtering on the disturbance
was a three-stage filter with a relatively sharp frequency cutoff.

As predicted, the slower the fluctations in the disturbing variable, the
higher is the disturbance-qo correlation, and the lower are the other two
correlations.

I have also attached the runnable file, track.exe, for those with PCs but
no compilation facilities. The program first computes the three disturbance
tables, scales them, and shows them on the screen, pausing after each one.
A keystroke will end the pause. There are three runs of the tracking task,
one for each disturbance table. A keystroke starts each run, and at the end
of the run the three correlations are displayed. Move the mouse sideways to
keep the white marker aligned with the stationary red one.

The Borland graphics file, EGAVGA.BGI is also attached; it should be in the
directory where the executable program file is.
Best,

Bill P.
unit GrUtils;
{ Graphics Utilities Unit }

interface

uses
  Graph, bgidriv;

var
  GraphDriver, GraphMode, Error: integer;
  hsize, hcenter, vsize, vcenter: integer;

procedure InitGraphics;
procedure Retrace;

implementation

const BGIDIR = '\tp\bgi';

procedure retrace;
begin
case Graphdriver of
  Ega,Vga,Ega64,EgaMono: begin
           while (port[$3da] and 8) = 8 do ;
           while (port[$3da] and 8) = 0 do ;
          end;
  HercMono: begin
           while (port[$3ba] and $80) = 0 do ;
           while (port[$3ba] and $80) = $80 do ;
           end;
  ELSE begin
          while (port[$3da] and 8) = 0 do ;
          while (port[$3da] and 8) = 8 do ;
       end;
end;
end;

procedure Abort(Msg : string);
begin
  Writeln(Msg, ': ', GraphErrorMsg(GraphResult));
  Halt(1);
end;

procedure InitGraphics; {ADAPTS TO HARDWARE}
begin
  { Register all the drivers }
  if RegisterBGIdriver(@CGADriverProc) < 0 then
    Abort('CGA');
  if RegisterBGIdriver(@EGAVGADriverProc) < 0 then
    Abort('EGA/VGA');

  GraphDriver := Detect; { autodetect the hardware }
  InitGraph(GraphDriver, GraphMode, ''); { activate graphics }
  if GraphResult <> grOk then { any errors? }
  begin
    Writeln('Graphics init error: ', GraphErrorMsg(GraphDriver));
    Halt(1);
  end;
  GraphMode := getmaxmode;
  setgraphmode(GraphMode);
  vsize := getmaxy; hsize := getmaxx;
  vcenter := (vsize + 1) div 2;
  hcenter := (hsize + 1) div 2;
end;

begin
end.
unit mouse;

interface
uses dos,crt;

var mousex,mousey: integer;

function initmouse: boolean;
procedure readmouse;
function readbutton: integer;

implementation

var MouseR : registers;
    dx,dy: real;

{ ---------------------- Mouse Functions -----------------------------------}
function initmouse: boolean;
begin { false if mouse not found }
mousex := 0; mousey := 0;
dx := 0; dy := 0;
MouseR.ax := 0;
intr ($33, mouser);
if Mouser.ax <> $ffff then
  begin
   writeln('MOUSE NOT INSTALLED');
   delay(1000);
  end;
Initmouse := (MouseR.ax = $ffff);
end;

procedure readmouse;
begin
mouser.ax := 11;
intr ($33, mouser);
dx := dx + 0.7*(integer(MouseR.cx) - dx);
dy := dy + 0.7*(integer(MouseR.dx) - dy);
mousex := mousex + round(dx);
mousey := mousey - round(dy);
end;

function readbutton: integer; { returns 1,2,or 4}
begin
MouseR.ax := 3;
intr ($33, MouseR);
readbutton := MouseR.bx and 3;
end;

end. { of unit }

{$N+}
unit stats;
interface

var sx,sy,sx2,sy2,sxy,xbar,ybar,sigx,sigy,corr,regression,intercept: real;

procedure correl(ty: char; x,y: pointer ; datasize: integer);

implementation

type dataarraytype = array[0..2047] of integer;
     rdataarraytype = array[0..2047] of real;
     dataptrtype = ^dataarraytype;
     rdataptrtype = ^rdataarraytype;

procedure correl;
var n,u,v,w,z: real;
    i: integer;
    dxptr,dyptr: dataptrtype;
    rdxptr,rdyptr: rdataptrtype;
begin
if ty = 'i' then
begin
  dxptr := x; dyptr := y;
end
else
begin
  rdxptr := x; rdyptr := y;
end;
sx := 0.0; sy := 0.0;
n := datasize;
for i := 0 to datasize - 1 do
  begin
   if ty = 'i' then
   begin
    u := dxptr^[i]; v := dyptr^[i];
   end
   else
   begin
    u := rdxptr^[i]; v := rdyptr^[i];
   end;
   sx := sx + u; sy := sy + v;
  end;
xbar := sx/n; ybar := sy/n;
sx2 := 0.0; sy2 := 0.0; sxy := 0.0;
for i := 0 to datasize - 1 do
  begin
   if ty = 'i' then
   begin
    u := dxptr^[i]; v := dyptr^[i];
   end
   else
   begin
    u := rdxptr^[i]; v := rdyptr^[i];
   end;
   sx2 := sx2 + (u - xbar)*(u - xbar);
   sy2 := sy2 + (v - ybar)*(v - ybar);
   sxy := sxy + (u - xbar)*(v - ybar);
  end;
sigx :=sqrt(sx2/n);
sigy :=sqrt(sy2/n);
if (abs(sigx*sigy) > 0.0001) then
  z := sxy/(n * sigx * sigy)
else z := 0.0;
corr := z;
if abs(sigx) > 0.001 then regression := z * sigy/sigx
else regression := 9999.99;
intercept := ybar - regression*xbar;
end;

end.program track;
uses dos,crt,graph,grutils,mouse,stats;

var dist: array[0..2] of array[0..1799] of integer;
    qi,qo: array[1..1800] of integer;
    slow: real;
    d: integer;
    ch: char;

procedure makedist;

var i,j: integer;
    maxd: real;
    d: array[0..2] of real;
    ch: char;
begin
for j := 0 to 2 do
  begin
   d[0] := 0.0;
   d[1] := 0.0;
   d[2] := 0.0;
   case j of
    0: slow := 3.0;
    1: slow := 10.0;
    2: slow := 30.0;
   end;
   for i := 0 to 1799 do
    begin
     d[0] := d[0] + (20000.0*(random - 0.5) - d[0])/slow;
     d[1] := d[1] + (d[0] - d[1])/slow;
     d[2] := d[2] + (d[1] - d[2])/slow;
     dist[j,i] := round(d[2]);
    end;
   maxd := 0.0;
   for i := 0 to 1799 do
    if abs(dist[j,i]) > maxd then maxd := abs(dist[j,i]);
   for i := 0 to 1799 do
   begin
    dist[j,i] := round(dist[j,i]/maxd*300.0);
    putpixel(i div 6,vcenter - dist[j,i] div 2,white);
   end;
   ch := readkey;
   clearviewport;
  end;
end;

procedure showqi(i: integer);

const oldx: integer = 0;
begin
if i <> -200 then
line(hcenter + oldx,vcenter,hcenter + oldx,vcenter - 10)
  else
  begin
   clearviewport;
   setcolor(lightred);
   line(hcenter,vcenter + 11,hcenter,vcenter + 21);
   setcolor(white);
  end;
oldx := qi[abs(i)];
line(hcenter + oldx,vcenter,hcenter + oldx,vcenter - 10);
end;

procedure showcorr(j: integer);
var numstr: string;
begin
correl('i',@dist[j,0],@qo,1800);
str(corr:6:3,numstr);
outtextxy(0,0,'Correlation d vs qo = ' + numstr);
correl('i',@dist[j,0],@qi,1800);
str(corr:6:3,numstr);
outtextxy(0,15,'Correlation d vs qi = ' + numstr);
correl('i',@qi,@qo,1800);
str(corr:6:3,numstr);
outtextxy(0,30,'Correlation qi vs qo = ' + numstr);
end;

procedure doexp;
var i,j,k: integer;
begin
for j := 0 to 2 do
begin
  clearviewport;
  mousex := 0;
  for k := -200 to 1799 do
  begin
   i := abs(k);
   readmouse;
   qi[i] := dist[j,i] + mousex;
   qo[i] := mousex;
   showqi(k);
   delay(33);
  end;
  showcorr(j);
  ch := readkey;
end;
end;

begin
initgraphics;
setwritemode(XORput);
makedist;
doexp;
end.

Track.exe (58 Bytes)

Egavga.bgi (59 Bytes)

[Martin Taylor 980324 10:00

I'm more than a week behind in reading CSGnet postings, but in scanning
the subject lines of the messages I hadn't read, I came across this.

Bill Powers (980319.0632 MST)

Here is the program (below) for doing a compensatory tracking run at three
levels of difficulty. I'm attaching the source files for Units that are
used, as well as the main program, Track.pas

The program constructs three disturbance tables with slowing factors of 3,
10, and 30 to produce fast, medium, and slow changes in the disturbance. My
results for these disturbances are

                      fast medium slow
disturbance vs qo: -0.076 -0.892 -0.981
disturbance vs qi: 0.578 0.181 0.053
qi vs qo: 0.754 0.282 0.142

I should have done each of these trials many times, but my hand cramps up
so I did just one each. Somebody younger can do the multiple trials.

For the fastest disturbance I was barely able to control at all. The
slowest disturbance case was pretty easy. The filtering on the disturbance
was a three-stage filter with a relatively sharp frequency cutoff.

As predicted, the slower the fluctations in the disturbing variable, the
higher is the disturbance-qo correlation, and the lower are the other two
correlations.

Why do you say "as predicted?" We've known this for years. It's what
control loops with an integrating output function do, and that's the
kind of model we usually fit to human tracking data.

I'm not sure why you posted this, unless it was intended to contribute
toward determining the relative contributions of noise and of the
integrator output function to the correlation. If that was the intent,
two things are missing. The most critical is the control ratio. If the
correlation between disturbance and qi is substantially higher than
1/CR, then either my analysis is wrong or the model we usually use in
fitting such data is wrong.

If the correlation is substantially lower than 1/CR, then one may suspect
a large noise contribution. But we wouldn't be able to determine how
large until we model it.

There are two models that need to be tested. One has a pure integrator
as its output function, the other, as you suggested, a leaky integrator.
So far as I remember, we haven't tried fitting the leak rate as a parameter
in modelling human data--we've used integration rate (gain) and loop
transport delay, mainly. It would be interesting to see whether the
model fits better with a well characterized leak rate on the output
integrator.

In each model, you should insert noise, which in the simplest case would
be wide-band white noise independent of the disturbance bandwidth, and
might be injected into the perceptual signal. If the simple assumption
works, the best fit model parameters, including noise level, should be
the same for all your conditions. If we can use this technique to get
a handle on the internal noise in the loop, it would be a nice contribution
to our understanding. (But we have to remember that there is a contribution
from quantization noise in the environmental feedback function. With luck,
that's small compared to the additive component you would model).

But the first thing to do is to add a column for 1/CR to your posted data.

Martin

[Martin Taylor 980324 10:00

I'm more than a week behind in reading CSGnet postings, but in scanning
the subject lines of the messages I hadn't read, I came across this.

Bill Powers (980319.0632 MST)

Here is the program (below) for doing a compensatory tracking run at three
levels of difficulty.

[...]

As predicted, the slower the fluctations in the disturbing variable, the
higher is the disturbance-qo correlation, and the lower are the other two
correlations.

Why do you say "as predicted?" We've known this for years. It's what
control loops with an integrating output function do, and that's the
kind of model we usually fit to human tracking data.

I'm not sure why you posted this, unless it was intended to contribute
toward determining the relative contributions of noise and of the
integrator output function to the correlation.

Look at the thread "Control and correlation" especially Bill Powers
(980317.0754 MST):

That is an extremely interesting fact, and I'd love to see the data,
to see how the change in correlation with frequency (or upper cut-off?)
of the disturbance signal relates to the change in slope with frequency
of the leaky integrator. That would tell us quite a lot, I think. And
if you also have data that tie this into loop delay, it would be really
fascinating. Please post at least some of these data. I don't know what
you have, but presumably it involves simulations using disturbance signals
of varying low-pass bandwidths, though it would be a better test if it
involved disturbances of constant bandwidth but a varying centre frequency.
Perhaps you have it with sinusoidal disturbances at different frequencies
compared to the leaky integrator cutoff frequency?

I'll have to construct a simple tracking experiment to show this -- old
data has a tendency to disappear as I houseclean my disk. I'm sure you have
many examples of this from the sleep study. It's not a tiny effect; it's
obvious and robust.

I'm pretty sure this is Bill's followup to that. When you dropped from
sight for a while I kind of expected this might get lost in the
characteristic huff and hustle of tuning back in, so it stuck in the back
of my mind.

  Bruce Nevin

···

At 10:16 AM 3/24/98 -0500, Martin Taylor wrote:

[From Bill Powers (980324.1156 MST)]

Martin Taylor 980324 10:00--

Here is the program (below) for doing a compensatory tracking run at three
levels of difficulty. I'm attaching the source files for Units that are
used, as well as the main program, Track.pas

Why do you say "as predicted?" We've known this for years. It's what
control loops with an integrating output function do, and that's the
kind of model we usually fit to human tracking data.

I'm not sure why you posted this, unless it was intended to contribute
toward determining the relative contributions of noise and of the
integrator output function to the correlation. If that was the intent,
two things are missing. The most critical is the control ratio. If the
correlation between disturbance and qi is substantially higher than
1/CR, then either my analysis is wrong or the model we usually use in
fitting such data is wrong.

Which "disturbance" are you talking about? DS or d?

The data I posted were for the real performance, not the model. I did not
fit a model to these data. If you want to do so, I have sent you the source
code, which you are free to modify as desired. I need to concentrate on the
model I'm doing for the European CSG conference.

Since most of your comments are about the model performance, I will drop
this here. One thing: there is no integrator between the disturbance and
qi. That contribution is purely proportional, regardless of the model. So
there is no way to explain the low correlation by using your 1/CR and your
postulates about integrators. That wouldn't apply to the connection between
d and qi, which I repeat is purely proportional.

Also, is your CR guaranteed to be >= 1.0? If not, the correlations you
predict could be greater than 1.

Best,

Bill P.

[Martin Taylor 980325 09:45]

Bill Powers (980324.1156 MST)]

Martin Taylor 980324 10:00--

Here is the program (below) for doing a compensatory tracking run at three
levels of difficulty. I'm attaching the source files for Units that are
used, as well as the main program, Track.pas

Why do you say "as predicted?" We've known this for years. It's what
control loops with an integrating output function do, and that's the
kind of model we usually fit to human tracking data.

I'm not sure why you posted this, unless it was intended to contribute
toward determining the relative contributions of noise and of the
integrator output function to the correlation. If that was the intent,
two things are missing. The most critical is the control ratio. If the
correlation between disturbance and qi is substantially higher than
1/CR, then either my analysis is wrong or the model we usually use in
fitting such data is wrong.

Which "disturbance" are you talking about? DS or d?

DS, of course. OF COURSE!!! (AT least, if that's what you meant by using
the word "disturbance" in your tabulation of results, as I assumed you did).

The analysis uses the following assumptions (which you would know if you
had bothered to read the message in which it was presented): (1) No transport
delay around the loop; (2) qi = qo + d; (3) p = qi; (4) e = p + r;
(5) qo = integral (e).

The data I posted were for the real performance, not the model. I did not
fit a model to these data.

Fine. We know that low frequency disturbances lead to better control in
control loops with pure integrator output functions. The analysis shows
that for control loops with pure integrator output functions, the
correlation between disturbance and qi in the absence of noise is equal to
1/CR. So the analysis predicts that the correlation between disturbance
and qi becomes very low for very low frequency disturbances. That's what
you showed happened. So why bother showing it without the values for 1/CR?

Your data conform _qualitatively_ to my analysis. I asked you to add the
datum that would enable us to see whether they conform _quantitatively_,
data you are easily able to provide, data that would allow us at least
to estimate whether the noise contribution is important enough to be
susceptible to accurate modelling. You could do this, but you choose
not to, even in your response to my request that you show the data.

You said that you had evidence that the low value of low-frequency
correlation was due to noise. You said that the "correct" reason for the
low correlaion at low frequencies was noise. You said that this was so
because the output function at low frequencies did _not_ behave like
a pure integrator, but was leaky. A control system with a leaky
integrator would act at low enough frequencies like a proportional
system, for which the noise-free control system would (intuitively) show
increasing correlation between d and qi.

So far, in our modelling, we have used a model loop in which the output
is a pure integrator, and the claim is often made that such models fit the
data with such accuracy that no improvement is needed. We have not modelled
human data using the cut-off frequency of the leaky integrator as a
parameter. With a model that provides a good value for the cut-off
frequency parameter, we might be able to estimate the noise contribution
to the low correlation. Even assuming a pure integrator output function,
as we usually do, the deviation of the correlation below 1/CR might
give us an estimate of the relative contribution of noise.

Just as a precautionary measure, here, I should re-emphasize that I
personally believe that noise is a major contributor to the decorrelation.
I said this before, during, and after completing my analysis. I believe
this, but as yet have no evidence to support my belief. That's why I
find frustrating your unwillingness to post data that I know you to have.
By how much does the noise-free analysis miss the mark?

And, as I said before, without corroboration from the simulations at which
you are so adept and have done so many times, I am not 100% convinced in
my mind that the analysis is correctly done. It seems far too simple not
to be a well known result, if it is correct.

If you want to do so, I have sent you the source
code, which you are free to modify as desired. I need to concentrate on the
model I'm doing for the European CSG conference.

That's OK. Being swamped myself with other things, (and backed up to
March 15 with unread messages) I well appreciate that. Your source is
for a machine I don't have, which would mean putting unavailable time
into rewriting it and adding the required modifications.

Since most of your comments are about the model performance, I will drop
this here. One thing: there is no integrator between the disturbance and
qi. That contribution is purely proportional, regardless of the model.

Huh! Have you read ANYTHING about the analysis that so offends you? The
integrator is between the error signal and qo!!!

So
there is no way to explain the low correlation by using your 1/CR and your
postulates about integrators. That wouldn't apply to the connection between
d and qi, which I repeat is purely proportional.

So it is. The analysis postulates qi = qo + d. Very proportional indeed.

Also, is your CR guaranteed to be >= 1.0? If not, the correlations you
predict could be greater than 1.

The analysis is valid _only_ for a loop with a pure integrator output
function and no transport lag. In such a loop, CR is indeed guaranteed
to be greater than 1.0. When you insert transport lags and high gains,
CR can be <1.0, but then the analysis does not apply directly. I discussed
this point, and showed some of the qualitative differences to be expected,
in the message in which the analysis was presented. If you want to see
what I said, try reading the message.

Martin

[Martin Taylor 980325 10:15]

Bruce Nevin (Apparently 980324 11:39)

[Martin Taylor 980324 10:00

As predicted, the slower the fluctations in the disturbing variable, the
higher is the disturbance-qo correlation, and the lower are the other two
correlations.

Why do you say "as predicted?" We've known this for years. It's what
control loops with an integrating output function do, and that's the
kind of model we usually fit to human tracking data.

I'm not sure why you posted this, unless it was intended to contribute
toward determining the relative contributions of noise and of the
integrator output function to the correlation.

Look at the thread "Control and correlation" especially Bill Powers
(980317.0754 MST):

I'll have to construct a simple tracking experiment to show this -- old
data has a tendency to disappear as I houseclean my disk. I'm sure you have
many examples of this from the sleep study. It's not a tiny effect; it's
obvious and robust.

I'm pretty sure this is Bill's followup to that. When you dropped from
sight for a while I kind of expected this might get lost in the
characteristic huff and hustle of tuning back in, so it stuck in the back
of my mind.

My question was not why Bill did a simple tracking experiment, but why he
posted results that do not distinguish between the proposed alternative
mechanisms for the reduced correlation with low-frequency disturbances,
when he must have collected the data that would have allowed at least
a first-pass approach to estimating the contributions from the two
mechanisms.

Martin

···

At 10:16 AM 3/24/98 -0500, Martin Taylor wrote:

[From Bill Powers (980325.0857 MST)]

Martin Taylor 980325 09:45--

Which "disturbance" are you talking about? DS or d?

DS, of course. OF COURSE!!! (AT least, if that's what you meant by using
the word "disturbance" in your tabulation of results, as I assumed you did).

I have told you at least half a dozen times in the last couple of weeks
that when I say "d" or "disturbance" in technical writings, I mean the
disturbing variable and not DS. That is what I always mean and will always
mean. In the program and in the data I posted, I meant "d", whether or not
it is numerically equal to DS. The variable d is a physical quantity that
can be set or varied independently of the control system's output or input
quantities. I will never mean anything else by d.

DS, on the other hand, cannot be varied independently of qi and qo. It is
determined by qi and qo; in fact, DS = qi - Fe(qo).

The data I posted were for the real performance, not the model. I did not
fit a model to these data.

Fine. We know that low frequency disturbances lead to better control in
control loops with pure integrator output functions. The analysis shows
that for control loops with pure integrator output functions, so the

analysis predicts that the correlation between disturbance

and qi becomes very low for very low frequency disturbances. That's what
you showed happened. So why bother showing it without the values for 1/CR?

Because there is no integrator between d and qi. It is not true that " The
analysis shows that for control loops with pure integrator output
functions, the correlation between disturbance and qi in the absence of
noise is equal to 1/CR." Your analysis applies only if there is an
integrator between d and qi, which is not the case in this experiment. The
only integrator in a model of this behavior lies between qi and qo -- and
that integrator is in a closed loop, which your analysis does not take into
account. In this experiment, the connection between d and qi is _strictly
proportional_, so your analysis says nothing about it.

Your data conform _qualitatively_ to my analysis.

No they do not; you are confusing the correlation between qi and qo with
the correlation between d and qi. These correlations are different, and for
the d:qi correlation, the connection is proportional at all frequencies of
interest. Your analysis and reasoning apply (assuming they do apply) only
to the correlation between qi and qo.

I asked you to add the
datum that would enable us to see whether they conform _quantitatively_,
data you are easily able to provide, data that would allow us at least
to estimate whether the noise contribution is important enough to be
susceptible to accurate modelling. You could do this, but you choose
not to, even in your response to my request that you show the data.

I did not match a model to the data. That is an extra step, involving
running a model over and over while adjusting its parameters for best fit
to the data. If that is easy for me to do, it is just as easy for you do
to. I gave you a program that will generate data using a human subject. You
are too used to dreaming up ideas that other people have to try to carry
out. I have no contract with you. If this prediction relating to CR is
important to you, it is important enough for you to put in some work
investigating it. Why should I labor to make your points for you?

You said that you had evidence that the low value of low-frequency
correlation was due to noise. You said that the "correct" reason for the
low correlaion at low frequencies was noise. You said that this was so
because the output function at low frequencies did _not_ behave like
a pure integrator, but was leaky.

Please be more precise. I said that this was so _for the correlation
between qi and qo_ because the output function tended toward proportional
at low frequencies because of the leakiness. I did not say that this was
the explanation for the low correlation between d and qi; that argument is
unnecessary for that correlation, because the effect of d on qi is
proportional from the start, at all frequencies of any interest.

A control system with a leaky
integrator would act at low enough frequencies like a proportional
system, for which the noise-free control system would (intuitively) show
increasing correlation between d and qi.

No, a delayless noise-free proportional system with a constant reference
signal would show the same high correlation between d and qi, and between
qi and qo, at all frequencies. The solutions of the equations show that
both qo and qi are exactly proportional to the disturbance. The
correlations would all be 1 or -1, _exactly_.

So far, in our modelling, we have used a model loop in which the output
is a pure integrator, and the claim is often made that such models fit the
data with such accuracy that no improvement is needed. We have not modelled
human data using the cut-off frequency of the leaky integrator as a
parameter.

We certainly have. Varying the integrating constant and the slowing factor
is equivalent to varying the cutoff frequency. Look at the last demo in
Demo 2: the integration factor and leakage (as well as the delay) are
adjustable as part of achieving the best fit of the model to the data.

With a model that provides a good value for the cut-off
frequency parameter, we might be able to estimate the noise contribution
to the low correlation. Even assuming a pure integrator output function,
as we usually do, the deviation of the correlation below 1/CR might
give us an estimate of the relative contribution of noise.

So work it out, Martin. All this armchair intuitive guessing loses its
appeal very rapidly.

Just as a precautionary measure, here, I should re-emphasize that I
personally believe that noise is a major contributor to the decorrelation.
I said this before, during, and after completing my analysis. I believe
this, but as yet have no evidence to support my belief. That's why I
find frustrating your unwillingness to post data that I know you to have.
By how much does the noise-free analysis miss the mark?

You do not know what data I have. I could say that you have the data you
need, too, from your "sleep" studies. I took the time to write a program
from which you can generate all the data you want. I know you know how to
program. What's keeping you from matching a model to the data and doing
your own analysis?

And, as I said before, without corroboration from the simulations at which
you are so adept and have done so many times, I am not 100% convinced in
my mind that the analysis is correctly done. It seems far too simple not
to be a well known result, if it is correct.

Pooey. Don't give me that motivational flattery crap. It's no easier for me
to do what you want than it would be for you. Of course it's a LOT easier
for you if I do the work, but that is hardly a motivation for me.

If you want to do so, I have sent you the source
code, which you are free to modify as desired. I need to concentrate on the
model I'm doing for the European CSG conference.

That's OK. Being swamped myself with other things, (and backed up to
March 15 with unread messages) I well appreciate that. Your source is
for a machine I don't have, which would mean putting unavailable time
into rewriting it and adding the required modifications.

OK, so now we've established our priorities. I'll just assume that when
your postulates about CR attain a greater priority than your other
interests, you will do something to settle the issue -- and in the interim,
avoid speaking as if the questions are settled.

Best,

Bill P.

[Martin Taylor 9870326 23:42]

Bill Powers (980325.0857 MST)]

The data I posted were for the real performance, not the model. I did not
fit a model to these data.

Fine. We know that low frequency disturbances lead to better control in
control loops with pure integrator output functions. The analysis shows
that for control loops with pure integrator output functions, so the

analysis predicts that the correlation between disturbance

and qi becomes very low for very low frequency disturbances. That's what
you showed happened. So why bother showing it without the values for 1/CR?

Because there is no integrator between d and qi. It is not true that " The
analysis shows that for control loops with pure integrator output
functions, the correlation between disturbance and qi in the absence of
noise is equal to 1/CR."

Well, if it isn't true, show where the analysis is incorrect, rather
than simply asserting that it isn't true. Verbal mouth-waving won't cut it.
I repeat what I wrote at the start of the message to which yours is a
response (Martin Taylor 980325 09:45):

+The analysis uses the following assumptions (which you would know if you
+had bothered to read the message in which it was presented): (1) No transport
+delay around the loop; (2) qi = qo + d; (3) p = qi; (4) e = p + r;
+(5) qo = integral (e).

At the end of your message, you instruct me to "avoid speaking as if the
questions are settled." A little introspection might go a long way here,
I suggest. A little matter of pots, kettles, and colour?

Your analysis applies only if there is an
integrator between d and qi, which is not the case in this experiment.

The analysis applies only if there is an integrator between e and qo,
which is the case in the loop analyzed. Between d and qi there is a
summator whose other input is qo. No integrator!!!!!!!

The
only integrator in a model of this behavior lies between qi and qo -- and
that integrator is in a closed loop, which your analysis does not take into
account.

The integrator is between e and qo.

My analysis is _only_ of the closed loop. In what way does analyzing the
closed loop not take account of the closed loop?

Question: Why do you write all this nonsense about the analysis? It's
really puzzling.

In this experiment, the connection between d and qi is _strictly
proportional_, so your analysis says nothing about it.

In the analysis, qi = d + qo. Does that count as "strictly proportional?"
The analysis is only about the correlation between d and qi.

Your data conform _qualitatively_ to my analysis.

No they do not; you are confusing the correlation between qi and qo with
the correlation between d and qi.

I haven't analysed the correlation between qi and qo. I analyzed the
correlation between d and qi.

These correlations are different, and for
the d:qi correlation, the connection is proportional at all frequencies of
interest. Your analysis and reasoning apply (assuming they do apply) only
to the correlation between qi and qo.

No. That correlation has not been examined.

I asked you to add the
datum that would enable us to see whether they conform _quantitatively_,
data you are easily able to provide, data that would allow us at least
to estimate whether the noise contribution is important enough to be
susceptible to accurate modelling. You could do this, but you choose
not to, even in your response to my request that you show the data.

I did not match a model to the data.

But you did measure your control ratios, I assume. I think you had to, as
part of determining the correlations--though I guess there are ways to
avoid doing so if you really want to. Those control ratio numbers are the
data I ask, for the umpty-Nth time, that you add to your tabulation.
Why the strong resistance to doing so?

If this prediction relating to CR is
important to you, it is important enough for you to put in some work
investigating it. Why should I labor to make your points for you?

Because you already collected the data, and published part of it in a
way that was useless for comparing the two mechanisms that you seemed to
want to compare. It's trivial for you to add those numbers, given that
you have already gone to the trouble to post the correlations that
conform to the predictions of the analysis.

···

-----------------------

You said that you had evidence that the low value of low-frequency
correlation was due to noise. You said that the "correct" reason for the
low correlaion at low frequencies was noise. You said that this was so
because the output function at low frequencies did _not_ behave like
a pure integrator, but was leaky.

Please be more precise. I said that this was so _for the correlation
between qi and qo_ because the output function tended toward proportional
at low frequencies because of the leakiness. I did not say that this was
the explanation for the low correlation between d and qi; that argument is
unnecessary for that correlation, because the effect of d on qi is
proportional from the start, at all frequencies of any interest.

Interesting. Again you assert that the analysis is demonstrably wrong, but
you apparently base your assertion on intuition rather than on an actual
demonstration--or a reanalysis or a simulation.

I would be gratified with such a demonstration. The analysis is available to
you. I can find no error in it. It could be shown to be wrong either by
pointing out a mathematical error, or by a simulation of the case
analyzed, if the simulation came up with a result different from what the
analysis predicts.

Meanwhile, the next best thing is to look at the data you have, and see
whether the result is even close.

A control system with a leaky
integrator would act at low enough frequencies like a proportional
system, for which the noise-free control system would (intuitively) show
increasing correlation between d and qi.

No, a delayless noise-free proportional system with a constant reference
signal would show the same high correlation between d and qi, and between
qi and qo, at all frequencies. The solutions of the equations show that
both qo and qi are exactly proportional to the disturbance. The
correlations would all be 1 or -1, _exactly_.

That is not the system in question. I was paraphrasing what you said:

+(Bill Powers 980315.0339 MST)
+ The modeled relation
+between qi and qo does involve a leaky integration. However, the
+decorrelation is the greatest at the lowest frequencies, where the phase
+shift due to the integration is the least and the output function behaves
+most nearly like a proportional function.

So far, in our modelling, we have used a model loop in which the output
is a pure integrator, and the claim is often made that such models fit the
data with such accuracy that no improvement is needed. We have not modelled
human data using the cut-off frequency of the leaky integrator as a
parameter.

We certainly have. Varying the integrating constant and the slowing factor
is equivalent to varying the cutoff frequency.

I'm sorry, but I must be very dense. I don't know how to determine the
frequency at which the output function turns over from being flat to
a 6 dB per octave slope, given the slowing factor and the integrating
constant. And I don't remember learning how the concept of "slowing
factor" applies to a continuous analogue system. It's a way of avoiding
digital artifacts, not a property of the analogue system being modelled.
--------------------

You do not know what data I have.

I know that when we were trying to work through the "information about
the disturbance" question a few years ago, you presented a whole lot of
data of the kind I asked about--correlations between d and qi.

I could say that you have the data you
need, too, from your "sleep" studies.

True, but they haven't been analysed in the way I know you had analysed
your data. The sleep data could be used, and I have contemplated doing it.
But I
thought you would be more convinced by analyses you did on your own data,
if the results came out as the analysis predicts, and you wouldn't be
shy about letting everyone know if they didn't.

---------------

...
avoid speaking as if the questions are settled.

I have avoided it so far, as I presume you are aware from reading my
messages, which have been about how to test the question.

However, I have not noted such avoidance on your part. I read statements
such as

It is not true that " The
analysis shows that for control loops with pure integrator output
functions, the correlation between disturbance and qi in the absence of
noise is equal to 1/CR."

That sounds a little bit as if a question is settled, doesn't it? Moreover,
the settled question isn't even about how the control system works, but
about what my analysis shows. Apparently you have quite settled the question,
and the analysis does not show what it purports to show. Interesting,
that:-)

Reciprocity might be nice, in avoiding talking as if questions were settled.
They aren't readily settled by assertion, especially when the assertion
contradicts a mathematical analysis without showing any error in the
analysis.
------------------

I'll just assume that when
your postulates about CR attain a greater priority than your other
interests, you will do something to settle the issue.

When other people are relying on me to do things for them (and in some cases
actually paying me to do it) I put that at a higher priority than my own
theoretical interests, yes. And it is why I am still only up to March 18
in the backlog of CGSnet messages (other than recent messages in this
thread), despite spending nearly three hours on the mail this evening.

Martin

[From Bill Powers (980326.0014 MST)]

Martin Taylor 9870326 23:42--

The data I posted were for the real performance, not the model. I did not
fit a model to these data.

Fine. We know that low frequency disturbances lead to better control in
control loops with pure integrator output functions. The analysis shows
that for control loops with pure integrator output functions, so the

analysis predicts that the correlation between disturbance

and qi becomes very low for very low frequency disturbances. That's what
you showed happened. So why bother showing it without the values for 1/CR?

You ask me to show what is wrong with your analysis.

You define CR (980312 or thereabouts) this way:

The control ratio (CR) is the ratio between the fluctuations that would
occur in p in the absence of control and the fluctuations in p when the
perception is controlled. In other words, CR = d/p when the transform
between qi and p is the unit transform.

In the absence of control, the only influence on qi is d, so qi(no control)
= d. Also by your definition, p in the denominator is equal to qi(control).
Thus your definition of the control ratio reduces to

CR = d/p = qi(no control)/qi(control), or in terms of observables,

CR = d/qi(control).

Since qi can be zero when d is nonzero, the calculation of CR can involve
division by zero, as you have written the equation for CR. The sum of d/qi
is indeterminate.

Obviously, you do not mean either "d" or "qi" in the equation you give for
CR. You mean something like RMS fluctuations in the values of these
variables, which would involve a very different mathematical expression.

To translate your equation into something that makes mathematical sense, we
would have to substitute (for example) the RMS value of d for d, and the
RMS value of qi for qi (in order to translate the loose term
"fluctuations"). But these values are not uniquely related to the waveform
or frequency spectrum of d. Therefore CR, if calculated in terms of RMS
values or sigmas, is not related to the correlation between d and qi (or
anything else).

Your fast and loose derivation is simply not rigorous. It isn't even
mathematical. I'm sure you know how to construct a mathematical proof, but
you haven't done it here.

As Richard Kennaway showed, our conjecture about the correlation between a
variable and its integral being zero is correct only for the special case
of periodic waveforms with a zero integral. Since the disturbance waveform
we use is nonperiodic and nonrepeating, your analysis in terms of Fourier
components does not apply, meaning that your general conclusion is false.
This invalidates the remainder of your argument.

Need we go on with this?

Best,

Bill P.

[Martin Taylor 980326 10:00]

Bill Powers (980326.0014 MST)

The control ratio (CR) is the ratio between the fluctuations that would
occur in p in the absence of control and the fluctuations in p when the
perception is controlled. In other words, CR = d/p when the transform
between qi and p is the unit transform.

In the absence of control, the only influence on qi is d, so qi(no control)
= d. Also by your definition, p in the denominator is equal to qi(control).
Thus your definition of the control ratio reduces to

CR = d/p = qi(no control)/qi(control), or in terms of observables,

CR = d/qi(control).

Since qi can be zero when d is nonzero, the calculation of CR can involve
division by zero, as you have written the equation for CR. The sum of d/qi
is indeterminate.

Obviously, you do not mean either "d" or "qi" in the equation you give for
CR. You mean something like RMS fluctuations in the values of these
variables, which would involve a very different mathematical expression.

Yes. That's what I said verbally, but did not write correctly. I intended
the form to be the same as you usually use for control ratio
CR = RMS(d)/RMS(qi(control))

As Richard Kennaway showed, our conjecture about the correlation between a
variable and its integral being zero is correct only for the special case
of periodic waveforms with a zero integral.

Not having reached Kennaway's derivation in my attempt to read the backlog
of messages, I was unaware of what he said. Having now read it, I see that
he says what I said--that the DC component is the only component that
leads to a non-zero correlation. He merely went a different way to get
the same answer. We both talk about Fourier-decomposable waveforms, which
in fact refers to any waveform achievable by physical systems.

Since the disturbance waveform
we use is nonperiodic and nonrepeating, your analysis in terms of Fourier
components does not apply, meaning that your general conclusion is false.

Perhaps. I only claimed it to be true for physically achievable signals.
I do believe you used such signals in your experiments.

Need we go on with this?

No longer than it takes to get you to post the data needed to test the
accuracy of the analysis. Why not do it?

I'm interested in the way that anything stated by Richard Kennaway is
necessarily true, while anything stated by me is necessarily false.
Especially when we come to the same conclusion!

Martin

[From Bill Powers (980327.0552 MST)]

Martin Taylor 980326 10:00--

Yes. That's what I said verbally, but did not write correctly. I intended
the form to be the same as you usually use for control ratio
CR = RMS(d)/RMS(qi(control))

I never use the term "control ratio." Neither do I use the ratio of random
fluctuations with and without control to determine loop gain.

Since the disturbance waveform
we use is nonperiodic and nonrepeating, your analysis in terms of Fourier
components does not apply, meaning that your general conclusion is false.

Perhaps. I only claimed it to be true for physically achievable signals.
I do believe you used such signals in your experiments.

The disturbance waveform I used in the experiment is nonperiodic and
nonrepeating. It is derived by smoothing the output of a random number
generator which could run (I believe) for several hundred hours as we are
using it without repeating-- it is therefore physically achievable. A very
long time ago, before I knew about random number generators, I used an
audio signal generated by an FM receiver tuned between stations to generate
random disturbance waveforms. I presume that this signal, to, was
nonperiodic and nonrepeating, and physically achievable.

Need we go on with this?

No longer than it takes to get you to post the data needed to test the
accuracy of the analysis. Why not do it?

Because I don't choose to.

I'm interested in the way that anything stated by Richard Kennaway is
necessarily true, while anything stated by me is necessarily false.
Especially when we come to the same conclusion!

Richard knows how to construct a mathematical proof that doesn't contain
errors or require vague verbalisms to arrive at a conclusion; his track
record even in the short time he has been on the net is considerably better
than yours. Your mathematical derivations are, to say the least, careless.
I will leave it to Richard to say whether he has reached the same
conclusions as you.

Best,

Bill P.

[From Bill Powers (970418.0500 MST)]

Bruce Abbott (970417.2200 EST)--

This confuses the accuracy of predicting an observed Y-value given
knowledge of its X-value with the accuracy of pinning down the slope and
intercept of the line used to generate the prediction.

I thought the whole point was to predict Y given X. The basis hypothesis is
"These data represent a linear relationship between X and Y." The
least-squares fit then gives you the slope and intercept of the best
straight-line fit. But this does not say that the best fit to the data is a
straight line; you are assuming a straight line. Computing Pearson's r is
the same thing as proposing a linear model.

Even if we knew the parameters of
the line perfectly, the tendency of points to scatter away from the line
limits one's ability to predict the value of a given observed point's
Y-value from its X-value.

The precision of the estimates of the _line's_ parameters depends directly
on the number of independent points in the sample. The precision of the
estimates of a _point's_ position depends on the precision of estimates of
the line's position and the degree of scatter of Y-values about the
X-value in

I understand what you're saying. If the data do in fact represent an
underlying linear relationship with added noise, then the slope of the
regression line, given a decent number of data points, will accurately
reflect the underlying linear relationship. But that is only one limiting
case. Suppose that what was actually going on was that each point lay on a
_different_ line, with a different slope and intercept, with some added
random noise as well. In this case the hypothesis of a single underlying
linear relationship is wrong, yet you will still be able to compute,
accurately, a correlation and a regression line. The regression line,
however, will not reflect the real situation that held when ANY of the data
points was taken, except by chance.

When you have a very high correlation, the spread of the possible actual
regression lines becomes less; in the limit of a perfect correlation, there
is only one regression line possible and it will fit all of the points.

Let us assume that there is a simple function relating Y to X. Let us
further assume that Y is not only affected by X, but also by a small
random disturbance. How would you go about nailing down the true
function, given the ability to sample Y-values at any desired X-value?

One way would be to set X to a given value and then repeatedly sample Y.
You would end up with a normal distribution of Y-values centered about
their mean. With a large enough sample you could estimate the the true
value of Y (its value in the absence of the random disturbance) to any
desired degree of precision. You might then try another X-value and
repeat the process, then again, and again, until you had swept out the
function over the range of interest. You might even be able to find a
mathematical formula that perfectly fit the estimated true Y-values.

Now, computing Pearson r for the set of points you observed in this
process, you discover that it is 0.886. Only one bit of information about
the position of Y given X, and yet you have established that the
underlying function is, say, cubic, with an extremely high level of
confidence. Go figure.

I am figuring. If the function is a cubic, the correlation of Y with X^3
will be very high -- but the correlation of Y against X might well be 0.866
or lower. The reason there is only one bit of information in the
relationship between Y and X is that y = aX + b is the wrong model. But
there would be far more information in the relation Y = X^3.

The procedure you describe would not end up with a correlation of 0.866, but
a far higher correlation. For each value of X, you would have many values of
Y, enough to establish the mean for each pair of points with a high
accuracy, averaging out the random variations. This is in fact how good data
are obtained in physical measurements in the presence of noise. You would
end up with a curve showing the static relationship between X and Y, to
which you could then fit a curve by standard methods, and of course with all
the hazards of curve-fitting to deal with. You would NOT use a Pearson's r
on the raw data, because that would amount to arbitrarily fitting a straight
line to the curve. Instead, you would calculate the correlation between the
fitted curve and the data, an entirely different procedure. The fitted curve
is the model; your hypothesis is that this curve fits the data, not that a
straight line fits the data. If your model's best fit to the data proved to
give a correlation of 0.866, you would have to count the model as a failure:
at best, it could only predict the sign of Y given the sign of X.

Linear models are "powerful" only if you have some independent way of
knowing that the relationship should be linear.

No, I mean by "powerful" the ability to improve prediction relative to what
is possible. In a large number of cases a linear model will work nearly as
well as one embodying the "true" nonlinear function. In otherwords, linear
models often turn out to be excellent approximations, at least over some
practical range of values in which one has an interest.

When you say that a linear model will work nearly as well as the "true"
function, you're assuming that the "true" function is linear. The whole
point is that if you don't know what the true function is, ASSUMING a linear
model produces only whatever results from assuming that model. If you get a
correlation of 0.866, your model is clearly not very good, or the data are
pretty bad.
......

And also because [linear models so often turn out to work surprisingly
well even when they are not actually the correct function.

What do you mean, they turn out to work surprisingly well? How do you know
they do? You can't mean that a linear model with a correlation of 0.866
_predicts_ surprisingly well; it can predict only the sign of the
relationship. In order to know how good a prediction really is, you have to
have some way of determining what the _actual_ relationship between X and Y
is, but if you knew that, why would you go through all this statistical stuff?

You can't get away from the fact that with a correlation of only 0.866,
you could not distinguish a straight line from an exponential
relationship >by ANY means. But I should let Richard have his say on >>that.

I think you can see now that this is nonsense. But by all means, let
Richard have his say.

Yes, let's. I do not think it is nonsense. You are talking about _assuming_
a straight line relationship, determining the best-fit straight line, and
then concluding that the real relationship must be a straight line. THAT is
what is nonsense.

Best,

Bill P.

[From Bruce Abbott (970419.2050 EST)]

Bill Powers (970418.0500 MST) --

Bruce Abbott (970417.2200 EST)

This confuses the accuracy of predicting an observed Y-value given
knowledge of its X-value with the accuracy of pinning down the slope and
intercept of the line used to generate the prediction.

I thought the whole point was to predict Y given X. The basis hypothesis is
"These data represent a linear relationship between X and Y." The
least-squares fit then gives you the slope and intercept of the best
straight-line fit. But this does not say that the best fit to the data is a
straight line; you are assuming a straight line. Computing Pearson's r is
the same thing as proposing a linear model.

You can also use Pearson r to assess the fit of a nonlinear model to
nonlinear data.

I understand what you're saying. If the data do in fact represent an
underlying linear relationship with added noise, then the slope of the
regression line, given a decent number of data points, will accurately
reflect the underlying linear relationship. But that is only one limiting
case. Suppose that what was actually going on was that each point lay on a
_different_ line, with a different slope and intercept, with some added
random noise as well. In this case the hypothesis of a single underlying
linear relationship is wrong, yet you will still be able to compute,
accurately, a correlation and a regression line. The regression line,
however, will not reflect the real situation that held when ANY of the data
points was taken, except by chance.

Yes. If other, uncontrolled variables are having a strong influence on
function whose parameters you are trying to determine, while you are doing
the measurements, you are in trouble, right here in River City. That's the
problem with inference -- you can never be sure there isn't some other
explanation than the one you propose. But of course, there are ways to
assess whether the sort of thing you suggest is actually happening.

When you have a very high correlation, the spread of the possible actual
regression lines becomes less; in the limit of a perfect correlation, there
is only one regression line possible and it will fit all of the points.

Correct.

If the function is a cubic, the correlation of Y with X^3
will be very high -- but the correlation of Y against X might well be 0.866
or lower. The reason there is only one bit of information in the
relationship between Y and X is that y = aX + b is the wrong model. But
there would be far more information in the relation Y = X^3.

This is total nonsense.

The procedure you describe would not end up with a correlation of 0.866, but
a far higher correlation. For each value of X, you would have many values of
Y, enough to establish the mean for each pair of points with a high
accuracy, averaging out the random variations. This is in fact how good data
are obtained in physical measurements in the presence of noise. You would
end up with a curve showing the static relationship between X and Y, to
which you could then fit a curve by standard methods, and of course with all
the hazards of curve-fitting to deal with. You would NOT use a Pearson's r
on the raw data, because that would amount to arbitrarily fitting a straight
line to the curve. Instead, you would calculate the correlation between the
fitted curve and the data, an entirely different procedure. The fitted curve
is the model; your hypothesis is that this curve fits the data, not that a
straight line fits the data. If your model's best fit to the data proved to
give a correlation of 0.866, you would have to count the model as a failure:
at best, it could only predict the sign of Y given the sign of X.

Let's simplify the situation a little by assuming that the true relationship
between X and Y _is_ linear: Y = X. This way we won't get into a dispute
about how much of the reduction in correlation (using the actual points
rather than the means at each X-value) is due to nonlinearity in the true
function.

The observed Y values consist of the true Y plus a good deal of random
error. You sample Y 100 times at each of 10 X-values ranging from 10 to
100. You then compute the mean Y at each X. Here are the results (taken
from a Minitab simulation):

     Y Y'
    10 12.01
    20 18.75
    30 32.46
    40 41.25
    50 51.27
    60 59.63
    70 69.41
    80 81.80
    90 91.38
   100 103.01

Here, Y is the actual Y value and Y' is the value derived from averaging
observed values of Y at each X value. The 95% confidence interval for these
estimates is Y' plus or minus 3.9. This means that, on average, the true
value of Y will fall within plus or minus 3.9 units of the sample mean on
95% of samples.

The correlation of predicted vs actual Y values in the original data set of
1000 points is 0.830. Thus, the best-fitting regression line accounts for
about 69% of the variation in Y', leaving about 31% of the variance
unaccounted for. This relationship is just about useless for predicting the
observed value of Y from X.

On the other hand, the correlation between the _mean_ predicted and actual Y
values is 0.999. Variation in X thus accounts for 99.8% of the variation in
mean predicted Y, leaving only 0.02% of variation unaccounted for. The
standard error of estimate for mean Y' from X is 1.45, for a 95% CI of + or
- 2.84, which is within approx. 3% over the 90-unit range of Y-values examined.

So, the same data that do not permit one to make useful predictions of
observed Y from knowledge of X do permit one to make a highly accurate
determination of the relationship between X and Y absent the influence on Y
of uncontrolled variation, i.e., the true underlying relationship.

When you say that a linear model will work nearly as well as the "true"
function, you're assuming that the "true" function is linear.

No, I'm not. A sine wave is not linear, is it? Yet over the range of
90-270 degrees, a straight line fits extremely well. A linear function will
fit an exponential function, or a logarithmic function, or a hyperbolic
function, extremely well over some range of values of the function.
_That's_ what I mean. By no means am I assuming that the "true" function is
linear. All it need be is monotonic over range of values under consideration.

The whole
point is that if you don't know what the true function is, ASSUMING a linear
model produces only whatever results from assuming that model. If you get a
correlation of 0.866, your model is clearly not very good, or the data are
pretty bad.

Again, a correlation of 0.866 is pretty damned good for estimating the
regression line. If the underlying function isn't linear, it certainly is
being fairly well approximated by one. In the demo a fit with a
correlation of 0.83 estimates the slope at 1.01 with an SD of 0.02, and the
intercept at 0.53 with an SD of 1.33. The true values were 1.00 and 0.00,
respectively.

What do you mean, they turn out to work surprisingly well? How do you know
they do? You can't mean that a linear model with a correlation of 0.866
_predicts_ surprisingly well; it can predict only the sign of the
relationship.

Oh, Bill, cut it out. What you are saying is nonsense. Trust me.

You are talking about _assuming_
a straight line relationship, determining the best-fit straight line, and
then concluding that the real relationship must be a straight line. THAT is
what is nonsense.

No, see above determination that a relationship is within 3% of linear (with
95% confidence), when the correlation of X and Y in the individual data
points is _less_ than 0.866. According to your logic, I shouldn't even be
able to predict the _sign_ of this relationship!

Regards,

Bruce

In a statistics textbook, an interpretation of r in terms of
displacement was presented. The bell shaped curve was divided into
tenths. A tenth was defined as .6745 times the population standard
deviation. The percentage of cases in each tenth from 1 through 10 was:
.31%, 1.80%, 6.72%, 16.13%, 25%, 25%, 16.13%, 6.72%, 1.80%,
.31%.

A table was given in the text which showed for different correaltions
between test A and test B, what the number of "tenths" displacement
was.

As an example, the correlation between an IQ test given at the
beginning of first grade and a reading achievement test given at the end
of first grade is about .60. If one knows a person's IQ score, how
much does that predict how well the person will do in reading
achievement. Let us suppose that a person fell in the 6th tenth of the
IQ distribution. The probability is: 29/100 for being in the 6th
tenth( 0 displacement), 73/100 for being in the 5th, 6th or 7th tenth(
1 displacement), 94/100 for being in the 4th, 5th, 6th, 7th, 8th
tenth(2 displacement), etc.. of the reading achievement distribution.

If the school policy was to recommend special reading help to all
students who might be expected to score in the 1st, 2nd, or 3rd tenth,
I would not recommend this student for such help. If a different
student had an IQ score in the 4th tenth, the probability is 73/100
that the student would fall in the 3th, 4th or 5th tenth of the reading
achievement. I would probably recommend this student for special help.
The 4th tenth of the IQ distribution is an IQ of 80 to 90.

For the case of a correlation of .90, a test A score in the 6th tenth,
would lead to the expectations: 55/100 to be in the 6th tenth, 98/100
to be in 5th, 6th or 7th tenth, 99.9/100 to be in 4th, 5th, 6th, 7th or
8th tenths of test B score.

···

From: David Goldstein
Subject: Correlations
Date: 4/19/97

[From Bill Powers (970420.1053 MST)]

From: David Goldstein
Subject: Correlations
Date: 4/19/97

In a statistics textbook, an interpretation of r in terms of
displacement was presented. The bell shaped curve was divided into
tenths. A tenth was defined as .6745 times the population standard
deviation. The percentage of cases in each tenth from 1 through 10 was:
.31%, 1.80%, 6.72%, 16.13%, 25%, 25%, 16.13%, 6.72%, 1.80%,
.31%.

A table was given in the text which showed for different correaltions
between test A and test B, what the number of "tenths" displacement
was.

As an example, the correlation between an IQ test given at the
beginning of first grade and a reading achievement test given at the end
of first grade is about .60. If one knows a person's IQ score, how
much does that predict how well the person will do in reading
achievement. Let us suppose that a person fell in the 6th tenth of the
IQ distribution. The probability is: 29/100 for being in the 6th
tenth( 0 displacement), 73/100 for being in the 5th, 6th or 7th tenth(
1 displacement), 94/100 for being in the 4th, 5th, 6th, 7th, 8th
tenth(2 displacement), etc.. of the reading achievement distribution.

At a correlation of 0.5, you will correctly classify only 0.0006 of the
population even assuming that the IQ measure is exactly correct; at a
correlation of 0.8, you will correctly classify only 0.14 of the population.
So at 0.6, the proportion of correct classifications will be closer to the
lower number than the higher. I don't know how to relate your division into
"tenths" to these numbers, but I suspect that it, too, will result in a
large preponderance of misclassifications.

If the school policy was to recommend special reading help to all
students who might be expected to score in the 1st, 2nd, or 3rd tenth,
I would not recommend this student for such help. If a different
student had an IQ score in the 4th tenth, the probability is 73/100
that the student would fall in the 3th, 4th or 5th tenth of the reading
achievement. I would probably recommend this student for special help.
The 4th tenth of the IQ distribution is an IQ of 80 to 90.

As I said, even if the IQ test leaves no uncertainty as to the actual IQ,
the fact that the correlation with reading scores is only 0.6 means that
your decision as to the need for reading help (either way) has only a small
chance of being correct in any individual case. As the person who makes this
decision for a large number of students, you will benefit in the long run by
using this basis for deciding. But as a student being evaluated, you would
be better off if you were not subject to this evaluation. As a first grade
student, you are going to get only one chance at either being given help if
you need it during your first-grade year, or being treated as normal if you
don't need it. So population statistics mean nothing to you. What matters to
you is the huge likelihood of being misclassified and the consequences if
this happens.

I hope that Richard Kennaway will comment on this.

Best,

Bill P.

[From Bill Powers(970420.0917 MST)]

Bruce Abbott (970419.2050 EST) --

Bill Powers (970418.0500 MST) --

Bruce Abbott (970417.2200 EST)

I understand what you're saying. If the data do in fact represent an
underlying linear relationship with added noise, then the slope of the
regression line, given a decent number of data points, will accurately
reflect the underlying linear relationship. But that is only one limiting
case. Suppose that what was actually going on was that each point lay on
a _different_ line, with a different slope and intercept, with some added
random noise as well. In this case the hypothesis of a single underlying
linear relationship is wrong, yet you will still be able to compute,
accurately, a correlation and a regression line. The regression line,
however, will not reflect the real situation that held when ANY of the
data points was taken, except by chance.

Yes. If other, uncontrolled variables are having a strong influence on
function whose parameters you are trying to determine, while you are doing
the measurements, you are in trouble, right here in River City.

The point I was making did not have to do with other uncontrolled variables
affecting the observed relationship. I'm talking about the assumption that
the _same_ straight-line function was at work for _each_ of the data points.
This is especially important when you use data from many people to determine
the best-fit straight line. The differences between individual
characteristics are not likely to be a matter of random variations in the
sense that if you measured the same individual again, you would get a
randomly different result. They are differences in characteristics that can
be quite stable for a given person -- like the "k factor" in models of
tracking behavior. If you measured the k-factor over a group of people, you
would come up with a lot of scatter, but this is because the individuals
genuinely differ in this measure. So when you use such population data to
speak of how "people" track, you're generating a fictional picture which
really describes _no_ person unless that person happens to have a k-factor
near the mean.

If the function is a cubic, the correlation of Y with X^3
will be very high -- but the correlation of Y against X might well be
0.866 or lower. The reason there is only one bit of information in the
relationship between Y and X is that y = aX + b is the wrong model. But
there would be far more information in the relation Y = X^3.

This is total nonsense.

I don't think you disagree with me -- I've probably stated my point poorly.
Take another function where the discrepancy is more obvious. If Y = sin(X),
and the data cover a range of 10*pi in X, then the correlation of Y against
X will be close to zero, and the regression line will predict that Y = 0*x.
However, if you calculate U = sin(X) for each X, and then compute the
correlation of Y versus U, you will find a correlation of 1 and a regression
line of Y = U. All of this assumes that there is no noise in the measurements.
...................

Let's simplify the situation a little by assuming that the true
relationship between X and Y _is_ linear: Y = X. This way we won't get
into a dispute about how much of the reduction in correlation (using the
actual points rather than the means at each X-value) is due to
nonlinearity in the true function.

The observed Y values consist of the true Y plus a good deal of random
error. You sample Y 100 times at each of 10 X-values ranging from 10 to
100. You then compute the mean Y at each X. Here are the results (taken
from a Minitab simulation):

    Y Y'
   10 12.01
   20 18.75
   30 32.46
   40 41.25
   50 51.27
   60 59.63
   70 69.41
   80 81.80
   90 91.38
  100 103.01

Here, Y is the actual Y value and Y' is the value derived from averaging
observed values of Y at each X value. The 95% confidence interval for
these estimates is Y' plus or minus 3.9. This means that, on average, the
true value of Y will fall within plus or minus 3.9 units of the sample
mean on 95% of samples.

The correlation of predicted vs actual Y values in the original data set
of 1000 points is 0.830. Thus, the best-fitting regression line accounts
for about 69% of the variation in Y', leaving about 31% of the variance
unaccounted for. This relationship is just about useless for predicting
the observed value of Y from X.

On the other hand, the correlation between the _mean_ predicted and actual
Y values is 0.999. Variation in X thus accounts for 99.8% of the
variation in mean predicted Y, leaving only 0.02% of variation unaccounted
for. The standard error of estimate for mean Y' from X is 1.45, for a 95%
CI of + or - 2.84, which is within approx. 3% over the 90-unit range of
Y-values examined.

So, the same data that do not permit one to make useful predictions of
observed Y from knowledge of X do permit one to make a highly accurate
determination of the relationship between X and Y absent the influence on
Y of uncontrolled variation, i.e., the true underlying relationship.

Yes, this is what I tried to describe -- glad to know it's true. If we have
reason to assume that there is truly a single linear relationship between X
and Y -- for example, because these are all repeated measurements of a
single individual -- then we can come up with a highly accurate estimate of
an underlying linear relationship.

However, our ability to predict Y on the basis of a given measurement of X
is still very poor, if the correlation is only 0.866. This is not important
from the standpoint of a person who deals with populations rather than
individuals, or who is more concerned with the track record than with each
isolated case. But we're talking about individual measurements.

When you say that a linear model will work nearly as well as the "true"
function, you're assuming that the "true" function is linear.

No, I'm not. A sine wave is not linear, is it? Yet over the range of
90-270 degrees, a straight line fits extremely well. A linear function
will fit an exponential function, or a logarithmic function, or a
hyperbolic function, extremely well over some range of values of the
function. _That's_ what I mean. By no means am I assuming that the "true"
function is linear. All it need be is monotonic over range of values
under consideration.

We're at cross-purposes here. To fit a straight line is to assume that the
straight line fits, and then see how well it fits. If you thought the curve
was a sine-wave, you'd assume a sine-wave and see how well it fits.

Actually, when you say that a straight line fits the sine wave over that
range extremely well, you're speaking in terms of the correlation, not the
actual errors of fit. If you compare the straight line to the sine wave at
an angle of 45 degrees, the straight line predicts a value of 0.5 and the
sine-wave a value of 0.707, an error of about 30 percent (theory predicts
low). I'll take your word that the correlation is 0.99-plus; this only goes
to show how correlations exaggerate the goodness of fit. We tend to think of
a number like 99% as "almost perfect," but if the number is a correlation
it's far from that.

When I proposed a standard of 0.95 correlations, a lot of people threw up
their hands and said that would be impossible. But that degree of
correlation assumes a _mean_ measurement error of almost 10 percent and
outliers much larger than that, much worse than what we expect of freshman
students in a beginning physics lab.

Again, a correlation of 0.866 is pretty damned good for estimating the
regression line. If the underlying function isn't linear, it certainly is
being fairly well approximated by one. In the demo a fit with a
correlation of 0.83 estimates the slope at 1.01 with an SD of 0.02, and
the intercept at 0.53 with an SD of 1.33. The true values were 1.00 and
0.00, respectively.

Let's try to stay focussed on the main point, which is how useful the
regression line is for predicting Y from X for a single instance. If you use
a million points, you can say that the regression line is accurate to one
part in a thousand, or whatever it turns out to be -- but you still can't
predict Y from a single measure of X with any degree of usefulness.

Every argument against this conclusion turns out to convert the problem into
one of population measures or multiple-trial measures. That's not the point.
If a screening test for a job applicant can be shown to correlate 0.866 with
success on the job, for the _employer_ that is a very worthwhile test,
because the employer is concerned only about the long-term benefit, over
many applicants. But for the applicant, who will either get the job or be
turned down, the test is grossly unfair, or else grossly too lenient --
depending on which way the inevitable misclassification goes. Richard
Kennaway showed that with a correlation of 0.866, only 26 percent of the
applicants would be correctly classified at the 5% confidence level. There
is a very large chance of getting the job when you won't be able to handle
it, or being turned down when the job is well within your abilities. The
benefits of using this test are all on the side of the employer.

What do you mean, they turn out to work surprisingly well? How do you
know they do? You can't mean that a linear model with a correlation of
0.866_predicts_ surprisingly well; it can predict only the sign of the
relationship.

Oh, Bill, cut it out. What you are saying is nonsense. Trust me.

It's not nonsense. I work for the government, so YOU can trust ME. We're
talking about single predictions; you can refine your regression line until
you're blue in the face, and you will won't be able to make a single
prediction that does better than predict the sign of Y at the .05 level, if
the correlation is only 0.866.

You are talking about _assuming_
a straight line relationship, determining the best-fit straight line, and
then concluding that the real relationship must be a straight line. THAT
is what is nonsense.

No, see above determination that a relationship is within 3% of linear
(with 95% confidence), when the correlation of X and Y in the individual
data points is _less_ than 0.866. According to your logic, I shouldn't
even be able to predict the _sign_ of this relationship!

Apples and oranges. You're talking about _estimating the relationship_ by
using the whole data set. I'm talking about _predicting the magnitude_ of
one point in Y from knowing the magnitude of one value of X and the
relationship. You can increase the accuracy of the relationship, or the
apparent relationship, as much as you like by making more observations. But
if the correlation is still only 0.866, your ability to predict one Y from
one X remains as low as ever.

Best,

Bill P.

[From Bruce Abbott (970420.1600 EST)]

Bill Powers(970420.0917 MST) --

Bruce Abbott (970419.2050 EST)

Bill Powers (970418.0500 MST)

If the function is a cubic, the correlation of Y with X^3
will be very high -- but the correlation of Y against X might well be
0.866 or lower. The reason there is only one bit of information in the
relationship between Y and X is that y = aX + b is the wrong model. But
there would be far more information in the relation Y = X^3.

This is total nonsense.

I don't think you disagree with me -- I've probably stated my point poorly.
Take another function where the discrepancy is more obvious. If Y = sin(X),
and the data cover a range of 10*pi in X, then the correlation of Y against
X will be close to zero, and the regression line will predict that Y = 0*x.
However, if you calculate U = sin(X) for each X, and then compute the
correlation of Y versus U, you will find a correlation of 1 and a regression
line of Y = U. All of this assumes that there is no noise in the measurements.

Well, I don't disagree with _that_, but it seems to have little to do with
what you said before about the number bits of information in a single
observation and being able to ascertain the _form_ of a _relationship_.

So, the same data that do not permit one to make useful predictions of
observed Y from knowledge of X do permit one to make a highly accurate
determination of the relationship between X and Y absent the influence on
Y of uncontrolled variation, i.e., the true underlying relationship.

Yes, this is what I tried to describe -- glad to know it's true. If we have
reason to assume that there is truly a single linear relationship between X
and Y -- for example, because these are all repeated measurements of a
single individual -- then we can come up with a highly accurate estimate of
an underlying linear relationship.

However, our ability to predict Y on the basis of a given measurement of X
is still very poor, if the correlation is only 0.866. This is not important
from the standpoint of a person who deals with populations rather than
individuals, or who is more concerned with the track record than with each
isolated case. But we're talking about individual measurements.

Yes, I stated that at the outset. I said that I agreed with Kennaway's
analysis, but was afraid that his conclusion about the prediction of
_individual_ Ys given X would be mistakenly taken to apply to the problem of
estimating the underlying _relationship_ between X and Y. Judging from what
I heard, that fear was justified.

When you say that a linear model will work nearly as well as the "true"
function, you're assuming that the "true" function is linear.

No, I'm not. A sine wave is not linear, is it? Yet over the range of
90-270 degrees, a straight line fits extremely well. A linear function
will fit an exponential function, or a logarithmic function, or a
hyperbolic function, extremely well over some range of values of the
function. _That's_ what I mean. By no means am I assuming that the "true"
function is linear. All it need be is monotonic over range of values
under consideration.

We're at cross-purposes here. To fit a straight line is to assume that the
straight line fits, and then see how well it fits. If you thought the curve
was a sine-wave, you'd assume a sine-wave and see how well it fits.

Of course. But let's say that the "true" function relating i to p in the
perceptual input function is logarithmic. However, within the range of
input values normally experienced by the system, p = k1*i provides a very
good approximation. This linear model will do nearly as well as the correct
one in modeling the behavior of the control system; that's why linear
functions are often described as "powerful," and that is why they are
usually assumed (as a first approximation) when the actual function is unknown.

Actually, when you say that a straight line fits the sine wave over that
range extremely well, you're speaking in terms of the correlation, not the
actual errors of fit. If you compare the straight line to the sine wave at
an angle of 45 degrees, the straight line predicts a value of 0.5 and the
sine-wave a value of 0.707, an error of about 30 percent (theory predicts
low). I'll take your word that the correlation is 0.99-plus; this only goes
to show how correlations exaggerate the goodness of fit. We tend to think of
a number like 99% as "almost perfect," but if the number is a correlation
it's far from that.

The error would actually be quite a bit smaller than that, as the line is
not fit betwee y = 1 and y = 0 (yielding 0.5 at the midpoint), but at some
intercept that minimizes the rms error. It slightly overestimates Y in the
middle and slightly overestimates it at the ends. Also, in the situation I
described, the line was being fit between Y = +1 and Y = -1; the midpoint is
0 and the linear function passes through it.

When I proposed a standard of 0.95 correlations, a lot of people threw up
their hands and said that would be impossible. But that degree of
correlation assumes a _mean_ measurement error of almost 10 percent and
outliers much larger than that, much worse than what we expect of freshman
students in a beginning physics lab.

You can achieve such large correlations only when the measurement error is
small relative to the excursions of the variables in question, and
everything you need to measure is accessible for measurement. This happens
to be the case in tracking experiments, but it shouldn't be taken on faith
that it will be true in other studies where the relevant variables are not
available for direct measurement. Freshman students in a beginning physics
lab do not face that problem, and most of the time, neither do their professors.

This is not to suggest that one shouldn't try for better data. A little bit
of those billions of dollars that go into physics might provide a few
psychologists with some of the necessary instrumentation.

Again, a correlation of 0.866 is pretty damned good for estimating the
regression line. If the underlying function isn't linear, it certainly is
being fairly well approximated by one. In the demo a fit with a
correlation of 0.83 estimates the slope at 1.01 with an SD of 0.02, and
the intercept at 0.53 with an SD of 1.33. The true values were 1.00 and
0.00, respectively.

Let's try to stay focussed on the main point, which is how useful the
regression line is for predicting Y from X for a single instance. If you use
a million points, you can say that the regression line is accurate to one
part in a thousand, or whatever it turns out to be -- but you still can't
predict Y from a single measure of X with any degree of usefulness.

That's _your_ main point, and I don't disagree with it. Mine is that such
results are nevertheless important in helping us to understand what
variables are involved and how they are related -- important information for
model-building. And presumably, if we could _then_ identify those _other_
relations that contribute to an individual observation departing from the
predicted, we'd be in a position to develop models capable of generating
those extremely high correlations, and we'd be able to predict the
individual case with reasonable certainty.

Every argument against this conclusion turns out to convert the problem into
one of population measures or multiple-trial measures. That's not the point.

I have _never_ argued against this conclusion! I have argued against
_using_ this conclusion to make statements about the usefulness of moderate
correlations for purposes _other than_ making predicitions for individual cases.

If a screening test for a job applicant can be shown to correlate 0.866 with
success on the job, for the _employer_ that is a very worthwhile test,
because the employer is concerned only about the long-term benefit, over
many applicants. But for the applicant, who will either get the job or be
turned down, the test is grossly unfair, or else grossly too lenient --
depending on which way the inevitable misclassification goes. Richard
Kennaway showed that with a correlation of 0.866, only 26 percent of the
applicants would be correctly classified at the 5% confidence level. There
is a very large chance of getting the job when you won't be able to handle
it, or being turned down when the job is well within your abilities. The
benefits of using this test are all on the side of the employer.

Yes, and as I mentioned recently in an earlier post on this subject, that is
why very high correlations are demanded for psychological tests used for
such screening.

What do you mean, they turn out to work surprisingly well? How do you
know they do? You can't mean that a linear model with a correlation of
0.866_predicts_ surprisingly well; it can predict only the sign of the
relationship.

Oh, Bill, cut it out. What you are saying is nonsense. Trust me.

It's not nonsense. I work for the government, so YOU can trust ME.

Now I _know_ I've got to be careful! (;->

It _is_ nonsense. To fix it, you need to change the word "relationship" to
"Y given X" (or "X given Y," it works both ways). And by the way, the
_sign_ comes into the debate because Kennaway used normal variates, i.e.,
z-scores. The sign refers to the direction of the deviation from the mean
of Y, given a certain deviation (in a given direction) from the mean of X.

Apples and oranges. You're talking about _estimating the relationship_ by
using the whole data set. I'm talking about _predicting the magnitude_ of
one point in Y from knowing the magnitude of one value of X and the
relationship. You can increase the accuracy of the relationship, or the
apparent relationship, as much as you like by making more observations. But
if the correlation is still only 0.866, your ability to predict one Y from
one X remains as low as ever.

Yes -- you've made my point! Everyone happy now?

Cheerfully,

Bruce

Date 04/20/97

Bill, what do you mean by a correct classification?

Also, what is the lowest value of probability which would be acceptable
to you in this case: If a student is in the 6th tenth of the
distribution on test X, the probability is ??? that the student will be
in the 6th tenth of the distribution on test Y. What is the lowest ???
which would be acceptable?

From the table in the statistics book:

        Correlation Probability Of Same Tenth
        .00 .19
        .10 .20
        .30 .22
        .40 .24
        .50 .26
        .60 .29
        .70 .34
        .80 .41
        .90 .55
        .95 .71
        .98 .91
        1.00 1.00

With a correlation of .60, the probability of correctly predicting the
tenth of the distribution on Test Y given Test X is .29. One would be
wrongly classified 71% of the time, not the small number you mentioned.

This assumes that predicting the same tenth is an acceptable outcome.
If one is less fussy, say displacement no more than 1 tenth, then my
previous post applies.

···

From: David Goldstein
Subject: Bill Powers (970420.1053 MST)

[From Richard Kennaway (970421.1735 BST)]

Bill Powers (970420.1053 MST):

At a correlation of 0.5, you will correctly classify only 0.0006 of the
population even assuming that the IQ measure is exactly correct; at a
correlation of 0.8, you will correctly classify only 0.14 of the population.

Those are the proportions that will be *confidently* correctly classified
at the 5% level. The proportion actually correctly classified will always
be at least 50%, assuming the correlation is non-negative. For c=0.5, 67%
are correctly classified (table 3 of my paper), but you have almost no idea
which ones. For c=0.8, 80% are correctly classified: 14.2% confidently,
and 65.8% by lucky guessing.

As I said, even if the IQ test leaves no uncertainty as to the actual IQ,
the fact that the correlation with reading scores is only 0.6 means that
your decision as to the need for reading help (either way) has only a small
chance of being correct in any individual case.

...

I hope that Richard Kennaway will comment on this.

Your words seem spot on.

The question that such a low correlation would immediately raise in my mind
is: can I find some better way of deciding who to give extra help with
reading?

···

__
\/__ Richard Kennaway, jrk@sys.uea.ac.uk, http://www.sys.uea.ac.uk/~jrk/
  \/ School of Information Systems, Univ. of East Anglia, Norwich, U.K.

Richard Kennaway wrote:

···

[From Richard Kennaway (970421.1735 BST)]

Bill Powers (970420.1053 MST):
>At a correlation of 0.5, you will correctly classify only 0.0006 of the
>population even assuming that the IQ measure is exactly correct; at a
>correlation of 0.8, you will correctly classify only 0.14 of the population.

Those are the proportions that will be *confidently* correctly classified
at the 5% level. The proportion actually correctly classified will always
be at least 50%, assuming the correlation is non-negative. For c=0.5, 67%
are correctly classified (table 3 of my paper), but you have almost no idea
which ones. For c=0.8, 80% are correctly classified: 14.2% confidently,
and 65.8% by lucky guessing.

>As I said, even if the IQ test leaves no uncertainty as to the actual IQ,
>the fact that the correlation with reading scores is only 0.6 means that
>your decision as to the need for reading help (either way) has only a small
>chance of being correct in any individual case.
...
>I hope that Richard Kennaway will comment on this.

Your words seem spot on.

The question that such a low correlation would immediately raise in my mind
is: can I find some better way of deciding who to give extra help with
reading?

__
\/__ Richard Kennaway, jrk@sys.uea.ac.uk, http://www.sys.uea.ac.uk/~jrk/
  \/ School of Information Systems, Univ. of East Anglia, Norwich, U.K.

From: David Goldstein
Subject: Re.: Correlations
Date: 4/21/97

I noticed that the numbers that Richard was quoting as correct
classification were close to the ones in the table in the column
labelled "1" tenth displacement:

        Correlation Probability of Displacement < 1 tenth
        .00 .53
        .10 .55
        .20 .58
        .30 .61
        .40 .64
        .50 .69
        .60 .73
        .70 .81
        .80 .89
        .90 .98
        .95 .999

Using this as the definition of correct classification, a school policy
which says that children who are likely to fall in the 1,2, or 3 tenth
of the reading achievement distribution should be given extra help,
wouild mean that if the IQ score was in the 4th tenth ( 80 to 90 or
lower) should receive extra help. This would be wrong, 27% of the time
.