[From Bill Powers (970423.0649 MST)]
Bruce Abbott (970423.1210 EST)--
Bill Powers (970422.0757 MST) --
I didn't really understand this example when it was given, but now I do.
You have a function Y = f(X1, X2, X3, X4). There are no other variables,
known or unknown, affecting Y. Therefore Y, obviously, is perfectly
predicted by the function of the four variables, no matter what the
function is and no matter how the individual variables change.
The randomness of the four X variables is a red herring. They could vary
in any way, random or not, correlated or not, in any combination, and Y
would still be predicted perfectly by them. There is no random component
in the X variables: they are perfectly known.
Since the values of the four variables are perfectly known, this is not a
statistical problem.
Yes it is, Bill.
Wouldn't it be exactly the same problem if the four X's varied according to
independent but smooth, analytical, and perfectly predictable functions of
time? How would the problem be any different?
The statistical problem is that you, the researcher, do not _know_ how (or
even _if_) P1, P2, P3, and P4 relate to R. Discovering whatever
relationship exists is a problem of statistical inference.
That's not a _statistical_ problem, it's just a _mathematical_ problem. You
may have learned the mathematics of rotating axes and minimizing functions
in the context of statistics, but it has nothing fundamental to do with
random variables.
Examining each Px variable separately, you discover that each correlates
approximately 0.50 with R; thus approximately 25% of the variance in Px
and R is shared. None of the Px variables is doing a particularly good
job of "explaining" the variance in R.
However, you submit these four predictors and the response variable to a
multiple correlation analysis, which searches for simultaneous weightings
of the individual predictors that will yield the best prediction of R
(definitely a statistical problem). This analysis reveals that all the
variance in R can be accounted for by a linear combination of P1, P2, P3,
and P4, and the analysis gives you the optimum weights, given the data.
You have discovered a powerful model for predicting R, even though each
individual predictor was by itself a poor predictor of R.
In any function y = f(x1..x4), none of the x's does a "particularly good
job" of predicting the value of y even in the complete absence of
uncertainty. That's because y is a function of all the x's, not just one x,
and lacking any further statements we have to assume that the x's vary
independently (that is, none can be expressed as a function of the others).
However, if you have four samples of the function, with the x's being
independent of each other, you get four equations in four unknowns, which is
sufficient to solve for any non-stochastic linear function of the x's, y =
a*x1 + ..d*x4. You solve for the coefficients, and in the case you used you
find that they are all equal to 1. This is what your multiple correlational
analysis will find, too, because it is simply a method for finding the best
solution -- the best set of coefficients in the linear equation. If you set
all variances to zero, this method reduces to a way of solving simultaneous
algebraic equations.
It doesn't matter how the values of the four x's are generated, as long as
they're independent and the x's are observed exactly. This is why I said
their randomness is a red herring. This doesn't make a statistical problem
out of the situation; in order for a statistical problem to exist, the
relation of the x's to y would have to contain a random component, or the
observation of each x would have to contain a random component, or both. In
other words, in any given sample, the four x's would not add up to y, but to
y plus some random variation. In that case you _would_ need a statistical
analysis and much more than 4 samples to find the best set of coefficients
or weightings.
Even in that case it wouldn't matter how the individual x's were generated;
whether they each came from a random number generator or an independent
analytical or logical function. The x's are not the unknowns in the
statistical analysis; the coefficients or weightings are the unknowns. The
x's are _observed values_, and in your example are observed without any
uncertainty.
To make it a statistical problem you would have to say
Y(theoretical) = f(X1(actual), X2(actual), X3(actual), X4(actual)),
Y(observed) = f(X1(observed), X2(observed),
X3(observed), X4(observed)),
where
Xn(0bserved) = Xn(actual) + rn
rn = random variable
...
No, this does not make it a statistical problem. It makes it a somewhat
_different_ statistical problem. Now your individual predictors are not
only poorly related to R, they are also subject to unreliable measurement.
That is the case where it would matter that there is a random component in
the predictors. In fact, you could have considered your example to be a case
in which you find that the predictors are _highly_ unpredictable; every time
you measure them, you get a different value, with the whole set of values
having a normal distribution around zero for each predictor. If you looked
at it that way, the best estimate of the value of each predictor would be
zero, and according to the underlying relationship y = x1 + x2 + x3 + x4,
you would predict y = 0 for every observation. But that is not the case you
used; you assumed that each value of a predictor was meaningful and was
measured accurately, and that for every sample, the sum of the values
predicted y exactly. So the uncertainty in measuring the predictors was
zero, as was the uncertainty in the relationship of the predictors to y. The
only "uncertainty" remaining was your own, because you had not yet solved
the equations to find the values of the coefficients.
[For the case of 50% variance in the measurements]
This isn't _great_, but remember, we are assuming that the noise in the
measurement is as large as the variation in the signal being measured --
clearly a very bad measurement.
Yes, that is the case when the measurement's correlation with the actual
values of the variables is only r^2 = 0.5 (Kennaway) or r = 0.707. A very
bad measurement, as you say.
With noise amounting to 10% of signal, we
could account for 96% of the variance in R using the four predictor >variables.
... and of course that would imply a correlation of sqrt(0.92) or about 0.96
between each predictor measurement and the true value of the predictor.
My point in raising the example was that a combination of predictors,
which individually correlate rather poorly with the response variable,
could nevertheless do an excellent job of predicting R.
And I'm saying that this is a spurious conclusion, which obviously does not
endear me to you. I'll agree that a combination of four predictors can
improve the _relative_ uncertainty in the response variable (compared to
that of the predictors) by a factor of sqrt(4), or 2. But in your example
there was no uncertainty: the coefficients were exact, and the measurements
were exact. There was no _statistical_ uncertainty to reduce; only the
psychological uncertainty of not yet having solved the equations to find the
exact answer. When there is true statistical uncertainty, there is no exact
answer: only a mean plus unpredictable fluctuations.
My initial demo proved
that this is so. Random measurement error reduces this ability in
proportion to the size of the error, yet even then my statement holds: the
combination of predictors does better than any one of them alone.
Yes, a little bit, when there is any uncertainty in the data, and in terms
of _relative_ variance. The absolute variance in the result is always
greater than the variance in any observation. For _particular_ functions
relating the x's to the y's, the result can be far more variable than the
variance in the observations in either absolute or relative terms,
especially if the x's are correlated. Consider the relationship
qi = Fe(o) + Fd(qd) + noise.
when a control system is involved. Fe(o) is highly correlated with Fd(qd),
yet the correlation of each with qi is usually less than 0.1.
A low simple correlation between two variables does not preclude this
relationship from being scientifically useful.
Well, that's the point you hope to convince us of, but it depends entirely
on where you set your standard for scientific usefulness, and on what use
you intend to make of the results. If you sell widgets, a correlation of 0.1
between color and sales is useful to know about, provided that all colors
cost the same amount. If your future depends on the next customer buying a
widget, a correlation of 0.1 is pretty useless.
Best,
Bill P.