Draft paper on causal analysis and control systems

CSGnet Archive CSG2011
1

I've completed a first draft of a paper on extracting causal information from correlations in the context of dynamical systems with feedback (which of course includes control systems). "Causation does not imply correlation: robust violations of the faithfulness axiom" is currently available at
http://www.cmp.uea.ac.uk/~jrk/temp/RK-CausNonCorr20110331.pdf Comments welcome.

Readers of CSGNET may not be familiar with causal analysis as a field of statistics that forms the background to this paper. It goes back quite a long way but is still a matter of some controversy, even within the field, with, for example, this paper: http://www.springerlink.com/content/u4h7765l87v817x4/, and a whole conference entitled "Causality in Crisis?".

The gist of my paper is that dynamical systems tend to show patterns of correlation that may be drastically different from the causal links. Even without going as far as control systems, it is not difficult to demonstrate that a bounded differentiable function of time has in the long run zero correlation with its first derivative. A simple physical example is the relation between voltage across a capacitor and current through it; or in discrete time, daily bank balance and daily payments into or out of one's bank account (over a period in which one is getting neither richer nor poorer).

I mention the Test for the Controlled Variable and reference Bill's books.

There's no-one at my university who does causal analysis, but a colleague in statistics and machine learning suggested a few people elsewhere that might be able to take a view on the paper. I sent someone the draft last week (carefully avoiding April Fools Day!) but haven't had a reply yet.

···

--
Richard Kennaway, jrk@cmp.uea.ac.uk, http://www.cmp.uea.ac.uk/~jrk/
Tel. 01603 593212
School of Computing Sciences,
University of East Anglia, Norwich NR4 7TJ, U.K.

2

[From Rick Marken (2011.04.04.1510)]

I've completed a first draft of a paper on extracting causal information
from correlations in the context of dynamical systems with feedback (which
of course includes control systems). �"Causation does not imply correlation:
robust violations of the faithfulness axiom" is currently available at
http://www.cmp.uea.ac.uk/~jrk/temp/RK-CausNonCorr20110331.pdf Comments
welcome.

What a great paper, Richard! I'm no mathematician but I can follow
it pretty well. And so well written! Super.

Did you ever see the paper I did with a student on this topic? It
deals only with the "no correlation with causality" situation for
control systems and we illustrate the problem using experimental data.

In your paper you did discuss "The Test" as a way of getting around
the "faithfulness" problem with control systems. But the Test implies
a particular model of the system, and it's the right model if the
system is indeed a control system. But it wouldn't be the right model
for the voltage and current across a capacitor, of course. So it seems
to me that the general solution to the problem of determining
causality from correlation is having the correct model of the system;
a correct model being one that behaves like the real system under
identical circumstances. So it's the correlation of model with actual
behavior that is the test of causality; the causal (functional)
connections in the model can be considered to exist (tentatively,
pending more tests of the model and system) if the model behaves just
like the real system.

So I think that's what I would propose as an addition to your paper
(if you agree with my point); that ultimately causality is an
inference based on the match of model to actual behavior under many
different experimental circumstances. It's all about modeling (and
testing the model against observed behavior, of course), which should
make the mathematicians happy since that's what math is, right:
models.

Best regards

Rick

···

On Mon, Apr 4, 2011 at 10:44 AM, Richard Kennaway <jrk@cmp.uea.ac.uk> wrote:

Readers of CSGNET may not be familiar with causal analysis as a field of
statistics that forms the background to this paper. �It goes back quite a
long way but is still a matter of some controversy, even within the field,
with, for example, this paper:
http://www.springerlink.com/content/u4h7765l87v817x4/, and a whole
conference entitled "Causality in Crisis?".

The gist of my paper is that dynamical systems tend to show patterns of
correlation that may be drastically different from the causal links. �Even
without going as far as control systems, it is not difficult to demonstrate
that a bounded differentiable function of time has in the long run zero
correlation with its first derivative. A simple physical example is the
relation between voltage across a capacitor and current through it; or in
discrete time, daily bank balance and daily payments into or out of one's
bank account (over a period in which one is getting neither richer nor
poorer).

I mention the Test for the Controlled Variable and reference Bill's books.

There's no-one at my university who does causal analysis, but a colleague in
statistics and machine learning suggested a few people elsewhere that might
be able to take a view on the paper. �I sent someone the draft last week
(carefully avoiding April Fools Day!) but haven't had a reply yet.

--
Richard Kennaway, jrk@cmp.uea.ac.uk, Richard Kennaway
Tel. 01603 593212
School of Computing Sciences,
University of East Anglia, Norwich NR4 7TJ, U.K.

--
Richard S. Marken PhD
rsmarken@gmail.com
www.mindreadings.com

3

[From Bill Powers (201…04.04.1647 MDT)]

I’ve completed a first draft of
a paper on extracting causal information from correlations in the context
of dynamical systems with feedback (which of course includes control
systems). “Causation does not imply correlation: robust
violations of the faithfulness axiom” is currently available at


http://www.cmp.uea.ac.uk/~jrk/temp/RK-CausNonCorr20110331.pdf
Comments welcome.

It’s extraordinary that you are able to write a paper containing a
rigorous mathematical analysis and yet explain it as you go so a
less-gifted reader has a fighting chance of understanding it. I came very
close to understanding it. Of course I knew how it was going to come out,
but I had no idea that you would extract such very simple conditions as
resulting in causation without correlation. So nice. How do you suppose
natural selection managed to make something like that seem so pleasing?
Does this mean I’m going to reproduce again? I hope that’s just another
noncausal correlation.
This whole business ought to be the death knell of naive empiricism. If
one psychologist says “Effect B correlates with cause A,” he is
very likely to add, in a sort of unthinking reflex, “… but of
course correlation does not imply causation.” That doesn’t prevent
him from speaking of A as a cause and B as its effect. What you’ve done
is keep him from adding “…but of course if A does cause B, that
would explain the correlation.”
I think the problem here is the idea of causation itself. That’s a very
old and primitive way of explaining natural phenomena. When a remote
ancestor explained that lightning causes thunder, he was saying that
something invisible connected the lightning to the thunder, some magical
thing called causation. An effect with no mechanism. It just, like, you
know, causes it. Ever since then we’ve been burdened by that word,
which basically has no useful meaning at all.

Best,

Bill

···

At 06:44 PM 4/4/2011 +0100, Richard Kennaway wrote:

4

(Gavin Ritz 2011.04.05.11.43NZT)

[From Bill Powers
(201…04.04.1647 MDT)]

···

At 06:44 PM 4/4/2011 +0100, Richard Kennaway wrote:

Really great paper, this is a big issue in
science.

Regards

Gavin

I’ve completed a first
draft of a paper on extracting causal information from correlations in the
context of dynamical systems with feedback (which of course includes control
systems). “Causation does not imply correlation: robust violations
of the faithfulness axiom” is currently available at

http://www.cmp.uea.ac.uk/~jrk/temp/RK-CausNonCorr20110331.pdf
Comments welcome.

It’s extraordinary that you are able to write a paper containing a rigorous
mathematical analysis and yet explain it as you go so a less-gifted reader has
a fighting chance of understanding it. I came very close to understanding it.
Of course I knew how it was going to come out, but I had no idea that you would
extract such very simple conditions as resulting in causation without correlation.
So nice. How do you suppose natural selection managed to make something like
that seem so pleasing? Does this mean I’m going to reproduce again? I hope
that’s just another noncausal correlation.
This whole business ought to be the death knell of naive empiricism. If one
psychologist says “Effect B correlates with cause A,” he is very
likely to add, in a sort of unthinking reflex, “… but of course
correlation does not imply causation.” That doesn’t prevent him from
speaking of A as a cause and B as its effect. What you’ve done is keep him from
adding “…but of course if A does cause B, that would explain the
correlation.”
I think the problem here is the idea of causation itself. That’s a very old and
primitive way of explaining natural phenomena. When a remote ancestor explained
that lightning causes thunder, he was saying that something invisible connected
the lightning to the thunder, some magical thing called causation. An effect
with no mechanism. It just, like, you know, causes
it. Ever since then we’ve been burdened by that word, which basically has no
useful meaning at all.

Best,

Bill

5

Well said ... you made my day!

:slight_smile:

David Cross
d.cross@tcu.edu
www.davidcross.us

···

On Apr 4, 2011, at 6:23 PM, Bill Powers wrote:

[From Bill Powers (201..04.04.1647 MDT)]

At 06:44 PM 4/4/2011 +0100, Richard Kennaway wrote:

I've completed a first draft of a paper on extracting causal information from correlations in the context of dynamical systems with feedback (which of course includes control systems). "Causation does not imply correlation: robust violations of the faithfulness axiom" is currently available at
http://www.cmp.uea.ac.uk/~jrk/temp/RK-CausNonCorr20110331.pdf Comments welcome.

It's extraordinary that you are able to write a paper containing a rigorous mathematical analysis and yet explain it as you go so a less-gifted reader has a fighting chance of understanding it. I came very close to understanding it. Of course I knew how it was going to come out, but I had no idea that you would extract such very simple conditions as resulting in causation without correlation. So nice. How do you suppose natural selection managed to make something like that seem so pleasing? Does this mean I'm going to reproduce again? I hope that's just another noncausal correlation.

This whole business ought to be the death knell of naive empiricism. If one psychologist says "Effect B correlates with cause A," he is very likely to add, in a sort of unthinking reflex, "... but of course correlation does not imply causation." That doesn't prevent him from speaking of A as a cause and B as its effect. What you've done is keep him from adding "...but of course if A does cause B, that would explain the correlation."

I think the problem here is the idea of causation itself. That's a very old and primitive way of explaining natural phenomena. When a remote ancestor explained that lightning causes thunder, he was saying that something invisible connected the lightning to the thunder, some magical thing called causation. An effect with no mechanism. It just, like, you know, causes it. Ever since then we've been burdened by that word, which basically has no useful meaning at all.

Best,

Bill

6

Richard,

What you have written seems clear and to the point. I have one question, one comment and a couple of suggestions.

Question: Who are the intended audience, and where do you contemplate publishing? Or is it simply teaching material?

Comment: Has this not been proved before? At least for the causal relation between a variable and its derivative, I thought that the zero correlation was well known. At least in 1998 I took it for granted in my derivation of the maximum correlation between perception and disturbance in a control system with a pure integrator output function <http://www.mmtaylor.net/PCT/Info.theory.in.control/Control+correl.html&gt;\. The approach to zero for control systems and the use of non-correlation to suggest the existence of control is a different matter.

Suggestions:
1. You might mention other cases in which correlation is intrinsically nonzero, such as the symmetric distributions used in WIkipedia as an examples of direct association with zero correlation <http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient&gt;\. The associations plotted there could very easily be causal as could any number of functional relationships (examples: x varies over a range +-1, symmetrically about zero, y = x^2; x varies over a range large compared to 2*pi, y = sin(x)).

2. You might add a sentence both in the introduction and in the conclusion that nothing in the proofs or the discussion relates to causality or non-causality in cases of non-zero correlation. It's all too easy for someone to think that since you prove that causality can occur without correlation, therefore correlation can occur without causality, whereas in fact you say nothing about the latter situation.

For the appropriate audience, it's a useful reminder.

Martin

···

On 2011/04/4 1:44 PM, Richard Kennaway wrote:

I've completed a first draft of a paper on extracting causal information from correlations in the context of dynamical systems with feedback (which of course includes control systems). "Causation does not imply correlation: robust violations of the faithfulness axiom" is currently available at
http://www.cmp.uea.ac.uk/~jrk/temp/RK-CausNonCorr20110331.pdf Comments welcome.

Readers of CSGNET may not be familiar with causal analysis as a field of statistics that forms the background to this paper. It goes back quite a long way but is still a matter of some controversy, even within the field, with, for example, this paper: http://www.springerlink.com/content/u4h7765l87v817x4/, and a whole conference entitled "Causality in Crisis?".

The gist of my paper is that dynamical systems tend to show patterns of correlation that may be drastically different from the causal links. Even without going as far as control systems, it is not difficult to demonstrate that a bounded differentiable function of time has in the long run zero correlation with its first derivative. A simple physical example is the relation between voltage across a capacitor and current through it; or in discrete time, daily bank balance and daily payments into or out of one's bank account (over a period in which one is getting neither richer nor poorer).

I mention the Test for the Controlled Variable and reference Bill's books.

There's no-one at my university who does causal analysis, but a colleague in statistics and machine learning suggested a few people elsewhere that might be able to take a view on the paper. I sent someone the draft last week (carefully avoiding April Fools Day!) but haven't had a reply yet.

7

[From Richard Kennaway (2011.04.05.0730 BST)]

Martin Taylor [mmt-csg@MMTAYLOR.NET] writes:

Question: Who are the intended audience, and where do you contemplate
publishing? Or is it simply teaching material?

It's intended for people working in causal analysis, as a research paper. I've been reading the causal analysis literature, and I haven't seen anything like this there. Most of it assumes acyclic causal relationships. There are a few papers on causal analysis for cyclic causal graphs, and in one of them (the Itani paper in the references) there is a brief mention of difficulties with systems that "compensate in the face of perturbations", but nothing that I've seen looks squarely at this issue. They all just add conditions to exclude cases their methods can't handle. The Lacerda et al. reference is another example. It looks at systems in discrete time evolving according to x(t+1) = linear combination of the variables at time t + noise, but require that in the equation for x, the coefficient of x(t) is not 1. So you can't have x(t+1) = x(t) + other terms, which is the discrete-time equivalent of dx/dt = those other terms. If you have x(t+1) = kx(t) + other terms, with |k| < 1, this is a leaky integrator, which they recommend sampling on a timescale longer than the leakage. This actually avoids seeing the transient effects which could tell you more about what the system is doing.

1. You might mention other cases in which correlation is intrinsically
nonzero, such as the symmetric distributions used in WIkipedia as an
examples of direct association with zero correlation
<http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient&gt;\.
The associations plotted there could very easily be causal as could any
number of functional relationships (examples: x varies over a range +-1,
symmetrically about zero, y = x^2; x varies over a range large compared
to 2*pi, y = sin(x)).

I mentioned sin and cos, but with those examples, there's a very clear relationship between the variables just from eyeballing the graph. Using smooth random noise and then sampling on a timescale greater than the autocorrelation time of the waveform yields pure bivariate Gaussian noise, which is opaque to any statistical technique of discerning a relationship between the variables.

2. You might add a sentence both in the introduction and in the
conclusion that nothing in the proofs or the discussion relates to
causality or non-causality in cases of non-zero correlation. It's all
too easy for someone to think that since you prove that causality can
occur without correlation, therefore correlation can occur without
causality, whereas in fact you say nothing about the latter situation.

We've had a go-around on this in the past, I think. No correlation without causality is basically the Markov assumption, which in plainer English is a completeness assumption about the set of variables that one is considering. The set must include all variables that causally connect any variables in the set. One might call this "Markov closure". When the set of variables is Markov-closed, all other external influences on the variables must consist of independent sources of noise, each with a direct causal connection to just one of the variables in the set.

The Markov assumption has also been disputed, but the present paper just tackles the faithfulness axiom.

···

--
Richard Kennaway, jrk@cmp.uea.ac.uk, Richard Kennaway
School of Computing Sciences,
University of East Anglia, Norwich NR4 7TJ, U.K.

8

(Gavin Ritz 2011.04.05.18.56NZT)

[From Richard Kennaway
(2011.04.05.0730 BST)]

Richard

Can you show this non relationship
using mathematical category theory constructs?

Regards

Gavin

Martin Taylor [mmt-csg@MMTAYLOR.NET] writes:

Question: Who are the intended audience, and where
do you contemplate

publishing? Or is it simply teaching material?

It’s intended for people working in causal analysis,
as a research paper. I’ve been reading the causal analysis literature, and I
haven’t seen anything like this there. Most of it assumes acyclic causal
relationships. There are a few papers on causal analysis for cyclic causal
graphs, and in one of them (the Itani paper in the references) there is a brief
mention of difficulties with systems that “compensate in the face of
perturbations”, but nothing that I’ve seen looks squarely at this issue. They
all just add conditions to exclude cases their methods can’t handle. The
Lacerda et al. reference is another example. It looks at systems in discrete
time evolving according to x(t+1) = linear combination of the variables at time
t + noise, but require that in the equation for x, the coefficient of x(t) is
not 1. So you can’t have x(t+1) = x(t) + other terms, which is the
discrete-time equivalent of dx/dt = those other terms. If you have x(t+1) =
kx(t) + other terms, with |k| < 1, this is a leaky integrator, which they
recommend sampling on a timescale longer than the leakage. This actually
avoids seeing the transient effects which could tell you more about what the
system is doing.

  1. You might mention other cases in which
    correlation is intrinsically

nonzero, such as the symmetric distributions used
in WIkipedia as an

examples of direct association with zero
correlation

http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient.

The associations plotted there could very easily
be causal as could any

number of functional relationships (examples: x
varies over a range ±1,

symmetrically about zero, y = x^2; x varies over a
range large compared

to 2*pi, y = sin(x)).

I mentioned sin and cos, but with those examples,
there’s a very clear relationship between the variables just from eyeballing
the graph. Using smooth random noise and then sampling on a timescale greater
than the autocorrelation time of the waveform yields pure bivariate Gaussian
noise, which is opaque to any statistical technique of discerning a
relationship between the variables.

  1. You might add a sentence both in the
    introduction and in the

conclusion that nothing in the proofs or the
discussion relates to

causality or non-causality in cases of non-zero
correlation. It’s all

too easy for someone to think that since you prove
that causality can

occur without correlation, therefore correlation
can occur without

causality, whereas in fact you say nothing about
the latter situation.

We’ve had a go-around on this in the past, I think. No
correlation without causality is basically the Markov assumption, which in
plainer English is a completeness assumption about the set of variables that
one is considering. The set must include all variables that causally connect any
variables in the set. One might call this “Markov closure”.
When the set of variables is Markov-closed, all other external influences on
the variables must consist of independent sources of noise, each with a direct
causal connection to just one of the variables in the set.

The Markov assumption has also been disputed, but the
present paper just tackles the faithfulness axiom.

···

–

Richard Kennaway, jrk@cmp.uea.ac.uk,
Richard Kennaway

School of Computing Sciences,

University of East Anglia, Norwich NR4 7TJ, U.K.

9

[From Richard Kennaway (2011.04.05.10:11 BST)]

(Gavin Ritz 2011.04.05.18.56NZT)

Richard
Can you show this non relationship using mathematical category theory constructs?

Um...no. I really don't see a connection.

···

--
Richard Kennaway, jrk@cmp.uea.ac.uk, Richard Kennaway
Tel. 01603 593212
School of Computing Sciences,
University of East Anglia, Norwich NR4 7TJ, U.K.

10

[Martin Taylor 2011.04.05.10.44]

[From Richard Kennaway (2011.04.05.0730 BST)]

Martin Taylor [mmt-csg@MMTAYLOR.NET] writes:

Question: Who are the intended audience, and where do you contemplate
publishing? Or is it simply teaching material?

2. You might add a sentence both in the introduction and in the

conclusion that nothing in the proofs or the discussion relates to
causality or non-causality in cases of non-zero correlation. It's all
too easy for someone to think that since you prove that causality can
occur without correlation, therefore correlation can occur without
causality, whereas in fact you say nothing about the latter situation.

We've had a go-around on this in the past, I think. ...
The Markov assumption has also been disputed, but the present paper just tackles the faithfulness axiom.

Yes, my suggestion was simply that you make this point clearer, not to rehash the argument as to whether correlation without some causal linkage is possible.

Martin