Solution to disturbance info in perceptual signal

[From Bill Powers (930401.2200)]

Allen Randall (930401) --

I've jazzed up your C code a bit, adjusting it for the optimum
ko (10.0), and adding a correlation function to show various
correlations around the loop. I changed the printout to show
sp+qX instead of just qX; the result is that that column now
matches the disturbance except during transients. Instead of
printing to the screen, the program now prints to a file called
"allendat."

The main addition is a definitive solution to the problem of
whether "the perceptual signal contains information about the
disturbance."

The solution is simple, and both Rick and I have been totally
overlooking it. I can't say that Martin and you have been
overlooking it because you've been thinking mainly in terms of
information theory and can't be called to account in the same
way about PCT. Here is the answer to our argument:

···

-----------------------------------
When we are talking about an optimally-performing control system
that uses an integrator in its output, there are two answers:

1. The perceptual signal does not contain ANY SIGNIFICANT
information about the disturbance.

2. The perceptual signal contains ALMOST PERFECT information
about THE FIRST DERIVATIVE OF THE DISTURBANCE.
------------------------------------

To show this, I have added a variable dd which is just the first
difference of the disturbance, and put it into the printout just
before the error signal. I show the error signal to remove the
effect of the constant reference signal, which is set to 20.0.
Ignore the 0th line, which is a startup transient.

The results follow:

           qd qi dd se qo qX

   0: qd= 0.00 qi= 0.00 dd= 0.00 se=20.00 qo=20.00 qX=-20.00
   1: qd= 0.00 qi=20.00 dd= 0.00 se= 0.00 qo=20.00 qX= 0.00
   2: qd= 0.00 qi=20.00 dd= 0.00 se= 0.00 qo=20.00 qX= 0.00
   3: qd= 0.00 qi=20.00 dd= 0.00 se= 0.00 qo=20.00 qX= 0.00
   4: qd= 0.00 qi=20.00 dd= 0.00 se= 0.00 qo=20.00 qX= 0.00
   5: qd= 1.00 qi=21.00 dd= 1.00 se=-1.00 qo=19.00 qX= 2.00
   6: qd= 2.00 qi=21.00 dd= 1.00 se=-1.00 qo=18.00 qX= 3.00
   7: qd= 3.00 qi=21.00 dd= 1.00 se=-1.00 qo=17.00 qX= 4.00
   8: qd= 4.00 qi=21.00 dd= 1.00 se=-1.00 qo=16.00 qX= 5.00
   9: qd= 5.00 qi=21.00 dd= 1.00 se=-1.00 qo=15.00 qX= 6.00
  10: qd= 6.00 qi=21.00 dd= 1.00 se=-1.00 qo=14.00 qX= 7.00
  11: qd= 7.00 qi=21.00 dd= 1.00 se=-1.00 qo=13.00 qX= 8.00 12: qd= 8.00
  qi=21.00 dd= 1.00 se=-1.00 qo=12.00 qX= 9.00
  13: qd= 9.00 qi=21.00 dd= 1.00 se=-1.00 qo=11.00 qX=10.00
  14: qd=10.00 qi=21.00 dd= 1.00 se=-1.00 qo=10.00 qX=11.00
  15: qd=10.00 qi=20.00 dd= 0.00 se= 0.00 qo=10.00 qX=10.00
  16: qd=10.00 qi=20.00 dd= 0.00 se= 0.00 qo=10.00 qX=10.00
  17: qd=10.00 qi=20.00 dd= 0.00 se= 0.00 qo=10.00 qX=10.00
  18: qd=10.00 qi=20.00 dd= 0.00 se= 0.00 qo=10.00 qX=10.00
  19: qd=10.00 qi=20.00 dd= 0.00 se= 0.00 qo=10.00 qX=10.00
  20: qd=10.00 qi=20.00 dd= 0.00 se= 0.00 qo=10.00 qX=10.00
  21: qd=10.00 qi=20.00 dd= 0.00 se= 0.00 qo=10.00 qX=10.00
  22: qd=10.00 qi=20.00 dd= 0.00 se= 0.00 qo=10.00 qX=10.00
  23: qd=10.00 qi=20.00 dd= 0.00 se= 0.00 qo=10.00 qX=10.00
  24: qd=10.00 qi=20.00 dd= 0.00 se= 0.00 qo=10.00 qX=10.00
  25: qd=10.00 qi=20.00 dd= 0.00 se= 0.00 qo=10.00 qX=10.00
  26: qd=10.00 qi=20.00 dd= 0.00 se= 0.00 qo=10.00 qX=10.00
  27: qd= 8.00 qi=18.00 dd=-2.00 se= 2.00 qo=12.00 qX= 6.00
  28: qd= 6.00 qi=18.00 dd=-2.00 se= 2.00 qo=14.00 qX= 4.00
  29: qd= 4.00 qi=18.00 dd=-2.00 se= 2.00 qo=16.00 qX= 2.00
  30: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  31: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  32: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  33: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  34: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  35: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  36: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  37: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  38: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  39: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  40: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  41: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  42: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  43: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  44: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  45: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  46: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  47: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  48: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00
  49: qd= 4.00 qi=20.00 dd= 0.00 se= 0.00 qo=16.00 qX= 4.00

Correlations

o vs d: -1.00000
o vs qX: 1.00000
d vs qX: 1.00000
o vs sp: 0.03156 /* no info about ouput in perceptual sig */
d vs sp: -0.03156 /* no info about dist. in percept. signal*/

dd vs sp: 1.00000 /* perfect info about d(d)/dt in p. sig. */

The correlations are done for all the data entries except the
first, to eliminate the startup transient. These aren't real
correlations because of the distributions, but they convey the
idea well enough.
I hope we can now put this argument behind us.
--------------------------------------------------------------
Here is Allen's C code, revised:

/* Martin Taylor's Mystery Function */
/* Based on Bill Powers' Primer */
/* Allan Randall, Martin Taylor, Jeff Hunter */
/* Modified by Bill Powers for optimum gain; correlations added,
   with declarations and change to new ANSI style for
   Borland C 2.0. printout changed to show se + qX .
   Also, shows first differences of disturbance and
   correlations of first difference with perceptual signal */

#include <stdio.h>
#include <conio.h>
#include <math.h>

int max_loop,i; /* number of iterations, loop counter*/
int dist_time; /* iterations between disturbances */

double qi, qo, qd, qX; /* input, output, disturbance, mystery
*/
double dd; /* first difference of disturbance */
double sp, sr, se; /* percept, reference, error */
double ki, ko, kf; /* constants: in, out, feedback */
double dt; /* time step of discrete simulation */

double dlist[100]; /* lists of variables for correlations */
double ddlist[100];
double selist[100];
double splist[100];
double olist[100];
double qXlist[100];

double Mystery(double r,double p); /* function declarations */
double Output(double e);

double xbar,ybar,sx,sy,ssx,ssy,sxy;
double correl,regress,sigx,sigy;

void correlation(double *x, double *y, int size)
{
int i;
sx = sy = ssx = ssy = sxy = 0.0;
for(i=0;i<size;++i)
  {
   sx += x[i];
   sy += y[i];
  }
  xbar = sx/size;
  ybar = sy/size;
for(i=0;i<size;++i)
  { ssx += (x[i] - xbar) * (x[i] - xbar);
   ssy += (y[i] - ybar) * (y[i] - ybar);
   sxy += (x[i] - xbar) * (y[i] - ybar);
  }
sigx = sqrt(ssx/size);
sigy = sqrt(ssy/size);
correl = sxy/(size*sigx*sigy);
regress = correl*sigy/sigx;
}

/* The output function */
double Output(double e)
{
  return (qo + ko*e*dt);
}

/* The mysterious function that uses r and p *only* to
reconstruct d */

double Mystery(double r,double p)
{
  double e;

  e = r - p;
  return (qX - ko*e*dt);
}

void main ()
{
FILE *fp;
  fp = fopen("Allendat","w");

  max_loop = 50;
  dist_time = 1;

  sr = 20.0;

  qi = qo = qd = qX = dd = 0.0;

  ki = 1.0;
  ko = 10.0;
  kf = 1.0;
  dt = 0.1;

  clrscr();
  fprintf(fp,
  " qd qi dd se qo
qX\n\n");

  /* MAIN LOOP */
  for (i=0; i<max_loop; i++) {
     /* Disturbance */ if (i > 4 && i < 15) qd = qd + 1;
     if (i > 26 && i < 30) qd = qd - 2;
     dd = qd - dd;

     /* Percept */
     qi = kf*qo + qd;
     sp = ki*qi;

     /* Iterate the Mystery Signal */
     /* Will it match the disturbance??? */
     qX = Mystery(sr,sp);

     /* Iterate the Output Signal */
     se = sr - sp;
     qo = Output(se);

     olist[i] = qo;
     dlist[i] = qd;
     selist[i] = se;
     splist[i] = sp;
     qXlist[i] = qX;
     ddlist[i] = dd;

     fprintf(fp,"%4d: qd=%5.2f qi=%5.2f dd=%5.2f se=%5.2f
qo=%5.2f qX=%5.2f\n",
            i, qd, qi, dd, se, qo, sp + qX);
     dd = qd;
  }
  fprintf(fp,"\nCorrelations\n\x0d\x0a");
  correlation(&olist[1],&dlist[1],49);
  fprintf(fp,"o vs d: %7.5f\n",correl);
  correlation(&dlist[1],&qXlist[1],49);
  fprintf(fp,"o vs qX: %7.5f\n",correl);
  correlation(&dlist[1],&qXlist[1],49);
  fprintf(fp,"d vs qX: %7.5f\n",correl);
  correlation(&olist[1],&splist[1],49);
  fprintf(fp,"o vs sp: %7.5f\n",correl);
  correlation(&dlist[1],&splist[1],49);
  fprintf(fp,"d vs sp: %7.5f\n",correl);
  correlation(&ddlist[1],&splist[1],49);
  fprintf(fp,"dd vs sp: %7.5f\n",correl);
}
----------------------------------------------------------

PS to Martin Taylor: I loved the Shagsper citation.

Best,

Bill P.

[Allan Randall (930402.1130 EST)]

Bill Powers (930401.2200)

When we are talking about an optimally-performing control system
that uses an integrator in its output, there are two answers:

1. The perceptual signal does not contain ANY SIGNIFICANT
information about the disturbance.

2. The perceptual signal contains ALMOST PERFECT information
about THE FIRST DERIVATIVE OF THE DISTURBANCE.

Are you then saying that this almost perfect information about the
first derivative is not "significant" information? Or are you
claiming that information about derivative is not information about
disturbance? It's not clear to me which position you are taking.

Correlations
o vs d: -1.00000
o vs qX: 1.00000
d vs qX: 1.00000
o vs sp: 0.03156 /* no info about ouput in perceptual sig */
d vs sp: -0.03156 /* no info about dist. in percept. signal*/
dd vs sp: 1.00000 /* perfect info about d(d)/dt in p. sig. */

You and Rick continually insist on measuring information with things
like correlation and variance statistics. This is not accurate. By
your own assumption that correlation measures information, qX has
information about d, but sp does not. But qX was extracted from sp!
The *real* definitions of information (I gave working versions in
a previous post) define information algorithmically in terms of
extractability!

I hope we can now put this argument behind us.

It would appear from the above (and from Rick's posting) that we are
still seriously in disagreement, but correct me if I'm wrong. I still
get the impression that you and Rick do not think that there is any
*useful* information in the percept about disturbance. I would claim
that this is the very information the control system is interested in!
The fact that the control system collects information through a channel
which it is trying to block *sounds* contradictory, I know. But it
really isn't. This is the nature of a closed-loop control system.
Information used to block the channel is collected through that very
channel.

···

-----------------------------------------
Allan Randall, randall@dciem.dciem.dnd.ca
NTT Systems, Inc.
Toronto, ON