E. coli calculations; falsifiability

[From Bill Powers (960223.0630 MST)]

Martin Taylor 960222 18:30 --

Good, more results on the E. coli calculations.

Some questions. In 2 dimensions, there seems to be a tendency to move
away from the target up to a radius of 2 units. But this makes sense
only if we include the case where the object moves _past_ the target,
its final position being measured not at its closest approach but at the
end of an integer number of steps. Consider this case:

        start end
          X ------------|-----------|
                     :
                     :
                     :
                     :
                     :
                     o target

The first step brings the object closer to the target, but the next step
takes the object farther from the target than the starting position. So
this would appear to be a move _away_ from the target, because of the
discrete nature of the steps, even though the initial step brought the
object closer to the target.

I don't know if this will make much difference in your final results,
but you really should treat the continuous case to find the actual point
of closest approach. With the discrete approach you are always going to
end up with one extra step that takes the object past the point of
closest approach. For small numbers of steps this can skew your results
considerably.

Approximating the continuous case involves reducing the size of a step
in relation to a unit of distance in the space. If you made the step one
tenth of the size above, the overshoot would be only one tenth of a unit
of distance, and the same move as above would end with the object closer
to the target instead of further away.

I'd like to understand how you pick a new direction at random. The way I
would do it would be to pick a random delta-x[i] for each x[i], then
multiply the distance moved in each dimension by

          delta-x[i]

···

---------------
       sqrt(sum(delta-x[i]^2))
             i

To get the change in position on each axis.

Is this what you did?
----------------------------------------------------------------------
REMI COTE (960223.1330 EST) --

     Remi: the basic of science I think is not to consider a theory
     "supported" by fact, but unfalcified by fact. Skinerian have
     plenty of experimental data that "support" their theory. It is not
     what determine their value.

That's an "s" in "basis" and "falsified." No extra charge for the
service.

The idea that observations cannot support a theory, but can only falsify
it, applies (in my opinion) only to a certain kind of theory, the kind
that is presented as a qualitative statement that is either true or
false. Another way to say this is that falsification is the only choice
when there is some reasonable possibility that an observation could
agree with a theory by chance. If observation matches theory, but could
have done so even if the theory is wrong, we could only reject obviously
wrong theories, and would have to accept theories when we couldn't
disprove them in this way.

However, suppose that the match of observation to theory is
quantitative. Imagine a theory that predicts the behavior of a capacitor
charged to an initial voltage and then discharged by connecting a known
resistor across its terminals. The theory allows us to calculate the
detailed behavior of the voltage as time passes; it says that the
voltage decays along a curve E0*exp(-t/RC). We can calculate the exact
form of this decay curve and compare the predicted curve point by point
with the actual measured curve. We find that the theoretical curve
matches the observed curve to our limits of measurement.

Now, what does it mean to calculate the probability that the observed
voltage curve matched the theoretical curve "by chance?" To calculate
the probability, we would have to know the domain of all possible ways
the voltage _could have behaved_. For example, the voltage might have
followed a decaying exponential curve for one second and then might have
turned into a sine wave, or started exponentially increasing, or
followed a curve like a*t - b*t^2/6 + 145*cos(t/10) + 11. In short,
there is simply no way to limit the possible behaviors of the voltage on
an _a priori_ basis (other than by using the same theory we are
testing). The domain of "possible" behaviors is infinite, and quite
likely multiply infinite, so the _a priori_ probability that any
particular time-course of behavior will be observed is exactly zero.

When we are dealing with a quantitative theory like PCT, which is used
(where possible) to predict detailed behavior through time, the
falsification problem is turned upside down. In a tracking experiment,
we observe that a handle moves in a highly irregular pattern for a
period of one minute. The theory says (approximately) that this pattern
is quantitatively equal and opposite to the pattern of a disturbance
that we apply to the target, the cursor, or both. When we run a
theoretical model of this behavior (as a computer simulation), we get
very nearly the same irregular pattern of behavior, so a plot of the
subject's handle positions against time lies on top of a plot of the
model's handle position against time.

In terms of probabilities, the chances that a quantitative model comes
even within 10% of the real behavior are still zero, because there are
no physical constraints on the subject; the subject could have moved the
handle in any pattern whatsoever, including not moving it at all (which
some subjects have been known to do!). Even if we eliminate all
physically-impossible movements, we are still left with multiple
infinities of possible movement patterns, so we simply can't calculate
the probability that the subject's behavior could have matched the model
within 10% "by chance" even on a single experimental run.

We do see deviations of the model's behavior from that of the
experimental subject. But these deviations are small in comparison with
the range of handle movements -- perhaps 10% RMS. So now the question
becomes, "Are these deviations from the theory statistically
significant?"

In order to say that the theory is incorrect in some regard, we would
have to show that the deviations are systematic, and not just the result
of random noise in the system. If the deviations are random, then we
could improve the model by adding a random noise generator to it, but
that would make our predictions of the _mean_ behavior worse. Our theory
concerns the systematic component of the behavior.

Now falsifying the theory would require that we find a systematic
deviation of the actual behavior from the predicted behavior. Even if we
did find such a deviation, however, this would imply only that the model
needs some small modification to eliminate that deviation. The model,
being quantitative and involving adjustable parameters, can't be judged
in terms of "true" or "false." It can be judged only in terms of
"better" or "worse", with "perfect" being easily definable: no
detectable deviation of prediction from observation.

And in the background, we must keep in mind that the chances of the
actual behavior pattern coming even within 10% of the theoretical
pattern _by chance_ are zero.

This is very different from a theory constructed as a logical
proposition. The only detectable deviation from a logical proposition is
the difference between "true" and "false." Such a theory cannot deviate
in the slightest from being true without being falsified. Wherever you
set the threshold of probability (such as p < 0.05), the only choice you
have is to say that the theory is true or it is false. You can't say
it's "partly true." If you say "the mean behavior rate was 10 presses
per minute," the only way in which this statement can vary _logically_
is to add a "not" -- "the mean behavior rate is NOT 10 presses per
minute." So if the mean behavior rate is 9.99999 presses per minute, the
statement is false.

We can judge the support given by the data to a theory on a continuous
scale. If the data deviate by a large amount from the theory, the
support is weak and the theory probably has something seriously wrong
with it. But if the data match the theory within some small amount, we
can say that the support for the theory is very good, although the
theory could still be better. It's pretty obvious when a theory predicts
the wrong _form_ of behavior -- when theory says that the decay of the
capacitor's voltage should be linear, and it is actually a decelerating
curve.

If a theory of behavior says that a disturbance should lead to a greater
net disturbance, whereas the real behavior shows that it leads to a
lessening of the net disturbance, the theory is wrong; it predicts the
wrong _form_ of behavior. But if the theory predicts correctly, we know
only that the theory as expressed belongs to a class of equivalent
theories that would all predict the same behavior. The theory is right
in that it is a member of the right class. But since there are other
theories that would make the same quantitative prediction, we do not
know whether we have the right member of this class. In a control-system
model, we express the model in as simple a form as possible. But there
are other forms: for example, the perceptual signal might be multiplied
by ten in the input function and, when it enters the comparator, be
divided by ten. We would be unable to see any behavioral difference
between this model and the simpler one.
------------------------------
One last comment. You seem to imply that any theory that can't be
falsified must be treated as true. What I say is quite different: that
any theory that can't be TESTED must be left as undecided. When a theory
rests on premises that can't be tested, there is no way either to
falsify or (in the case of quantitative theories) to verify the theory.
So the theory, as our British friends say, is a non-starter. It's only a
possibility among other equal possibilities, including the possibility
of a theory that exactly contradicts the proposed one.

     I hope that there are no anthropologist, or paleontologist on this
     list.

If such people were on this list, do you think I should keep quiet about
this subject?
-----------------------------------------------------------------------
Best to all,

Bill P.