[From Bill Powers (921223.1530)]
Wayne Hershberger (921223.0930)--
I am delighted that you're delighted with my remarks on
conceptual and perceptual EVs (environmental variables). If you
could ever bear to look back over our several years of
discussions on these matters, you might now see what I was trying
to say. Thank goodness I found a way that expresses my idea
(assuming that my present idea is really the same one I started
with!).
···
------------------------------------------------
The VOR is not perfect; nor is the optokinetic reflex. I think
the gain (eye-velocity/head-velocity) of these two reflexes
combined is only about .9, meaning that during active head
rotation of 50 deg/s, images slip across the retina at 5 deg/s.
OOPS. My abysmal ignorance catches me up once again. Please
explain the difference between these two reflexes!
A slip of 5 degrees per second per 50 degrees per second of
movement implies about a 10% error at the end of a movement,
doesn't it? This is pure coincidence, but the rule of thumb I've
been using to specify a "good" control system is a loop gain of
"5 to 10 and preferably greater." A control system with a loop
gain of 10 would allow a disturbance to have 10% of the effect it
would have without control. Splendid.
Note that when an object in the visual field moves 90 degrees and
you recenter it within 1 degree of the center of vision, the
implied loop gain is 90. If you recenter it within the limits of
optical acuity (say, 2 min of arc) the implied loop gain is 2700!
So the combined reflex is only 1/9 to 1/270th as accurate as the
visual centering control system.
-----------------------------------------------------------
Martin Taylor (921223.1440) --
I agree that it would be a good idea to give information theory a
rest until you can work up your paper on it.
To me, a model is only a concise way of making descriptions. A
generative model, as you put it, is distinct from a descriptive
theory. To me, it is not distinct. It only uses a few more
parameters to make a much better description, and if that
description is accurate, Occam's razor says that it is
preferred. But if its precision allows the world to say
"that's not right," it could be that the parameters are wrong.
If the parameters can be changed and still lie within the
bounds of "the theory", then the information contained in the
parameter specification has to be included in the "size" of the
description, so the case becomes less clear.
I think there are really two kinds of models. Consider the model
contained in a schematic diagram of a radio, plus the theory of
electronics that applies to the symbols in the schematic. Basically this
schematic is a specification for interconnecting
physical components having certain properties. In one part of the
schematic there will be several tuned circuits consisting of an
inductance, a capacitance, and some series or parallel
resistance. Depending on the exact values in henries, farads, and
ohms, the combined circuit will have a certain frequency response
in terms of amplitude versus frequency. The shape of the pass-
band measured this way is determined by the physical properties
of the components, and nothing else (assuming no important
loading by other circuits).
It follows that all abstract properties of the tuned circuits,
such as their combined "Q", bandwidth, rise and fall time for a
step input, and gain, are also determined by the properties of
its physical components.
It is possible to describe the behavior of this part of the radio
without reference to the physical components that comprise it.
One could, for example, determine the bandwidth, rise-time, Q, or
gain by observing how the circuit's output relates to its inputs.
This would lead to a descriptive model of the circuit, cast not
in terms of interactions among components, but in terms of
behavioral measurements.
Mathematical relationships among the measurements thus found
could be the basis for still greater generalization, for example
input-output power spectra or various kinds of transforms:
Fourier, Laplace, or z. And one could go to still greater degrees
of abstraction and express the input-output relationship in terms
of equivalent sampling frequencies, bit transfer rates,
information capacity, and so forth.
This whole genre of representation is what I mean by descriptive
models. They are models drawn from descriptions of whole-system
behavior, either with or without experimental input to the system
as a whole. They are all attempts to find simple invariants of
behavior -- simple, that is, in comparison with the potential
complexity of behavior of which the system is capable.
A generative model goes in the other direction from observations
of behavior. In effect, it is an attempt to draw the schematic
diagram of the system. It treats behavior as the outcome of more
detailed processes, as a consequence of the interactions among
components, no one component showing the behavior of the whole
system, but the whole-system behavior being the necessary outcome
of the properties of the components and their interactions. I
guess the latest buzz-word for this is "emergence." From the
standpoint of generative modeling, the behavior of the model, and
presumably of the real system being modeled in this way, is an
emergent phenomenon. In a generative model of a control system,
there is no component that controls. Control is an emergent
phenomenon.
A generative model is created by a feedback process. A model is
constructed and made to behave. Its behavior is perceived in relation to the
behavior of the real system, and the difference
is noted. On the basis of the difference, the construction of the
model is modified in a way that reduces the difference. The aim
is to construct a model that produces outputs like those of the
real system when presented with any possible inputs.
A descriptive model is generated by a process of induction, which
is also a feedback process but operating at a different level. A
generalization is proposed. The behavior of the real system is
observed, and its fit with the generalization is noted. If
exceptions are found, the generalization is changed until all
cases of real behavior are covered by it. In psychology (and
other fields), the generalizations are stated in statistical
terms, and the measures of behavior are also subject to
statistical representation. As a result, detailed deviations of
the observed behavior from the generalization are averaged out,
and only means, trends, and the like are compared. Therefore the
process of generalization can arrive at an end-point even through
individual instances of observed behavior depart markedly from
the general representation of it.
A basic difference between these kinds of models is the degree to
which imagination plays a part. A generative model begins as pure
imagination. One imagines components which, if they really
existed and really interacted as imagined, would produce behavior
like the real system's behavior. A person making a generative
model of the pass-band filter in a radio might imagine that there
are four successive tuned circuits with a certain 'Q', or that
there is a digital computer that creates an equivalent frequency
response. The model would consist of imagined coils, capacitors,
and resistors, or of a minimal microcomputer running a specific
program. The model would be given a simulated input waveform
similar to the waveform entering the real system, and its
operation would then produce an output waveform for comparison
with the output waveform of the real system. The differences in
output waveform would be minimized by adjusting the variables in
the model. If the resulting fit were within observational error
for all possible input waveforms, the model would be accepted.
Its components would be treated as real, and their values that
produce the best fit with real behavior would given as the values
of the critical variables. It is perfectly possible that the
analog model works just like the digital one; in that case both
have to be retained as viable alternatives.
When, as often happens with generative models, the real system
becomes amenable to dissection, a further refinement of the model
becomes possible. If the system, opened up, proves to contain
nothing resembling a digital computer, and many components that
show continuous input-output properties, then the version of the
model with coils, capacitors, and resistors is chosen. Such
detailed examination can show where the model is wrong: there
might be, for example, five stages of filtering instead of four,
and the assumed capacitances and inductances might prove to be
generated in part or totally by active local feedback through
amplifiers.
In short, even if the imagined model proves to be correct in the
large, it is unlikely to be correct in detail. It can, however,
easily be modified to become correct in detail, as far as the
dissection allows. Detailed enough dissection might show that the
coils, capacitors, and resistors of the model must be replaced
with other physical components that have equivalent properties.
But the development is always in the direction of more detail and
more precision.
A descriptive model does not enter this world of imagined
components and interconnections. It cannot, because its lowest
level of abstraction is the observed behavior of the whole
system, and it uses no imaginary components. It is, if you will,
strictly empirical. It can lead to more and more compact
descriptions in terms of broader and broader concepts, but at its
base is always behavior itself. There is no possibility of
arriving at greater and greater detail of explanation; in fact
the trend is always toward less and less detail.
For me, the choice between generative and descriptive models is
the choice between ever-more-precise prediction of behavior, and
ever-more general characterization of it. It is the choice
between understanding how the system works and making true but
non-predictive statements about the system's behavior. My choice
is the generative model; I simply find it more satisfying than
the other kind.
--------------------------------------------------
As to your work with Little Baby and Genetic Algorithms -- I
definitely count that as a step toward making a generative model.
How's your program for comparison of the results with real human
behavior coming along?
--------------------------------------------------------------
Best to all,
Bill P.