[From Bruce Abbott (960127.1525 EST)]
Bill Powers (960127.0600 MST) --
The basic problem I have with ideas like "reinforcements" and
"incentives" is that they assume effects of physical objects and events
on organisms for which there is no physical or neurological
justification. How, for example, does a piece of food act on an
organism? It has physiological effects when it is digested and broken
down into its chemical constituents, and it has neurological effects
when it interacts with sensory nerves such as those of touch, taste, and
smell. But many different objects have similar effects; they are also
broken down chemically into similar constituents, and they can excite
the very same sensory nerves -- but they are neither incentives nor
reinforcers.
But Bill, this naive conception of the reinforcer is not the way it is
generally viewed within EAB. What is missing is the whole process of
evolution by which the organism was constructed. To illustrate, certain
physical events must occur if the organism is to survive, among them the
periodic ingestion, in the right quantities, of appropriate nutrients. The
word "appropriate" is important here because different organisms have
different abilities to digest, absorb, and metabolize whatever ends up in
their digestive systems. Organisms that ingest the right things tend to
survive, those that don't, don't. Now, how is the organism to know what to
eat? For many organism, sugars are "good" whereas alkaline substances are
often "bad" (toxic). Over the course of evolution, sensors have evolved
that are capable of detecting sugars; these produce the qualitative
sensation we call "sweet." Other sensors have evolved that detect
alkalines; these produce the sensation we call "bitter." In an environment
in which sugars and toxic alkalines are common, an organism that comes into
the world with a strong liking for sweet and a strong disliking for bitter
has a selective advantage over others that fail to discriminate or even
worse, dislike experiencing sweet tastes and enjoy the bitter. We could
expect the population to be dominated by organisms whose systems are
prewired to accept the sweet and reject the bitter.
One can also imagine how such a relationship could come into being via
experience. An organism with no particular preference for or against sweet
or bitter might sample some of the things it finds in its environment. The
taste of bitterness followed in fairly short order by illness might lead to
the future rejection of substances with a bitter taste. But this case is
really not so different from the first one (innate dislike of bitter),
because in both cases the bitter taste was associated with a unpleasant
perceptual experience, either directly, or indirectly via poisoning. The
organism is equipped with innate mechanisms through which certain sensory
experiences are assigned a position along a dimension of
pleasant/unpleasant. That experience may be modulated by certain
physiological conditions of the organism; for example, sweet tastes may be
experienced as more pleasurable when the organism is in need of nourishment
than when it is stuffed. Indeed, under the latter condition a sweet taste
may be experienced as mildly unpleasant.
What I have briefly sketched here is a theoretical basis for the notion that
some sensory qualities are under certain conditions experienced as pleasant
and will be sought, whereas others will be experienced as unpleasant and
will be rejected and avoided. Things that produce the former sensory
quality serve as reinforcers; things that produce the latter sensory quality
serve as punishers. The rest are neutral.
This answers your question, "How does a piece of food act on an organism?".
It is not the sensory quality of the food per se, but the relationship
between that sensory quality and the perceptual/evaluative signal it
generates as a function of innate neurological wiring and the current
physiological state of the organism. It is not a property of the food alone.
Interestingly, this scheme easily can be incorporated within a
control-system model. What does it mean to say that the organism likes the
sensory experience? It means that the organism has a positive reference for
that sensory signal and will attempt to bring the signal up to that
reference. What does it mean to say that the organism dislikes the sensory
experience? It means that the organism has a zero reference for that
sensory signal and will attempt to bring the signal down to that level
(escape) and keep it there (avoidance). Neutral signals are not controlled.
Where this model differs a bit from the usual control model is the
assumption that some sensory input not only enters a control system in the
usual way, but also enters an "evaluative" function whose output determines
the reference level for the signal--whether the organism will seek to
continue or increase the signal or to reduce the signal to zero.
What this tells me is that reinforcingness or incentiveness does not
reside in the object. The only thing that makes an incentive an
incentive is the way the organism handles the sensory signals and the
chemical products. At the interface between organism and environment,
inputs are just inputs, having no special characteristics of their own
beyond their ordinary chemical and physical properties. All that a
sensory input can do is create a neural signal roughly proportional to
the degree of stimulation. When that has happened, everything that
follows is a consequence of the way the organism is organized inside.
Most EABers (when they think about such things) probably see it in much the
same way. This is why I get a bit annoyed when you repeat such phrases as
"the food doesn't know . . .", as if reinforcement theorists implicitly
assume that it does. They don't.
How does Killeen propose mechanisms? He starts out by giving incentives,
as physical objects, special properties that make them "prime movers" of
behavior.
Yes, and, I suspect, for reasons not far different from those I outlined
above. As such they are not properties of the objects themselves per se,
but of the effect of certain of those properties on systems within the organism.
Then he proposes that these special objects can selectively
but temporarily give value to memories of responses. But what is the
justification for supposing that neural signals representing one kind of
object will have an effect on specific memories that the same neural
signals, in other combinations, will not have?
See above. The problem is that Killeen, like most other reinforcement
theorists, chooses to simplify the exposition by just asserting that
incentives have these properties, without bothering to explain how. But
I'll bet if I ask him about it, Killeen will offer an explanation close to
if not identical with the one I sketched in above. I think I _will_ ask
him; perhaps I will be surprised by his answer.
There is no independent
evidence for the existence of the supposed memories, and there is no way
to measure the effect of the sensory signals on them, so we can't bypass
fundamental considerations and say "It just happens that way" as we do
with other phenomena like gravitation. There is no way to prove that a
piece of food has an effect on memories of responses that a piece of
gravel does not have. There is no way to prove that those memories even
exist.
You may be right, but I'm not as confident as you that there is no way to
prove whether or not such memories exist. If these memories do exist, they
should have consequences that can be assessed, just as, say, electrons do.
But I agree that an inability to empirically test for such memories would
weaken Killeen's theory.
There is, however, a mechanistic way to show how a memory of a response,
if it did exist, could lead to production of the same response. Bruce, I
think, is proposing this method: the memory is of a _perception_ of a
response, and it is used as a reference signal for the same control
system that originally produced the perception. So if we grant the
existence of a signal representing a past perception of a response, we
can explain in mechanistic terms how that memory can result in actions
-- perhaps different actions -- that produce the same perceived response
that was perceived in the past.
I think people can do a good job of describing their actions of a moment ago
fairly accurately; that seems to provide good evidence that actions leave
their own perceptual record, which can be retrieved from memory for a brief
period of time after their occurrence. I'd bet that a rat or pigeon can
access the same kind of perceptual record.
The basic constraint on our modeling is the mechanistic fact that at the
sensory interface, all sensory signals are alike. They are trains of
neural impulses. They don't have different colors or flavors to tell the
brain what they represent or what they imply. No one sensory input is
any more important than any other. It is the brain that has to find
order in these input signals, and that by acting on the outside world
has to impose order on them. If we find that some inputs seem to have
special importance, it is the brain that has given them this importance.
The environment only proposes; the brain disposes.
Given my discussion above on evolution, I would have to disagree in part
with the notion that all sensory signals are alike. It is true that all
sensory signals are trains of neural impulses, but the significance of the
impulses depends on what system they enter. If sweet and bitter do not
taste the same to me, and if my responses to them are very different
(controlling for higher levels of one and lower levels of the other, for
example), it is because the neural impulses from the two sets of receptors
arrive at different places in the brain and contribute differently to
different functions (e.g., an evaluative function may receive excitatory
impulses from sweet-receptor neurons and inhibitory ones from
bitter-receptor neurons). The current wiring may reflect innately
determined connections or may reflect alterations to those connections
produced in the course of learning. The brain may have to "find" the
significance of some signals; others may come with their "significance"
already wired in during the long process of evolution.
Regards,
Bruce