[From Bruce Abbott (960126.2040 EST)]
Bill Powers (960126.0500 MST) --
Killeen proposes that memories of responses which have occurred in
association with incentives are activated by the incentives and that
these memories of past responses somehow cause similar responses to
occur for some specific length of time proportional to the incentive. As
far as I can see, he proposes no "mechanism" at all. Everything he
proposes -- association, activation, conversion of memories into a
period of responding -- occurs by magic. He never mentions sensory
nerves, the CNS, muscles, and so on -- the actual mechanisms of
behavior.
Yes, you are right; I was using the term "mechanism" a bit loosely to refer
to the mathematical system Killeen proposes rather than any particular
physical substrate that might actually implement the proposed functions.
But you are right in saying that the incentive is not specific to any
particular behavior. Any behavior occurring at the time the incentive is
delivered tends to be repeated. Nothing is said that requires this to be
a feedback loop. If you pair incentives with the emission of any
specific behavior, eventually that incentive will come to "elicit" ("to
draw or bring out or forth") that behavior. Whether it does so through
the intermediary of short-term memory or in some other way is irrelevant
to the structure of the overall model. The operative part of the theory
is the effect of incentives on responses through memory associations.
Once the association is established, one incentive, regardless of its
origin, can somehow cause _a_ seconds of the kind of responding that has
occurred in association with the same incentive in the past. So I see
nothing in Killeen's theory to preclude predicting that incentives will
draw forth behavior even when they are not being caused by behavior.
Yes. It really is a nice Ptolemaic theory of behavior, replete with all the
right cycles, epicycles, and eccentrics to fit the observations, and just as
fundamentally wrong. As I described to Rick, it can, I believe, even give a
reasonable account of the effect of "free" incentive I suppose, but it's worth
thinking about. At a more general level, I would look for differences in
dynamics; Killeen's model is, after all, only an equilibrium model and can
only move to equilibrium at a rate determined by the functions involved and
their starting values. Control models usually have _high gain_; they exert
whatever output is needed (up to the limits of their abilities) to oppose
error. Unless the gain in the control system is very low, there should be
readily measured differences in the dynamics of the two systems. Or at
least so it seems to me, without really having thought very deeply yet about
the problem.
If the rats are first given the opportunity to associate the sound
of the feeder with the appearance of a food pellet in the cup,
acquisition of lever-pressing can be quite rapid -- a matter of two
or three pairings of lever-press and food delivery. This time-
scale seems too short for the reorganizing system, in that
intrinsic variables would hardly have been affected by then.Right. I would doubt that reorganization is involved here. This sounds
more like a higher-level system that has been acquired through previous
reorganizations. We might call it a search system. What a search system
has to do is to apply some behavior to various parts of the environment
until a wanted result occurs, and then to stop the search process and
continue the behavior in that place until the controlled variable is
brought to its reference level (when possible). This is like a
systematic version of reorganization -- it is more specialized and thus
less powerful, but it is much faster.
Yes, that's the kind of thing I've been arguing for -- something more
specialized and systematic, a kind of heuristic strategy as opposed to blind
variation.
I suppose that the main reason for starting with an association of the
sound of the feeder with delivery of a food pellet is that there is a
delay in delivery of the food pellet. The sound occurs immediately upon
pressing of the lever, but it takes a while for the food to reach the
dish. The object, I suppose, is to prevent the search from continuing
during the delay, so the wrong action seems to produce the food
delivery.
Yes, it's the old "assignment of credit" problem again: what action should
be associated with the arrival of the food? The most recent? The most
consistently present?
It would be interesting to study this search behavior. I imagine an
array of food receptacles into which pellets can be placed silently and
without being seen. The rat has to move close enough to the receptacle
to see its bottom, and when it can see if food is there it can't see
into any other receptacle. We would expect to see the rat doing some
sort of search pattern, and stopping it whenever food was present,
eating the food and then resuming the search (as long as it was hungry).
If the pellets were placed at random, the search pattern wouldn't be
related to the pellet-appearance pattern, and would be whatever general
search pattern the rat uses in the absence of regularities. But if there
were a pattern in the appearance of pellets, how complex a pattern could
the rat learn? Could it learn to search in a simple clockwise or
counterclockwise pattern? Could it detect a repeating pattern of greater
complexity and adjust its search pattern to anticipate where the next
pellet would arrive?
I believe that something like this has been investigated, although I am not
"up" on the literature in this area. One example is research using the
Olton radial arm maze, which consists of a set of alleyways radiating out
from a common center like spokes of a wheel. A curtain or other obstruction
at the end of each arm blocks the rat's view of what's behind. One finding
is that if you bait all the arms, the rat will visit each arm seemingly at
random, but will not revisit any arm it has just investigated. There are
all sorts of variations on this theme, but I'm not familiar with what has
been learned from them.
This sort of experiment could easily be done with humans using a mouse
to check various locations on a screen to see if they will produce some
wanted result. We could evolve a series of patterns of increasing
complexity, and directly compare a human's and a rat's ability to match
its search pattern to a complex pattern.
Sounds like an interesting study. One thing I do know, rats are much better
at learning spatial discriminations than other kinds.
Got to go -- thanks for the synopsis of reorganization theory. Re:
awareness. As I noted to Rick, I agree that (conscious) awareness is not a
necessary condition for control. If it were, I'd never make it home from work!
Regards,
Bruce