Dear CSG,
On Friday over at MIT I saw an interesting talk by Wolfram Shultz on the
dopamine system
and the anticipation of reward.
The abstract below gets the 0th order gist of the talk.
What the abstract leaves out, however, is that there are ensembles of
neurons that fire precisely WHEN a reward signal is expected
(i.e. the system is not only learning to expect that a reward will
arrive or not arrive, but also the precise timing of the reward).
If there is a mismatch between the timing of the expectation and
the reward, the activity of the dopamine neuron is suppressed at
the time (when a reward would be expected but is not delivered).
I think in PCT terms, this means that the system retains the time
structure of events in the reference signal that is being compared
with perceptually-generated (internal) reward signals.
It may also mean that associations are not simply conjunctions of
stimulus and reward that are in some rough temporal proximity,
but that the timings of stimulus and reward are an integral part of memory
traces, their readout mechanisms, and their comparison operations.
For a review of temporal structure in conditioning :
Miller R. R., Barnet R. C., 1993, "The role of time in elementary associations".
Current Directions in Psychological Science 2(4), 106-111.
Schultz referred to the work of Barto on temporal credit assignment and
also that of Gallistel, but I'm not entirely sure that these models retain
the full temporal structure of events.....
Since I get the CSG list in digest form, if you reply to the group regarding
this
topic, I'd much appreciate it if you could cc: a copy to me directly
at cariani@mac.com (thanks).
Take care everyone,
Peter Cariani
email cariani@mac.com
web http://www.cariani.com
Wolfram Schultz
Professor and Chair of Neurophysiology
Institute of Physiology and Program of Neuroscience
University of Fribourg
http://www.unifr.ch/neuro/schultz/
"Neuronal Activity in Primate Basal Ganglia and Frontal Cortex: from
Rewards to Goals of Behavior"
ABSTRACT:
Results from lesioning and psychopharmacological studies suggest that the
basal ganglia and frontal cortex are involved in the processing of reward
information and the use of this information for goal-directed behavior. In
our studies on monkeys we found that dopamine neurons respond primarily to
reward-related stimuli. The reward responses are compatible with the notion
of a teaching signal for reinforcement learning, transmitting a message on
how much a reward-related event occurs differently than predicted. Neurons
in orbitofrontal cortex differentiate between different food and fluid
rewards on the basis of their motivational value, as assessed by the reward
preferences of the subject. Neurons in the striatum are active during the
expectation of reward and integrate reward information into activity
related to the behavior leading to the reward. These data suggest that a
number of brain centers process differentially several aspects of reward
information for the control of goal-directed behavior.