Loop gain moduation in Aplysia

[From Bruce Abbott
(2009.05.09.1610 EDT)]

Eric Kandel and his
colleagues used the sea hare, Aplysia (a marine slug) to study the mechanisms
involved in habituation of the animal’s gill-withdrawal reflex. Squirting
a jet of water over Aplysia’s siphon or mantel area causes muscles in the
mantle area to contract, thus covering the delicate gills. If there is no
further stimulation, these muscles soon relax, uncovering the gills.
However, repeated applications of the jet soon lead to habituation of the
reflex: Aplysia “ignores” the sensation and the mantle does not
constrict.

The gill-withdrawal reflex is
adaptive in that it helps to protect Aplysia’s gills from being damaged
by predators. At the same time, Aplysia could not survive for long if it kept
its gills retracted too much of the time. However, objects in Aplysia’s
environment that pose no threat to Aplysia may repeatedly stimulate the mantle
or siphon. For example, wave action could cause fronds of seaweed to wipe
repeatedly across the mantle surface. Contraction of the mantle in this case is
unnecessary and worse, it reduces Aplysia’s ability to absorb oxygen and increases
the consumption energy. Habituation to such stimulation, when not accompanied
by activity in pain receptors indicating damage, is adaptive. But how does
habituation take place?

Neuroanatomical studies have
isolated the basic neural circuit involved in Aplysia’s gill-withdrawal
reflex. This circuit begins with sensory neurons in the mantle surface and
siphon. These generate neural impulses when these surfaces are disturbed by the
water jet or by touching. The sensory neurons have axon branches that synapse
directly on motor neurons and on interneurons whose terminals also synapse with
those same motor neurons. The synapses involved are all excitatory. Stimulation
of the sensory neurons thus results in stimulation of the motor neurons, which
in turn connect to the muscles that contract the mantle. (The system looks like
a simple S-R mechanism until one realizes that contraction of the mantle, by
covering the mantle and siphon surfaces, removes the source of stimulation.
Thus, it’s actually a closed-loop, negative feedback system that acts to
bring the sensation signals to zero whenever they are brought far enough above
zero to trigger contraction.)

Habituation of this reflex
could be accomplished via at least three means: (1) adaptation of the sensory
neurons. These neurons would then become insensitive to touch and would not
produce neural impulses in response. (2) Motor fatigue. The mantle muscles
might become fatigued because of rapid, repeated stimulation, so that the
muscles would become incapable of contracting. (3) The sensory neurons might
remain responsive to touch and fire neural impulses, but these impulses could
become ineffective in causing release of neurotransmitter at the synapses
between the sensory neurons and their target motor neurons and interneurons.
What the research has shown is that habituation is accomplished by the third
means, at least in the case of what is termed “short-term”
habituation. In short-term habituation, stimulation is repeated at relatively
short intervals and habituation gradually develops. Then, after a relatively
short period of non-stimulation, habituation disappears.

From a control-system
perspective, it seems to me that habituation of the gill-withdrawal reflex
reflects a temporary reduction in the output gain of the system. As habituation
proceeds, it takes stronger stimulation of the sensory neurons to produce a
given amount of output from the motor neurons, and thus a given degree of
mantle contraction. Other system gains appear to remain unaffected, so loop
gain decreases in proportion to the output gain. If this analysis is correct,
then habituation of Aplysia’s gill-withdrawal reflex (and perhaps
habituation of many other types of reflex action) provides a clear example of a
targeted “reorganization” involving an alteration of system gain
rather than a reorganization of system organization.

Bruce A.

[From Bill Powers (2009.05.09.1419 MDT)]

Bruce Abbott (2009.05.09.1610 EDT) --

BP: Very nice analysis, Bruce. I'd always meant to sit down with an Aplysia circuit diagram and figure that out, but you beat me to it.

BA: Eric Kandel and his colleagues used the sea hare, Aplysia (a marine slug) to study the mechanisms involved in habituation of the animal's gill-withdrawal reflex. Squirting a jet of water over Aplysia's siphon or mantel area causes muscles in the mantle area to contract, thus covering the delicate gills. If there is no further stimulation, these muscles soon relax, uncovering the gills. However, repeated applications of the jet soon lead to habituation of the reflex: Aplysia "ignores" the sensation and the mantle does not constrict.

...

BA: From a control-system perspective, it seems to me that habituation of the gill-withdrawal reflex reflects a temporary reduction in the output gain of the system. As habituation proceeds, it takes stronger stimulation of the sensory neurons to produce a given amount of output from the motor neurons, and thus a given degree of mantle contraction.

BP: It's also possible that the input threshold or the reference level changes. This appears to be a one-way control system; only perceptual signals above the threshold produce an error signal, and the gain is high enough to make the output simply turn on and off (though I'd like to experiment with small changes in stimulation to see what the transition really looks like). I don't suppose there is an actual neural reference signal (it would be inhibitory), but anything biochemical that biases the zero-point will serve as one.

Other system gains appear to remain unaffected, so loop gain decreases in proportion to the output gain.

How's that? If the system gains have been measured, that would probably settle the question of whether an effective change of reference signal is involved. That would not alter the loop gain if the comparator is approximately linear.

If this analysis is correct, then habituation of Aplysia's gill-withdrawal reflex (and perhaps habituation of many other types of reflex action) provides a clear example of a targeted "reorganization" involving an alteration of system gain rather than a reorganization of system organization.

Yes, nice. That would be like changing the weights as in ArmReorg if it's really gain that decreases. But in this simple a creature, with no blood-brain barrier to make things neater, there might be little difference between neural functions and neurochemical functions. They're all neurochemical in the final analysis, anyway. If we knew how the threshold is established, we could measure it or whatever it depends on and probably see it change. By "we" I mean one of those incredibly smart microbiochemists.

Best,

Bill P.

[From Dick Robertson, 2009.05.09.1640CDT]

[From Bill Powers (2009.05.09.1419 MDT)]

Bruce Abbott (2009.05.09.1610 EDT) –

BP: Very nice analysis, Bruce. I’d always meant to sit down with
an Aplysia circuit diagram and figure that out, but you beat me to it.

There is a copy of Kandel’s original circuit diagram on page. 114, of IMP. As I see it it shows a two level control system that suggests the upper level controls the sensitivity of the lower level. The mechanism, I suggest, would be by altering the reference signal (ie. changing the sensitivity) the first order system

BA: From a control-system perspective, it seems to me that
habituation of the gill-withdrawal reflex reflects a temporary
reduction in the output gain of the system. As habituation
proceeds, it takes stronger stimulation of the sensory neurons to produce
a given amount of output from the motor neurons, and thus a given
degree of mantle contraction.

BP: It’s also possible that the input threshold or the reference
level changes.

Yes, that is consistent with the circuitry, isn’t it?

Best,

Dick R.

[From Bruce Abbott (2009.05.10.0915
EDT)]

Dick Robertson, 2009.05.09.1640CDT

image00111.gif

···


DR: There is a copy of Kandel’s original circuit diagram on page. 114, of IMP.
As I see it it shows a two level control system that suggests the upper level
controls the sensitivity of the lower level. The mechanism, I suggest, would be
by altering the reference signal (ie. changing the sensitivity) the first order
system

I don’t have a copy of Introduction to Modern Psychology. Below is the
circuit I referred to, which includes only that portion of the mechanism
involved in short-term habituation. A slightly more elaborate diagram includes
input from the tail from neurons that respond to a shock that is administered to
the tail and produces sensitization (or dishabituation) of the gill-withdrawal
reflex. The axons of these neurons synapse on the terminal buttons of
the sensory neurons shown in the diagram below. This input immediately reverses
the habituation.

I’m wondering if this
slightly more elaborated diagram is the one found in IMP. If so, then the
interpretation as a two-level system with the upper level controlling
sensitivity in the lower-level system doesn’t quite fit, as I see it. Habituation
(reduced sensitivity) is brought about by repeated stimulation of the sensory
neurons and is thus carried out exclusively with in the lower-level system, not
as a function of a reference input from above. Dishabituation is brought about
by stimulation of sensory neurons in the tail (in this experiment). This does
bring in another input, but unless I’m not analyzing it correctly, this
input doesn’t appear to function as a reference signal.

Aplysia gill-withdrawal circuit.gif

More recent research has
investigated the basis of long-term habituation, which results when repeated
bouts of stimulation produce habituation that can last for hours or even days. Unlike
short-term habituation, long-term habituation appears to require protein synthesis
and some kind of retrograde signal (i.e., feedback) from the motor neurons back
to the sensory neurons in the mantle and siphon.

Bruce A.

[From Bill Powers (2009.05.11.1002 MDT)]

My oldest grandson Derek is 20 years old today. Guess how old that makes
me feel.

Bruce Abbott (2009.05.10.0915 EDT) –

BA: I don�t have a copy of
Introduction to Modern Psychology. Below is the circuit I referred to,
which includes only that portion of the mechanism involved in short-term
habituation.

BP: Unfortunately, a diagram like that doesn’t tell us what the circuit
does (as I’m sure you know). If we don’t know the functions performed by
each neuron, we can’t tell whether this is a differentiator, a two-level
control system, or any of a large number of other possibilities. Are
there any more connections that were just not drawn? Neurologists tend to
think of neural circuits as simply passing impulses along from one neuron
to the next, but if you consider a neural signal to be a
variable-frequency train of impulses, many other kinds of functions are
possible. For example in the diagram the input signal divides and takes
two different paths to get to the output neuron. If the interneuron path
is an integrator, and an open triangle indicates inhibition, and
inhibition is represented as subtraction, the signal from the
output becomes

output = input - integral(input)

which is roughly a time-differentiator. See B:CP chapter 2.

I was looking on the web for more detailed diagrams, but couldn’t find
any – and none of them contains any information about the transfer
functions at each synapse. Aplysia contains about 20,000 neurons, so
these little snippets don’t go very far toward giving us a picture of the
whole system

There is really no way to analyze the diagram without those details. It’s
like looking at a circuit diagram in which you can’t tell the transistors
from the resistors or the resistors from capacitors.

A slightly more elaborate
diagram includes input from the tail from neurons that respond to a shock
that is administered to the tail and produces sensitization (or
dishabituation) of the gill-withdrawal reflex.

How does the shock do that? Are its effects confined to the sensory
neurons in the tail? In the diagram the sensory neuron is shown coming
from the skin of the siphon, not from the tail. I don’t like shocks as
stimuli because their effects can spread all over the place, even far
from where the shock is administered. How about continuous tactile
stimuli?

The axons of these neurons
synapse on the terminal buttons of the sensory neurons shown in
the diagram below.

Do you mean the area where the vescicles containing neurotransmitters are
located? I’m not familiar with the terminology here.

This input immediately
reverses the habituation.

I wish we could find more descriptive terms that indicate what is
happening. “Habituation” is a description of a remotely-related
observation. Does the habituation signal change the amount of
neurotransmitter emitted per impulse or does it just add its own
excitatory or inhibitory neurotransmitter to whatever is being emitted
anyway?

When I started work at the VA research hospital in Chicago in 1950s, I
was full of ambitions. I was going to go to the medical library at the
hospital and start drawing and analyzing circuit diagrams. I very quickly
ran into the problems we have here: lack of vital information. Whoever
drew the diagrams of neural connections didn’t understand about transfer
functions and how important they are to understanding what the circuit
does. In those days all the neurologists did was chase single impulses
around – I’m not sure they do a lot more than that even now. It took me
about six months to give up that grand plan.

BA: Habituation (reduced
sensitivity) is brought about by repeated stimulation of the sensory
neurons and is thus carried out exclusively within the lower-level
system, not as a function of a reference input from above. Dishabituation
is brought about by stimulation of sensory neurons in the tail (in this
experiment). This does bring in another input, but unless I�m not
analyzing it correctly, this input doesn�t appear to function as a
reference signal.

BP: It’s possible that the upper neural path in the diagram has a time
constant measured in hours or even days – who knows, without any
information? And without all the other signals that aren’t shown, I think
we’re stuck.

BA: More recent research has
investigated the basis of long-term habituation, which results when
repeated bouts of stimulation produce habituation that can last for hours
or even days. Unlike short-term habituation, long-term habituation
appears to require protein synthesis and some kind of retrograde signal
(i.e., feedback) from the motor neurons back to the sensory neurons in
the mantle and siphon.

BP: Do you mean antidromic impulses going backward through the same nerve
fibers? Or are there feedback effects through the environment from the
gill action to sensory nerves in the siphon or elsewhere? Or perhaps
there are other neural paths not shown in this diagram?

I get that same frustrated feeling that I had back in the 1950s. If only
I could have been leaning over their shoulders when these researchers
were getting their ***** data. It seems to me that they hardly measured
anything.

Best,

Bill P.

[From Bill Powers (2009.05.15.0203 MDT)]

Bruce Abbott (2009.05.12.1140 EDT) –

BP: Unfortunately, a diagram
like that doesn’t tell us what the circuit does (as I’m sure you know).
If we don’t know the functions performed by each neuron, we can’t tell
whether this is a differentiator, a two-level control system, or any of a
large number of other possibilities. Are there any more connections that
were just not drawn?

BA: I’ve wondered about the role
of the interneuron in that diagram. My best guess is that it
prolongs or intensifies the contraction of the mantle muscles. (I haven’t
yet read enough of this literature to know what Kandel and colleagues
have determined its function to be within this circuit.) All the synapses
in the diagram are excitatory, by the way.

BP: The attached gives a little more information. I haven’t located the
Figure yet. It seems that the interneuron ONLY affects the amount of
neurotransmitter produced by impulses in the other axon – it does not
synapse directly with the motor neuron. So it appears to be purely a gain
control.

One thing is pretty clear: this is not a “learning” phenomenon.
Associating it with Hebbian learning is just a mistake, it seems to me.
This system always does the same thing; it doesn’t acquire any new way of
reacting to inputs. It habituates and dishabituates over and over, the
same way every time, without learning a thing.

Gently squirting water onto the mantle with a WaterPik resulted in
withdrawal of the gill and the siphon. Repeated squirts lead to less and
less response, and eventually none. The interpretation is that Aplysia
“learned to ‘ignore’ the stimulus… because it was not
harmful.” I doubt that statement rather thoroughly.

Apparently the normal case is for an occasional stimulation of the mantle
to cause withdrawal of the gill and siphon. This says that the gain is
being maintained at some fairly high level by activity in the interneuron
that raises the gain. Since that interneuron is activated by sensors in
the tail, there must be occasional stimulation of the tail under these
same normal circumstances. It is not unreasonable to assume that when
Aplysia is moving about, the tail is being stimulated.

When the mantle alone is stimulated, but not the tail, the gain begins to
decline gradually because there is no signal in the interneuron to
maintain the gain at a high level. This explains the habituation. If the
tail is then stimulated, the gain is increased and the habituation is
abolished: the normal response to a combined tail and mantle stimulation
returns. The gain control seems to respond rapidly to a brief signal from
the tail, raising the gain, with the gain dropping slowly after the
signal. So only a low rate of stimulation of the tail would be enough to
keep the gain normally high. Habituation can take place only when no
signals from the tail occur for a long time. And of course habituation is
not a phenomenon in itself; it’s just a name for declining gain.

I have no guess about what the controlled variable is - it’s probably
something that has to do with the circumstances under which both mantle
and tail are being stimulated occasionally. Apparently the tail sensors
have a rather high threshold – unless there’s another neuron affecting
signals coming from them.

BP earlier: I was looking on the
web for more detailed diagrams, but couldn’t find any – and none of them
contains any information about the transfer functions at each synapse.
Aplysia contains about 20,000 neurons, so these little snippets don’t go
very far toward giving us a picture of the whole system

The present article raises that number to 100,000.

Aplysia researchers have been
able to demonstrate what happens during habituation, both in terms of
changes in mantle contraction during repeated stimulation (typically once
every 10 seconds in demonstrations of short-term habituation) and in
terms of changes in neurotransmitter release, post-synaptic potentials,
and action potentials. Concerns about whether there might be other inputs
to this circuit that influence the time-course of habituation led to in
vitro experiments involving, in isolation, only the neurons shown in the
diagram I provided. Stimulation of the sensory neurons produced the same
pattern of changes found in the neurons of the intact animal. The
observed changes fully account for the changes in the gill-withdrawal
reflex during short-term habituation.

The problem here is not seeing the forest for the trees. In the Reiness
article, the description drops to the level of biochemistry and any Big
Picture is rapidly lost. At the same time, there’s that annoying habit
that biologists have of casually introducing purposive language at
entirely too high a level of abstraction, but with quotes so you know
they don’t really, really mean “ignore because it was not
harmful.” Of course they don’t believe that Aplysia can perceive an
abstraction like harmfulness, or that it can direct attention toward or
away from some stimulus, or that it has “reasons” for what it
does as implied by the word “because.” But if they don’t
believe that, why do they talk that way? Especially about an in
vitro
preparation?

Kandel and others have shown
that short-term habituation in this circuit depends exclusively on a
reduction in the amount of neurotransmitter being released with each
impulse of the sensory neuron. This reduction in turn results from
decreased Ca++ intake in the axon terminal during the action
potential. Ca++ intake is necessary for the transport of the
vesicles containing the neurotransmitter toward the synaptic gap. They’ve
also identified the cellular changes that reduce CA++ intake. All in all,
they make a convincing case for the mechanism of short-term habituation
as found in the gill-withdrawal reflex.

Right, but somehow they missed the fact that the normal case has to be
maintenance of the amount of neurotransmitter release at the higher end
of the range – otherwise there would be no response to habituate.

The shocks employed produce
localized effects in the tail. The reason shock is used is that it meets
the requirement of a stimulus that will produce sensitization. The input
must change suddenly and fairly strongly. Continuous tactile stimuli
wouldn’t work.

Like something starting to take a bite out of your tail. So the
controlled variable is something like a rate of change of stimulation of
the tail and mantle. Shocks, however, do not produce localized effects
unless they’ve changed the laws of electricity. A shock to your hand can
stop your heart. Electricity goes everywhere it can go, which is why I
don’t like shocks as a physiological stimulus.

When it come to neurologists
speculating about what some collection of neurons in the brain may be
doing, I share your frustration. In this case, however, researchers have
taken the approach of identifying a relatively simple system that
mediates a specific reflex and has been shown to demonstrate both
habituation and sensitization, taken careful measurements of neural
impulses, PSPs, changes taking place in calcium gates and vesicle
transport and so on, made observations on reduced preparations in vitro
and demonstrated that they reproduce those made on the intact in vivo
system, and so on. It seems to be precisely the sort of work you’ve been
looking for.

It is, but it’s masked by unnecessary complications and it stops short of
being useful – as usual. Either we get fanciful abstractions, or we’re
drowned in irrelevant (for purposes of understanding the circuit)
details. And when you finally start getting a handle on what’s happening,
it turns out that they missed the main fact – here, the fact that
something must have been maintaining the gain at a high level.

I wouldn’t get irritated if these biochemists didn’t come on as such
know-it-alls.

Best,

Bill

Memory in Aplysia.pdf (56.6 KB)

[From Rick Marken (2009.05.16.0900)]

Bruce Abbott (2009.05.16.0905 EDT)–

Yes, learning is a broad category, but it does have subcategories that have

been developed based on common experience and research: Habituation,

sensitization, classical conditioning, operant conditioning, sensory memory,

short-term or working memory, long-term memory (the latter subcategorized as

either declarative or procedural and having different physiological

substrates).

Surely you plan to explain in your text that these are all just different aspects of control. It would be particularly easy to do with classical and operant conditioning. That would be super!

Best

Rick

···


Richard S. Marken PhD
rsmarken@gmail.com

[From Bill Powers (2009.05.17.0910 MDT)]

Bruce Abbott (2009.05.16.0905 EDT) --

BP: Suppose that on your way home you encounter a road sign that says
"Detour" with an arrow pointing to one side. That is an experience, is it
not? And of course you turn and take the alternate route: that is a change
in behavior, isn't it?

BA: You turned at the Detour sign is because (1) previously you learned the
meaning of the Detour sign, and (2) you have a goal of reaching some
destination that probably cannot be reached by ignoring the Detour sign.
Turning at the Detour sign when you see it is not a change in behavior if
this is what you do every time. The learning took place earlier, when
experience taught you what the sign means, and what is likely to happen if
you ignore it.

BP:Yes, that's exactly my point. While experience doesn't "teach" you anything (experiences just happen; what you make of them depends on you, not them), you did alter your organization and acquired a simple control system that can follow the directions on a sign if you choose to do so. You acquired a control system, not a behavior or a specific reaction to a specific stimulus. Of course if you were born with that control system in working order, you would turn every time you happened to encounter a detour sign and we would not say that you learned that all over each time. We don't say you learned how to do a startle response even though occurrance of that response depends on what you experience.

With regard to Aplysia, the same principle applies. The properties of those neural nets are fixed; the gains rise and fall according to how the systems are used, and always in the same ways. We would say those properties are learned only if we observed the system first when those properties didn't exist, and then saw them come into being. And it would make a great deal of difference to know how those changes came to be. If the animal started to show responsiveness to tail-touches after a time, and we knew it had been anesthetized just previously, we would still not bestow the term "learning" on the change of behavior.

That's another way to assess whether you have learned, but I would say that
learning has also taken place if a person previously ignorant of the meaning
and implications of the sign now, after appropriate experiences, takes an
alternate route when encountering the sign.

I don't think that definition is airtight yet. "Post hoc ergo propter hoc" is still a logical error. "After which" is not the same as "because of which." This is why we need a scientific theory couched in a technical language. As we try to refine definitions, we are forced to seek ever-deeper understanding of the phenomenon. We are forced to propose a workable model because words alone can't express the relationships we suspect are working behind the scenes.

..........................

Just finished an hour's conversation with you via Skype and I've run dry for today.

Best,

Bill P.

[From Bruce Abbott (2009.05.12.1140
EDT)]

Bill Powers (2009.05.11.1002 MDT)

···

My oldest grandson Derek is 20 years old today. Guess how old that makes me
feel.

Well, look on the bright side: compared
to Methuselah, you’re still practically an infant! Oh, could you pass me
that pacifier?

Bruce Abbott (2009.05.10.0915 EDT)

BA: I don’t have a copy of Introduction to Modern
Psychology. Below is the circuit I referred to, which includes only that
portion of the mechanism involved in short-term habituation.

BP: Unfortunately, a diagram like that doesn’t tell us what the circuit does
(as I’m sure you know). If we don’t know the functions performed by each
neuron, we can’t tell whether this is a differentiator, a two-level control
system, or any of a large number of other possibilities. Are there any more
connections that were just not drawn? Neurologists tend to think of neural
circuits as simply passing impulses along from one neuron to the next, but if
you consider a neural signal to be a variable-frequency train of impulses, many
other kinds of functions are possible. For example in the diagram the input
signal divides and takes two different paths to get to the output neuron. If
the interneuron path is an integrator, and an open triangle indicates
inhibition, and inhibition is represented as subtraction, the signal from
the output becomes
output = input - integral(input)
which is roughly a time-differentiator. See B:CP chapter 2.

I’ve wondered about the
role of the interneuron in that diagram. My best guess is that it
prolongs or intensifies the contraction of the mantle muscles. (I haven’t
yet read enough of this literature to know what Kandel and colleagues have
determined its function to be within this circuit.) All the synapses in the
diagram are excitatory, by the way.

BP: I was looking on the web for more
detailed diagrams, but couldn’t find any – and none of them contains any
information about the transfer functions at each synapse. Aplysia contains
about 20,000 neurons, so these little snippets don’t go very far toward giving
us a picture of the whole system

BP: There is really no way to analyze the
diagram without those details. It’s like looking at a circuit diagram in which
you can’t tell the transistors from the resistors or the resistors from
capacitors.

Aplysia researchers have been
able to demonstrate what happens during habituation, both in terms of changes
in mantle contraction during repeated stimulation (typically once every 10
seconds in demonstrations of short-term habituation) and in terms of changes in
neurotransmitter release, post-synaptic potentials, and action potentials.
Concerns about whether there might be other inputs to this circuit that
influence the time-course of habituation led to in vitro experiments involving,
in isolation, only the neurons shown in the diagram I provided. Stimulation of
the sensory neurons produced the same pattern of changes found in the neurons
of the intact animal. The observed changes fully account for the changes in the
gill-withdrawal reflex during short-term habituation.

Kandel and others have shown
that short-term habituation in this circuit depends exclusively on a reduction
in the amount of neurotransmitter being released with each impulse of the
sensory neuron. This reduction in turn results from decreased Ca++ intake in
the axon terminal during the action potential. Ca++ intake is necessary
for the transport of the vesicles containing the neurotransmitter toward the
synaptic gap. They’ve also identified the cellular changes that reduce
CA++ intake.

All in all, they make a
convincing case for the mechanism of short-term habituation as found in the
gill-withdrawal reflex.

BA: A slightly more
elaborate diagram includes input from the tail from neurons that respond to a
shock that is administered to the tail and produces sensitization (or
dishabituation) of the gill-withdrawal reflex.

BP: How does the shock do that? Are its
effects confined to the sensory neurons in the tail? In the diagram the sensory
neuron is shown coming from the skin of the siphon, not from the tail. I don’t
like shocks as stimuli because their effects can spread all over the place,
even far from where the shock is administered. How about continuous tactile
stimuli?

I said that a slightly more
elaborate diagram includes input from the tail. It’s not the one I
presented. It might be on the one given in IMP.

The shocks employed produce
localized effects in the tail. The reason shock is used is that it meets the
requirement of a stimulus that will produce sensitization. The input must
change suddenly and fairly strongly. Continuous tactile stimuli wouldn’t
work.

BA: The axons of
these neurons synapse on the terminal buttons of the sensory neurons
shown in the diagram below.

BP: Do you mean the area where the vescicles
containing neurotransmitters are located? I’m not familiar with the terminology
here.

That’s right. It’
called an axon-axonal synapse. For what it’s worth, the neurotransmitter
released here is different from the one released by the sensory neuron.

BA: This input
immediately reverses the habituation.

BP: I wish we could find more descriptive
terms that indicate what is happening. “Habituation” is a description
of a remotely-related observation. Does the habituation signal change the
amount of neurotransmitter emitted per impulse or does it just add its own
excitatory or inhibitory neurotransmitter to whatever is being emitted anyway?

The reduction in the intensity
of gill-withdrawal (i.e., habituation) is brought about merely by repeated
touch stimulation of sensory neurons in the skin of the mantle or siphon. The “habituation
signal,” if it can be called such, is internal to the sensory neuron and
manifests itself as a reduction in number of membrane calcium (Ca++) gates that
open in response to the action potential arriving in the terminal buttons of
the sensory neuron. Less CA++ enters the terminal button, so there is less vesicle
movement, leading to reduced release of neurotransmitter into the synaptic gap.
Input from the sensory neurons in the tail, when they are stimulated strongly,
causes serotonin to be released at the axon-axonal synapse with the mantle-skin
sensory neuron terminals, inducing internal changes in the latter that reverse
the effect of habituation.

BP: When I started work at the VA research
hospital in Chicago in 1950s, I was full of ambitions. I was going to go to the
medical library at the hospital and start drawing and analyzing circuit
diagrams. I very quickly ran into the problems we have here: lack of vital
information. Whoever drew the diagrams of neural connections didn’t understand
about transfer functions and how important they are to understanding what the
circuit does. In those days all the neurologists did was chase single impulses
around – I’m not sure they do a lot more than that even now. It took me about
six months to give up that grand plan.

BA: Habituation (reduced sensitivity) is brought about by
repeated stimulation of the sensory neurons and is thus carried out exclusively
within the lower-level system, not as a function of a reference input from
above. Dishabituation is brought about by stimulation of sensory neurons in the
tail (in this experiment). This does bring in another input, but unless
I’m not analyzing it correctly, this input doesn’t appear to
function as a reference signal.

BP: It’s possible that the upper neural path in the diagram has a time constant
measured in hours or even days – who knows, without any information? And
without all the other signals that aren’t shown, I think we’re stuck.

Are you referring to the path
from sensory neuron through interneuron to motor neuron? Activity through this
pathway isn’t much different than that through the direct sensory-neuron,
motor-neuron pathway. The interneuron adds a synapse, which would induce a
delay between action potentials arriving at the motor neuron from the
interneuron versus sensory neuron. MY guess is that this delay prolongs the
contraction of the mantle a bit, but as the mantle reopens rather soon (a
matter of a second or so), I would doubt that there’s a long
time-constant involved here on the order of hours or days. On the other hand,
short-term habituation can last for many minutes – longer than the
typical interval between stimulation used to induce habituation in this system.

BA: More recent research has investigated the basis of
long-term habituation, which results when repeated bouts of stimulation produce
habituation that can last for hours or even days. Unlike short-term
habituation, long-term habituation appears to require protein synthesis and
some kind of retrograde signal (i.e., feedback) from the motor neurons back to
the sensory neurons in the mantle and siphon.

BP: Do you mean antidromic impulses going backward through the same nerve fibers?
Or are there feedback effects through the environment from the gill action to
sensory nerves in the siphon or elsewhere? Or perhaps there are other neural
paths not shown in this diagram?

It appears to be something local
to the synapse. I’m guessing that it involves a chemical signal
released by the postsynaptic neuron that is absorbed by the sensory neuron, but
the basis of this putative signal has yet to be identified.

BP: I get that same frustrated feeling that
I had back in the 1950s. If only I could have been leaning over their shoulders
when these researchers were getting their ***** data. It seems to me that they
hardly measured anything.

When it come to neurologists
speculating about what some collection of neurons in the brain may be doing, I
share your frustration. In this case, however, researchers have taken the
approach of identifying a relatively simple system that mediates a specific
reflex and has been shown to demonstrate both habituation and sensitization,
taken careful measurements of neural impulses, PSPs, changes taking place in
calcium gates and vesicle transport and so on, made observations on reduced preparations
in vitro and demonstrated that they reproduce those made on the intact in vivo
system, and so on. It seems to be precisely the sort of work you’ve been looking
for.

Bruce A.

[From Bruce Abbott
(2009.05.15.1720 EDT)]

Bill Powers (2009.05.15.0203 MDT)–

Bruce Abbott (2009.05.12.1140 EDT)

BP: Unfortunately, a diagram like that doesn’t tell us what the circuit does
(as I’m sure you know). If we don’t know the functions performed by each
neuron, we can’t tell whether this is a differentiator, a two-level control
system, or any of a large number of other possibilities. Are there any more
connections that were just not drawn?

BA: I’ve wondered about the role of the interneuron in
that diagram. My best guess is that it prolongs or intensifies the
contraction of the mantle muscles. (I haven’t yet read enough of this
literature to know what Kandel and colleagues have determined its function to
be within this circuit.) All the synapses in the diagram are excitatory, by the
way.

BP: The attached gives a little more information. I haven’t located the Figure
yet. It seems that the interneuron ONLY affects the amount of neurotransmitter
produced by impulses in the other axon – it does not synapse directly with the
motor neuron. So it appears to be purely a gain control.

Yes.

BP: One thing is pretty clear: this is not a
“learning” phenomenon. Associating it with Hebbian learning is just a
mistake, it seems to me. This system always does the same thing; it doesn’t
acquire any new way of reacting to inputs. It habituates and dishabituates over
and over, the same way every time, without learning a thing.

It’s
not Hebbian learning, and no one claims that it is. Evidence for Hebbian
learning has come from further studies of aplysia using a classical conditioning
paradigm, in which two stimuli have to be associated. Habituation and
sensitization do not involve an association and for this reason are referred to
as forms of “nonassociative” learning.

Whether
to call habituation and sensitization “learning” at all depends, I
suppose, on how you define learning. They do involve a change in behavior as a
result of experience, meeting the traditional definition of learning. In
short-term habituation and sensitization, the only thing that changes is what
might be viewed as the “weight” of the synaptic connections.
Connections are neither formed nor destroyed. In the long-term forms of these phenomena,
however, the number of synapses between the two neurons involved changes. Not
observed are synapses forming between neurons that formerly did not have such
synapses. So the “wiring diagram” still remains essentially
unchanged.

In
the “arm” demonstration that accompanies LCS III, reorganization
acts in the same way on existing connections, increasing the weights of some
and decreasing the weights on others. If habituation and sensitization do not
qualify as learning on the grounds that the wiring diagram doesn’t
change, then it seems to me that the form of reorganization implemented in the arm
demo doesn’t quality as learning, on the same grounds. My view is that
learning is involved in both cases.

BP: Gently squirting water onto the mantle
with a WaterPik resulted in withdrawal of the gill and the siphon. Repeated
squirts lead to less and less response, and eventually none. The interpretation
is that Aplysia “learned to ‘ignore’ the stimulus… because it was not
harmful.” I doubt that statement rather thoroughly.

BP: Apparently the normal case is for an
occasional stimulation of the mantle to cause withdrawal of the gill and
siphon. This says that the gain is being maintained at some fairly high level
by activity in the interneuron that raises the gain. Since that interneuron is
activated by sensors in the tail, there must be occasional stimulation of the
tail under these same normal circumstances. It is not unreasonable to assume
that when Aplysia is moving about, the tail is being stimulated.

BP: When the mantle alone is stimulated, but
not the tail, the gain begins to decline gradually because there is no signal
in the interneuron to maintain the gain at a high level. This explains the
habituation. If the tail is then stimulated, the gain is increased and the
habituation is abolished: the normal response to a combined tail and mantle
stimulation returns. The gain control seems to respond rapidly to a brief
signal from the tail, raising the gain, with the gain dropping slowly after the
signal. So only a low rate of stimulation of the tail would be enough to keep
the gain normally high. Habituation can take place only when no signals from
the tail occur for a long time. And of course habituation is not a phenomenon
in itself; it’s just a name for declining gain.

I have no guess about what the controlled variable is - it’s probably something
that has to do with the circumstances under which both mantle and tail are
being stimulated occasionally. Apparently the tail sensors have a rather high
threshold – unless there’s another neuron affecting signals coming from them.

I
would agree that the tail is probably being stimulated as aplysia is moving
about, but you may have overlooked something with respect to short-term
habituation. If you stop the train of repeated stimulation of the mantle
or siphon, the normal level of sensitivity slowly recovers. Based on what
little I’ve read about the cellular physiology involved, I imagine that
each touch of the mantle or siphon not only triggers mantle contraction (prior
to habituation), it produces some kind of transient deactivation of some
portion of the calcium gates in the terminal buttons. If you repeat the
stimulation soon enough, more deactivation occurs before the complete loss of deactivation
from the prior impulse. Consequently an increasing proportion of the
gates get deactivated, until there is no longer sufficient release of neurotransmitter
to fire the motor neuron or interneuron. A relatively long period of rest
allows the gates to recover, so habituation disappears.

Stimulation
arising from shocking the tail after habituation has been performed would
immediately reverse this cumulated deactivation of the Ca++ gates, producing immediate
dishabituation, as this form of recovery of the reflex is known.

BP earlier: I was looking on the web for more detailed
diagrams, but couldn’t find any – and none of them contains any information
about the transfer functions at each synapse. Aplysia contains about 20,000
neurons, so these little snippets don’t go very far toward giving us a picture
of the whole system

BP: The present article raises that number
to 100,000.

That
latter figure may be a mistake. The author of the paper you attached is a
biology professor; the paper is basically his lecture over the portion of the
textbook he is using that covers the neurology of learning. He’s not
involved in this research and is not an expert on aplysia. If there’s a
scientific paper claiming 100K neurons in aplysia, I haven’t seen that
source. But whatever the number, here we’re concerned with a portion that
numbers something like 24 sensory neurons and perhaps a similar number of
interneurons and motorneurons.

BA: Aplysia researchers
have been able to demonstrate what happens during habituation, both in terms of
changes in mantle contraction during repeated stimulation (typically once every
10 seconds in demonstrations of short-term habituation) and in terms of changes
in neurotransmitter release, post-synaptic potentials, and action potentials.
Concerns about whether there might be other inputs to this circuit that
influence the time-course of habituation led to in vitro experiments involving,
in isolation, only the neurons shown in the diagram I provided. Stimulation of
the sensory neurons produced the same pattern of changes found in the neurons
of the intact animal. The observed changes fully account for the changes in the
gill-withdrawal reflex during short-term habituation.

BP: The problem here is not seeing the
forest for the trees. In the Reiness article, the description drops to the
level of biochemistry and any Big Picture is rapidly lost. At the same time,
there’s that annoying habit that biologists have of casually introducing
purposive language at entirely too high a level of abstraction, but with quotes
so you know they don’t really, really mean “ignore because it was not
harmful.” Of course they don’t believe that Aplysia can perceive an
abstraction like harmfulness, or that it can direct attention toward or away
from some stimulus, or that it has “reasons” for what it does as
implied by the word “because.” But if they don’t believe that, why do
they talk that way? Especially about an in vitro preparation?

When
behaviorism was in its heyday, psychologists working with rats or other animals
used to put quotes around words denoting emotions (e.g., “fear”) to
alert the reader that this label is intended as shorthand for “behavior
that we humans might display if we were afraid, so something akin to this
experience might be present in the rat.” It was there to remind the
reader that we can’t get into a rat’s mind and experience what the
rat experiences, so we don’t really know whether that experience is
similar to our own. As time went on the quotes gradually disappeared, so now
cognitive psychologists don’t even acknowledge the possibility that the
rat may not have the same mental experiences that we humans do.

I
do sympathize with writers who are trying to communicate findings such as those
from aplysia research to students and other lay audiences. There is an
overpowering temptation to describe things to a lay audience in familiar terms.
Unfortunately, the audience probably will miss the fact that this language is
just a pedagogical device to facilitate communication, and will think of these
processes in exactly the terms used.

BA: Kandel and others
have shown that short-term habituation in this circuit depends exclusively on a
reduction in the amount of neurotransmitter being released with each impulse of
the sensory neuron. This reduction in turn results from decreased Ca++ intake
in the axon terminal during the action potential. Ca++ intake is
necessary for the transport of the vesicles containing the neurotransmitter
toward the synaptic gap. They’ve also identified the cellular changes
that reduce CA++ intake. All in all, they make a convincing case for the
mechanism of short-term habituation as found in the gill-withdrawal reflex.

Right, but somehow they missed the fact that the normal case has to be
maintenance of the amount of neurotransmitter release at the higher end of the
range – otherwise there would be no response to habituate.

See
above re: dishabituation.

BA: The shocks employed
produce localized effects in the tail. The reason shock is used is that it
meets the requirement of a stimulus that will produce sensitization. The input
must change suddenly and fairly strongly. Continuous tactile stimuli
wouldn’t work.

BA: Like something starting to take a bite
out of your tail. So the controlled variable is something like a rate of change
of stimulation of the tail and mantle. Shocks, however, do not produce
localized effects unless they’ve changed the laws of electricity. A shock to
your hand can stop your heart. Electricity goes everywhere it can go, which is
why I don’t like shocks as a physiological stimulus.

Actually,
a shock to the hand can’t stop your heart, unless the conduction pathway
to ground passes through the heart. Or so I understand. With respect to
aplysia, shock produces the same effect on the habituated gill-withdrawal
response as pinching the tail – a more natural stimulus, I’m sure
you’ll agree. Shock normally is used because it can be modulated
accurately and with precise timing.

BA: When it come to
neurologists speculating about what some collection of neurons in the brain may
be doing, I share your frustration. In this case, however, researchers have
taken the approach of identifying a relatively simple system that mediates a
specific reflex and has been shown to demonstrate both habituation and
sensitization, taken careful measurements of neural impulses, PSPs, changes taking
place in calcium gates and vesicle transport and so on, made observations on
reduced preparations in vitro and demonstrated that they reproduce those made
on the intact in vivo system, and so on. It seems to be precisely the sort of
work you’ve been looking for.

It is, but it’s masked by unnecessary complications and it stops short of being
useful – as usual. Either we get fanciful abstractions, or we’re drowned in
irrelevant (for purposes of understanding the circuit) details. And when you
finally start getting a handle on what’s happening, it turns out that they
missed the main fact – here, the fact that something must have been
maintaining the gain at a high level.

What
I’ve described here on CSGnet is only a small portion of the research
that has been conducted investigating nonassociative and associative learning
processes in aplysia. I’m starting to wade through some of it in preparation
for my Learning & Behavior text, but still have a long way to go before I
can say that I am familiar with and understand it. As the research progressed,
initial hypotheses and speculations about the circuitry gave way to other,
often more complex understandings. No doubt those ideas will have to be revised
further as new evidence comes in. However, the basic findings about what
changes at the synaptic and cellular level during the course of habituation and
sensitization seem to be holding up rather well. If anyone has access to
old issues of Scientific American, an early but quite nice description
of Kandel’s research on aplysia appeared in an article by Eric Kandel
entitled “Small Systems of Neurons” that appeared in September
1979, vol 241(3), 66-76.

More
recent research has demonstrated that what appears to be classical conditioning
can be obtained. This involves developing an association between a
previously neutral stimulus and the touch stimuli that elicit gill withdrawal.
The “association,” as expected, takes the form of changes in the
release of neurotransmitter at the synapse and changes in the number of
synapses between a given pair of neurons. Some of this research has been aimed
toward identifying whether the formation of this association follows the
Hebbian rule.

Bruce
A.

[From Bill Powers (2009.05.15.1709 MDT)]

Bruce Abbott (2009.05.15.1720 EDT) --

BP: One thing is pretty clear: this is not a "learning" phenomenon.

BA: Whether to call habituation and sensitization �learning� at all depends, I suppose, on how you define learning. They do involve a change in behavior as a result of experience, meeting the traditional definition of learning.

BP: Suppose that on your way home you encounter a road sign that says "Detour" with an arrow pointing to one side. That is an experience, is it not? And of course you turn and take the alternate route: that is a change in behavior, isn't it? So if this happens every week when the big ball game is scheduled, and you take the alternate route every week, do you learn, every week, to turn off the main road when you see a detour sign? Or is this merely a behavioral system working as it is supposed to work? I don't think your definition of learning can distinguish between performance of an acquired process and the acquisition of that process.

I think we would say that learning has taken place only if you do not wait until the detour sign appears, but find your own alternate route so you don't have to go the long way home. Or if you ignore the detour sign and find out the hard way why it's there, and then change your organization so you turn when you get to it. And once you've learned that, you don't have to learn it again

If Aplysia habituates every time a certain sequence of stimuli occurs, I don't think we can say it's learning anything. It's simply organized in such a way that habituation happens under these conditions. It lowers its gain when the same stimulus is applied again and again. I don't know if it had to learn to do that or if it's a built in organization -- I'd guess the latter. But once it is organized to do that, no further learning (if it ever happened) is necessary.

BA: In short-term habituation and sensitization, the only thing that changes is what might be viewed as the �weight� of the synaptic connections.

...

In the �arm� demonstration that accompanies LCS III, reorganization acts in the same way on existing connections, increasing the weights of some and decreasing the weights on others. If habituation and sensitization do not qualify as learning on the grounds that the wiring diagram doesn�t change, then it seems to me that the form of reorganization implemented in the arm demo doesn�t quality as learning, on the same grounds. My view is that learning is involved in both cases.

BP: The reorganizing system doesn't need to learn anything new to vary the weightings. It's already organized to do that. In the arm demo, a fixed subroutine takes care of that; I tried a number of different algorithms while I was learning how to program reorganization, but when I succeeded the algorithm became fixed and I no longer changed it. I did learn something, but once I learned it I didn't have to go on learning it.

The problem here is that "learning" is a poorly defined term -- really, a common-language term and not a scientific term at all. The same term is applied to many completely different phenomena -- reorganizing and memorizing, for example.

BA: I would agree that the tail is probably being stimulated as aplysia is moving about, but you may have overlooked something with respect to short-term habituation. If you stop the train of repeated stimulation of the mantle or siphon, the normal level of sensitivity slowly recovers. Based on what little I�ve read about the cellular physiology involved, I imagine that each touch of the mantle or siphon not only triggers mantle contraction (prior to habituation), it produces some kind of transient deactivation of some portion of the calcium gates in the terminal buttons. If you repeat the stimulation soon enough, more deactivation occurs before the complete loss of deactivation from the prior impulse. Consequently an increasing proportion of the gates get deactivated, until there is no longer sufficient release of neurotransmitter to fire the motor neuron or interneuron. A relatively long period of rest allows the gates to recover, so habituation disappears.

BP: Ah, good, I did overlook that possibility. So what you're saying is that under normal conditions there is some natural amount of neurotransmitter available for release. After a stimulus to the mantle which causes siphon and gills to be retracted, some of that neurotransmitter (or whatever supplies it) is used up. The neurotransmitter is gradually brought back up to the normal state, but if such stimuli repeat often enough there isn't time to restore what was lost and the level of neurotransmitter declines with each successive response to the stimulus, until there isn't enough left to produce another response, or the neural response is too small to cause the usual motor actions.

However, a sufficient stimulus to the tail can cause a large amount of neurotransmitter to become available again:

BA: Stimulation arising from shocking the tail after habituation has been performed would immediately reverse this cumulated deactivation of the Ca++ gates, producing immediate dishabituation, as this form of recovery of the reflex is known.

BP: So if there were repeated stimuli to the tail, there would be no habituation to the stimulation of the mantle, as the level of available neurotransmitter would frequently be brought back up to the normal state, or even above it. Each response to a mantle stimulation would depelete the available neurotransmitter, or the CA++ action that generates it, but the frequent tail stimuli would prevent the level from dropping to the point that the motor actions do not take place.

This is how I would prefer to approach this problem -- not by trying to make sense of common-language terminology, but by constructing a working model that will behave as the real system behaves. It takes a while to take into account all the information that's available, but at each stage the model has to be altered to agree with all that's known so far. I started out with a model that assumed the level of neurotransmitter was being maintained by tail stimulation, and that provided one explanation. However, you came up with new information (new to me) that said even without tail stimulation, the level of transmitter would gradually recover if no further mantle stimuli were applied for a while. OK, that meant that tail stimulation was not needed to cause this gradual recovery -- but the instant recovery of the normal level of neurotransmitter after tail stimulation meant that a rapid recovery could be produced that way. So now there is some background slow rate of recovery that is always present, and a fast rate upon tail stimulation.

As the role of the calcium channels is worked out, I'm sure we can now produce a working model of this circuit that will include many details that are known.

Actually, a shock to the hand can�t stop your heart, unless the conduction pathway to ground passes through the heart. Or so I understand. With respect to aplysia, shock produces the same effect on the habituated gill-withdrawal response as pinching the tail � a more natural stimulus, I�m sure you�ll agree. Shock normally is used because it can be modulated accurately and with precise timing.

Yeah, yeah, yeah. Excuses excuses. Shock does not just stimulate the one thing you want to stimulate. It's not a natural stimulus. It can affect every neuron and other cell within some range of the electrode and produce effects irrelevant to the phenomena under study. Saying that it's easy to manipulate is not a reason for using it if you can't prove you're not generating artifacts at the same time. I'm sure better methods can be devised.

More recent research has demonstrated that what appears to be classical conditioning can be obtained. This involves developing an association between a previously neutral stimulus and the touch stimuli that elicit gill withdrawal. The �association,� as expected, takes the form of changes in the release of neurotransmitter at the synapse and changes in the number of synapses between a given pair of neurons. Some of this research has been aimed toward identifying whether the formation of this association follows the Hebbian rule.

I just wish people would forget all that conditioning stuff, that Hebbian stuff, all that habituation and dishabituation nonsense, and just get on with the modeling. Hebb didn't say how his rule was implemented, and we can be sure it's not implemented by stating a rule. The rule is only a verbal approximation to what is actually happening.

Habituation is not a phenomenon, it's play on the word "habit". A model describes what is meant much more clearly and consistently. "Conditioning" doesn't describe anything; it's just a mnemonic label. The whole point of modeling is to express everything in terms of observable variables and measurable relationships (even if you have to invent some of them) so you can reproduce something that actually exists and happens. When you can do that, you don't need the vague labels -- or you've given the vague labels something precise to be attached to.

Best,

Bill P.

[From Rick Marken (2009.05.15.1800)]

Bruce Abbott
(2009.05.15.1720 EDT)]

I’m starting to wade through some of it in preparation
for my Learning & Behavior text, but still have a long way to go before I
can say that I am familiar with and understand it.

Wow. That’s terrific. Will it present learning and behavior from a control theory point of view? I can’t wait to see it. Good luck on it.

Best

Rick

···


Richard S. Marken PhD
rsmarken@gmail.com

[From Bruce Abbott (2009.05.16.0905 EDT)]

Bill Powers (2009.05.15.1709 MDT) --

Bruce Abbott (2009.05.15.1720 EDT)

BP: One thing is pretty clear: this is not a "learning" phenomenon.

BA: Whether to call habituation and
sensitization "learning" at all depends, I suppose, on how you define
learning. They do involve a change in behavior as a result of
experience, meeting the traditional definition of learning.

BP: Suppose that on your way home you encounter a road sign that says
"Detour" with an arrow pointing to one side. That is an experience, is it
not? And of course you turn and take the alternate route: that is a change
in behavior, isn't it? So if this happens every week when the big ball game
is scheduled, and you take the alternate route every week, do you learn,
every week, to turn off the main road when you see a detour sign? Or is this
merely a behavioral system working as it is supposed to work? I don't think
your definition of learning can distinguish between performance of an
acquired process and the acquisition of that process.

You turned at the Detour sign is because (1) previously you learned the
meaning of the Detour sign, and (2) you have a goal of reaching some
destination that probably cannot be reached by ignoring the Detour sign.
Turning at the Detour sign when you see it is not a change in behavior if
this is what you do every time. The learning took place earlier, when
experience taught you what the sign means, and what is likely to happen if
you ignore it.

BP: I think we would say that learning has taken place only if you do not
wait until the detour sign appears, but find your own alternate route so you
don't have to go the long way home. Or if you ignore the detour sign and
find out the hard way why it's there, and then change your organization so
you turn when you get to it. And once you've learned that, you don't have to
learn it again.

That's another way to assess whether you have learned, but I would say that
learning has also taken place if a person previously ignorant of the meaning
and implications of the sign now, after appropriate experiences, takes an
alternate route when encountering the sign.

BP: If Aplysia habituates every time a certain sequence of stimuli occurs, I
don't think we can say it's learning anything. It's simply organized in such
a way that habituation happens under these conditions. It lowers its gain
when the same stimulus is applied again and again. I don't know if it had to
learn to do that or if it's a built in organization -- I'd guess the latter.
But once it is organized to do that, no further learning (if it ever
happened) is necessary.

In habituation and sensitization, the changes in gain occur as a function of
the animal's recent experience (patterns of sensory input), and the result
is a change in behavior that appears to have adaptive value for aplysia. In
the short-term versions of these phenomena, these changes dissipate over a
relatively short period. They persist for much longer in the long-term
versions, which appear to operate via somewhat different mechanisms than the
short-term versions. These reversible changes resemble those seen in
short-term memory; indeed, memory may involve similar, though more complex
changes at the cellular level. Whether or not you prefer to categorize
habituation and sensitization as forms of learning, the cellular mechanisms
that have been revealed by studying them have contributed significantly
toward a better understanding of how neural circuits may learn.

BA: In short-term habituation and sensitization, the only thing that
changes is what might be viewed as the "weight" of the synaptic
connections.

...

BA: In the "arm" demonstration that accompanies LCS III, reorganization
acts in the same way on existing connections, increasing the weights of
some and decreasing the weights on others. If habituation and
sensitization do not qualify as learning on the grounds that the wiring
diagram doesn't change, then it seems to me that the form of
reorganization implemented in the arm demo doesn't quality as learning,
on the same grounds. My view is that learning is involved in both
cases.

BP: The reorganizing system doesn't need to learn anything new to vary the
weightings. It's already organized to do that. In the arm demo, a fixed
subroutine takes care of that; I tried a number of different algorithms
while I was learning how to program reorganization, but when I succeeded the
algorithm became fixed and I no longer changed it. I did learn something,
but once I learned it I didn't have to go on learning it.

O.K., have it your way: reorganization as implemented in the arm demo is not
learning. But I disagree. The weights change in ways that depend on the
arm's experience in controlling its various joints. Changes that reduce
error (by increasing or reducing the contributions of different outputs to
joint angle) tend to be retained over time, until the arm's pattern of
movement matches its reference pattern. The weight changes functionally
change the "wiring" of the system, even though the system's structure
doesn't change.

BP: The problem here is that "learning" is a poorly defined term -- really,
a common-language term and not a scientific term at all. The same term is
applied to many completely different phenomena
-- reorganizing and memorizing, for example.

Yes, learning is a broad category, but it does have subcategories that have
been developed based on common experience and research: Habituation,
sensitization, classical conditioning, operant conditioning, sensory memory,
short-term or working memory, long-term memory (the latter subcategorized as
either declarative or procedural and having different physiological
substrates).

BA: I would agree that the tail is probably being stimulated as aplysia
is moving about, but you may have overlooked something with respect to
short-term habituation. If you stop the train of repeated stimulation
of the mantle or siphon, the normal level of sensitivity slowly
recovers. Based on what little I've read about the cellular physiology
involved, I imagine that each touch of the mantle or siphon not only
triggers mantle contraction (prior to habituation), it produces some
kind of transient deactivation of some portion of the calcium gates in
the terminal buttons. If you repeat the stimulation soon enough, more
deactivation occurs before the complete loss of deactivation from the
prior impulse. Consequently an increasing proportion of the gates get
deactivated, until there is no longer sufficient release of
neurotransmitter to fire the motor neuron or interneuron. A relatively
long period of rest allows the gates to recover, so habituation
disappears.

BP: Ah, good, I did overlook that possibility. So what you're saying is that
under normal conditions there is some natural amount of neurotransmitter
available for release. After a stimulus to the mantle which causes siphon
and gills to be retracted, some of that neurotransmitter (or whatever
supplies it) is used up. The neurotransmitter is gradually brought back up
to the normal state, but if such stimuli repeat often enough there isn't
time to restore what was lost and the level of neurotransmitter declines
with each successive response to the stimulus, until there isn't enough left
to produce another response, or the neural response is too small to cause
the usual motor actions.

BP: However, a sufficient stimulus to the tail can cause a large amount of
neurotransmitter to become available again:

BA: Stimulation arising from shocking the tail after habituation has
been performed would immediately reverse this cumulated deactivation of
the Ca++ gates, producing immediate dishabituation, as this form of
recovery of the reflex is known.

BP: So if there were repeated stimuli to the tail, there would be no
habituation to the stimulation of the mantle, as the level of available
neurotransmitter would frequently be brought back up to the normal state, or
even above it. Each response to a mantle stimulation would depelete the
available neurotransmitter, or the CA++ action that generates it, but the
frequent tail stimuli would prevent the level from dropping to the point
that the motor actions do not take place.

Yes.

BP: This is how I would prefer to approach this problem -- not by trying to
make sense of common-language terminology, but by constructing a working
model that will behave as the real system behaves. It takes a while to take
into account all the information that's available, but at each stage the
model has to be altered to agree with all that's known so far. I started out
with a model that assumed the level of neurotransmitter was being maintained
by tail stimulation, and that provided one explanation. However, you came up
with new information (new to me) that said even without tail stimulation,
the level of transmitter would gradually recover if no further mantle
stimuli were applied for a while. OK, that meant that tail stimulation was
not needed to cause this gradual recovery -- but the instant recovery of the
normal level of neurotransmitter after tail stimulation meant that a rapid
recovery could be produced that way. So now there is some background slow
rate of recovery that is always present, and a fast rate upon tail
stimulation.

BP: As the role of the calcium channels is worked out, I'm sure we can now
produce a working model of this circuit that will include many details that
are known.

Yes -- did you overlook my description of how such a model might be
implemented? I had to leave blank for the time-being the mechanism that
produces the temporary deactivation of Ca++ gates, but I suspect that the
general organization is right.

BA: Actually, a shock to the hand can't stop your heart, unless the
conduction pathway to ground passes through the heart. Or so I
understand.
With respect to aplysia, shock produces the same effect on the
habituated gill-withdrawal response as pinching the tail - a more
natural stimulus, I'm sure you'll agree. Shock normally is used because
it can be modulated accurately and with precise timing.

BP: Yeah, yeah, yeah. Excuses excuses. Shock does not just stimulate the one
thing you want to stimulate. It's not a natural stimulus. It can affect
every neuron and other cell within some range of the electrode and produce
effects irrelevant to the phenomena under study. Saying that it's easy to
manipulate is not a reason for using it if you can't prove you're not
generating artifacts at the same time. I'm sure better methods can be
devised.

BA: More recent research has demonstrated that what appears to be classical

conditioning can be obtained. This involves developing an association
between a previously neutral stimulus and the touch stimuli that elicit
gill withdrawal. The "association," as expected, takes the form of
changes in the release of neurotransmitter at the synapse and changes
in the number of synapses between a given pair of neurons. Some of this
research has been aimed toward identifying whether the formation of
this association follows the Hebbian rule.

BP: I just wish people would forget all that conditioning stuff, that
Hebbian stuff, all that habituation and dishabituation nonsense, and just
get on with the modeling. Hebb didn't say how his rule was implemented, and
we can be sure it's not implemented by stating a rule. The rule is only a
verbal approximation to what is actually happening.

Actually, researchers are attempting to discover whether the circuits that
demonstrate conditioning in effect implement the Hebbian rule. Nobody
believes that neurons consult a rule -- you're just engaging in silly
hyperbole. That said, I agree that the modeling approach is the way to go.
But first, you have to know what characteristics the system in question
exhibits, to guide your model-building and test the product.

BP: Habituation is not a phenomenon, it's play on the word "habit". A model
describes what is meant much more clearly and consistently.

Habituation is indeed derived from the word "habit," but it is also a
reproducible phenomenon that is exhibited by a huge variety of systems
across almost (or perhaps all) species of animal. Saying that it isn't a
phenomenon doesn't make it go away. We need labels to provide a short-hand
way of referring to things, especially relatively complex ones. Again, I
agree that models need to be developed, but first you need to know what it's
a model of. Kandel focused on aplysia because he viewed habituation as
perhaps the simplest form of learning and wanted to understand its cellular
basis. Aplysia's gill-withdrawal reflex demonstrates the habituation
phenomenon and has the considerable advantage of being based on a nervous
system in which every neuron is the same from one animal to another -- each
uniquely identifiable and each "wired" to the others in the same way. If the
mechanisms discovered in aplysia proved to be general (extending perhaps
even to the human nervous system), then the research would advance
considerably our understanding of the neural mechanisms involved in
habituation. A similar argument can be advanced for the study of other
processes such as simple associative learning, if they could be demonstrated
in aplysia or other relatively simple animals (such as the nematode, which
has an even simpler nervous system than aplysia).

BP: "Conditioning" doesn't describe anything; it's just a mnemonic label.
The whole point of modeling is to express everything in terms of observable
variables and measurable relationships (even if you have to invent some of
them) so you can reproduce something that actually exists and happens. When
you can do that, you don't need the vague labels -- or you've given the
vague labels something precise to be attached to.

By the same token, "control" doesn't describe anything either. It derives
from a middle English word meaning "to keep a duplicate account or roll."
Given your attitude toward labels, I don't understand why you bother to use
it.

I'm being facetious, of course. I believe that the term is useful (and so do
you, I strongly suspect). It provides a short-hand way to refer to a
reproducible phenomenon, the phenomenon we call "control." Terms like
"habituation" or "classical conditioning" are similarly useful, even if they
are no substitute for models.

Bruce A.

[From Bruce Abbott
(2009.05.16.1400 EDT)]

Rick Marken (2009.05.16.0900) –

Bruce Abbott (2009.05.16.0905 EDT)

Yes, learning is a broad category, but it does have subcategories that have
been developed based on common experience and research: Habituation,
sensitization, classical conditioning, operant conditioning, sensory memory,
short-term or working memory, long-term memory (the latter subcategorized as
either declarative or procedural and having different physiological
substrates).

RM: Surely you plan to explain in your text
that these are all just different aspects of control. It would be particularly
easy to do with classical and operant conditioning. That would be super!

I do plan to introduce perceptual control theory and discuss how
it can explain many aspects of these phenomena. However, the reorganization
process as currently developed within PCT is still in a rather primitive state
of development and it is not at all clear how it could be applied to explain a
number of known learning phenomena.

Consider the following example. If you follow the illumination
of a cue light with grain, a hungry pigeon will approach and peck at the light
when it comes on, even if it’s at the other end of the chamber from the
food magazine. The phenomenon is called “sign-tracking.”
However, if the stimulus is diffuse rather than localized, the pigeon will
approach the food cup instead. I can understand the latter in terms of control
theory, but the former has me baffled. By approaching the localized cue light,
the pigeon delays getting the food.

An even more dramatic example develops when you illuminate a hungry
pigeon’s response key for several seconds and then follow this
illumination with an automatic operation of the food magazine. After a few such
pairings, the pigeon will begin to peck at the key, even though key-pecking
does not activate the magazine. If you explicitly program a contingency such
that pecking the key cancels the magazine operation that otherwise would occur
automatically, the pigeon still pecks at the key, thus loosing opportunities to
access grain.

The problem seems to be that the hungry pigeon comes equipped
with a strong innate urge to approach places that have been associated with
food. In nature this strategy normally would be effective in raising the probability
of finding food, but in the artificial laboratory situation where approaching such
a stimulus leads the pigeon away from the food source, the pigeon appears to
have difficulty suppressing this now-maladaptive behavior.

With respect to classical conditioning, it has been shown that a
conditioned salivary response to a stimulus that has been paired with food
develops even if salivation has been blocked with atropine during the
conditioning process. Under PCT, one might assume that under normal
circumstances salivation would become conditioned because, during reorganization,
salivation helps to better control some perception, such as the taste of the
food or the ease with which it can be chewed and swallowed. However, this model
would appear to predict that blocking salivation during the conditioning
process would prevent this action on the controlled variable. It would have no
beneficial effect in reducing error and would not be retained by the
reorganization process. Thus, one first glance at least, it would appear that at
least one model based on PCT is contradicted by the evidence.

There may be ways to explain these findings within the PCT
framework, but as these examples suggest, it is far from a trivial exercise to
do so.

Bruce A.

[From Bill Powers (2009.05.17.0828 MDT)]

Bruce Abbott (2009.05.16.0905 EDT)

Yes, learning is a broad category, but it does have subcategories that
have been developed based on common experience and research: Habituation,
sensitization, classical conditioning, operant conditioning, sensory memory,
short-term or working memory, long-term memory (the latter subcategorized as
either declarative or procedural and having different physiological
substrates).

My objection to calling all these things by the same name is that they are fundamentally different and should not be confused with each other. Does "conditioning" mean the same thing whether preceded by "classical" or "op-erant"? Imagine the shape physics would be in if we decided that the word "movement" could be used equally well and interchangeably to mean velocity, acceleration, or oscillation back and forth. Terms like learning are informal and subjective, describing more what the speaker is reminded of than any reliable or unique category of phenomena agreed to by all. The subdivisions are not examples of some concept common to all of them; they belong together only if one decides to ignore all the differences between them. Psychology is built on such terminology and that is one of the reasons it's not a science yet. It has no technical language.

In PCT we have a chance to establish a consistent terminology in which one term always means the same thing and meanings are not allowed to change in the middle of a sentence. Even so, some people have trouble with the difference between a perception and a reference condition or signal. Consistency is the first requirement of a technical language, because only with stable and mutually exclusive categories can we hope to translate from verbal descriptions to mathematical descriptions and back again. And if we can't do that translation, we will be doomed to live in the small simple fuzzy world that can be described in natural language. Everything we know will be only what we learned in kindergarten.

Best,

Bill P.

[From Rick Marken (2009.05.17.0930)]

Bruce Abbott
(2009.05.16.1400 EDT)–

Rick Marken (2009.05.16.0900) –

RM: Surely you plan to explain in your text
that these are all just different aspects of control. It would be particularly
easy to do with classical and operant conditioning. That would be super!
I do plan to introduce perceptual control theory and discuss how
it can explain many aspects of these phenomena. However…

Consider the following example. If you follow the illumination
of a cue light with grain, a hungry pigeon will approach and peck at the light
when it comes on, even if it’s at the other end of the chamber from the
food magazine…

An even more dramatic example develops when you illuminate a hungry
pigeon’s response key for several seconds and then follow this
illumination with an automatic operation of the food magazine. After a few such
pairings, the pigeon will begin to peck at the key, even though key-pecking
does not activate the magazine…

The problem seems to be that the hungry pigeon comes equipped
with a strong innate urge to approach places that have been associated with
food…

With respect to classical conditioning, it has been shown that a
conditioned salivary response to a stimulus that has been paired with food
develops even if salivation has been blocked with atropine during the
conditioning process…

…Thus, one first glance at least, it would appear that at
least one model based on PCT is contradicted by the evidence.

Well, of course. As Martin Taylor has shown, quite a bit of behavior is caused by internal or external stimuli, just as psychologists have always thought. So it’s unlikely that we will be able to explain this kind of behavior in terms of PCT; conventional theories are more appropriate. But it’s nice that you will be introducing PCT where possible.

Best

Rick

···

Richard S. Marken PhD
rsmarken@gmail.com