[From Bill Powers (2004.09.16.1037 MDT)]
SteveO 09/16/2004 8:13 CST –
Hello, Steve. Always glad to get basic questions, especially about those
chapters that nobody reads!
I’m confused about the difference
between the adder (Figure 3.5, pg 30) and
the multiplier (Figure 3.6, pg 30). I believe the discussion is on
page
- Are we saying the receiving cells are different between the
two
functions?
Yes, I think the difference would have to be there. The critical
properties would be the cell-wall capacitance and the time constant of
the post-synaptic potential that builds up inside the cell as each jolt
of neurotransmitter arrives. I’m a bit hazy about the details here; it’s
been some years since I reviewed the facts. At any rate, it takes a
certain amount of charge to change the potential enough to cause an
output spike to be generated, and generating that spike reduces the
potential so it has to build up for a while to generate the next output
spike. The calcium and potassium pumps in the cell wall reset the
threshold potential after each discharge.
If the capacitance of the cell walls is very large, the potential doesn’t
change much for a single synaptic event. If the input is blip and the
output of the cell is BANG, the process goes
blip-blip-blip-blip-blip-blip-blip-blip-BANG-blip-blip-blip-blip-blip-blip-blip-blip-BANG-blip-blip-blip-blip-blip-blip-blip-blip-BANG.
The faster the input blips arrive, the faster the output bangs occur. In
this case I’ve said it takes 8 blips to produce one output pulse, so the
output frequency is one eighth of the input frequency.
Now suppose we have two synapses on the same high-capacitance cell, both
providing blips of neurotransmitter at the same rate. Obviously it will
now take only half as long for the postsynaptic potential to change
enough to create an output spike. Half the interval means twice the
frequency. With three inputs it would take 1/3 as long and the output
frequency would be three times as high as with only one input. So the
frequency of the output spikes is the input frequency times the number of
inputs receiving impulses at the same rate.
Now, what if the frequency of a single input varies? If twice as many
impulses arrive per second, clearly it will take only half as long
to reach the firing threshold, and the output frequency will double. So
the output frequency is proportional to the input frequency. That gives
us all we need to say that this sort of high-capacitance cell will be an
adder: output frequency is the sum of all input frequencies.
Now what about the multiplier or logic gate? This, of course, was the
initial model inspired by the digital computer, which treated nerve cells
as logical elements and assumed that a cell fired only when two input
spikes occurred at the same time. To implement this sort of cell, the
capacitance would have to be made very low, so that a single input event
could bring the postsynaptic potential a substantial fraction of the way
to the firing threshold, but not all the way. Also, to make a
time-coincidence of inputs necessary, it would be necessary that the
capacitor be rather leaky, so the potential from a single input spike
would die out rapidly. You can see that with a non-leaky capacitor, an
input impulse would change the potential and then the potential would
just remain, so that a second input event occurring at any later
time could cause enough additional change in potential to produce an
output spike. This would be a different kind of semi-logical element in
which coincidences are not necessary – it would be interesting to see
what could be built out of such elements, which are half digital and half
analog.
At any rate, if the capacitance is small and the leakage is significant,
a second input event would have to occur before the change in potential
produced by the first input had died away too much. That would be the
classical “coincidence detector” form of AND gate. Input A AND
input B would have to occur within some small time slice in order to
generate an output spike.
Finally we get to the multiplier, which grows out of the
coincidence-detector kind of gate. Suppose that the input events occur at
random with some uniform distribution. The probability that two events
will occur within a given time is the product of the probabilities that
either event will occur. Suppose that one of the events occurs in
10 percent of the time intervals, at random. Let the second event occur
at a higher average frequency, say in 50% of the samples. If both events
must occur for an output spike to be generated, then clearly the
probability of a coincidence per sample time will be (0.1)*(0.5) or 0.05.
The average output frequency will be the product of the two average input
frequencies.
In neural processes, even when we consider logic, more than just
occurrance and non-occurrance of input impulses matters. If the threshold
is set high, then it could take three simultaneous inputs to create an
output event, which is like a three-input AND gate. But if another
high-frequency input arrived in a way that contributed only a very small
change in potential per impulse, the frequency of this input could
effectively change the threshold, turning the gate into a four-way
AND gate if the new input were inhibitory, or a two-way
AND gate or even into a simple OR gate if excitatory. If the threshold
were made low enough, the AND gate would become an OR gate since any of
the multiple inputs would be sufficient to produce an output. Actually,
we’re talking about an “any N out of M” gate, where N depends
on the firing threshold and M is the number of independent inputs. We
don’t see this sort of variable-logic element in any of our artificial
devices. Such devices might prove to do interesting things once we worked
out the methods of analysis.
We’ve come a long way from the old idea that synapses were simply places
where neural impuses were “relayed” to the next neuron in a
chain.
Best,
Bill P.