Hi, Henry --
cc to CSGnet, so [From Bill Powers (2010.03.12.1000)]
Trying to boil this down to one issue at a time. There are so many branching subjects that its hard to keep track.
Let's see if we can get together with the issue of timing, which I understand is pretty important for classical conditioning. [Added later: I think this leads to some pretty nice convergences of PCT with neuroscience].
HY: synchrony not strictly necessary. who said it's necessary? There are many neurons synapsing on a single neuron. the postsynaptic neuron can detect the coincidence of any two of these thousands of inputs, and strengthen the weaker one if the stronger one is strong enough.
That's pretty much the claim. Plus there's some optimal time window between the two inputs, the stronger one arriving shortly after the weaker one. I don't see a problem with synchrony. Also remember neurons can be very active all the time, and transmitters are released at thousands of synapses on a cell at any given time. If ten thousand people are calling you everyday, the likelihood of two of them call you around the same time, one after the other, is pretty high, right? Why do they need to have a discussion on how to synchronize?
BP: OK, does this mean it doesn't matter which weaker one is paired with which stronger one? But let's suppose there is a pairing. As I'm coming to understand, a fairly large number of repeated pairings is necessary to get the strengthening effect on the weaker one. This takes us back to the problem of how the neurons where the signals are coming from are made to fire at the same time enough times in a row.
Of course if it doesn't matter which pairs fire at any given time, then as you say a weaker one would always have a pretty good chance of firing just before a stronger one somewhere else fired. But as one of those quotes I sent said, this creates a problem in that all synapses will either go to maximum strength or zero strength, which doesn't sound very promising for learning anything. And what if it's important that the weaker one be associated with one and only one stronger one? Then it would matter which two signals are paired.
If the Hebbian rule is to be an explanation of classical conditioning, or the basis of learning to perceive a causal relationship (as Rescorla proposes), then the stronger one of the signals has to represent the US and the weaker one the CS. After many repetitions, the CS alone is able to produce the perceptual response; it is perceived as a signal that the US is about to occur, or more simply in PCT terms, it gives rise to the same change in perception that the US causes, and thus the same error signal.
There is fairly good agreement among CSGers that the US in classical conditioning is simply a disturbing variable that affects a controlled variable, perhaps one that is controlled by an inborn control system or perhaps by a learned one that is mistaken (by the observer) for an inherited one. The prick of a pin disturbs some input quantity and causes a signal for which the organism has an inherited reference level of zero. The response to the resulting error signal produces a motor action that opposes the effect of the pin -- pulls the skin away from the pin. In SR psychology this is looked upon as a piece of good luck for the organism but certainly not as an "intended" result. In PCT it's just an inherited control system, which can be modified by reorganization. Control of the CS (or its effects) can be improved by reacting to other variables that anticipate the onset of the CS, filling in the brief reaction time and perhaps even keeping the CS from occurring if the US happens to predict the CS. So this says that classical conditioning amounts to modifying the definition of the controlled variable by modifying the perceptual input function.
Besides, the timing of spikes with some optimal delay for plasticity is pretty commonplace given the organization of the brain. Think about input systems, say a stimulus sweeping across your visual field and activating cells sequentially.
Yes, this might be useful for velocity perception, too. There are many ways to get that, however.
I think it would be quite possible to set up a simulation of a control system with reorganization that would show the basic relationships of classical conditioning. We would start with your neuron that has 20,000 synapses on its dendrites (more practically, 200), each synapse receving a signal from a different sensory cell. All of these synapses would start out with small random weights, but a few of them would have large weights (the synaptic strength would be high). The output of the neuron would be a perceptual signal representing the magnitude of a US. I'm using the term neuron here just to mean one or more neurons in a perceptual input function.
The perceptual signal, we will say, is compared with a reference signal set to zero, so the error signal is simply the negative of the perceptual signal (which means a signal having an inhibitory effect on anagonist output or exciting an antagonist.*
Let's say that we start with just one active synapse at the input, with all the rest inert (weights small or zero). Let's say that the error signal activates some motor output that has a negative effect on the input quantity that affects the perceptual signal that is connected to the one active synapse. There's the control system. We can use the model of demo 3-1 in LCS3, so we can put in perceptual delays and such. But we have to add a bunch of other input quantities that affect the same perceptual input functions through weights that start out at zero -- they'd be modeled like disturbances, perhaps.
Oops. Holy Smokes. Are CS's nothing more than disturbances? This is another of those moments when an idea turns inside out or upside down. Consider the question of what causes a US. In the laboratory it's always an experimenter or some apparatus he built. The US, as the experimenter sees it, is the pin or the puff of air or whatever is used as a stimulus. What the observer DOESN'T see is the input quantity that is disturbed by the US.
That makes it too easy -- no learning needed. If the CS is just another disturbance that has the same effect on the input quantity that the US has, the control system is already set up to oppose that disturbance or any other. If the CS occurs, the same action will occur as when the US occurs. This might convince an observer that the CS must have been conditioned sometime before the observations took place, but that would be a mistake and not a true example of classical conditioning.
But that idea takes us a step toward the PCT model we want. In effect, we want the perceptual input function that starts out responding only to the CS to become reorganized so it responds to either the US or the CS. This means that the controlled variable gets redefined. After that, the controlled variable is no longer just the set of microvariables making up the input quantity affected directly by the US (and affected oppositely by the output action). It now includes a new set of microvariables such that disturbing them can also alter the perceptual signal being controlled.
So how does this new set of microvariables get created? The answer is that they don't -- they have been there all the time. They might even have been stimulating primary sensory receptors. But they have not been connected to the perceptual input function of the existing control system we're talking about, the one controlling the sensory effect of the US. To become connected they must send axons to the neurons of the existing perceptual input function (that detects the US) and the synapses must be created and strengthened until they are fully functional.
Now comes the reorganization trick.
Suppose there are 20,000 synapses on the neuron (or inputs to the neural net), one or a few of which represent the input quantity disturbed by the US. Let's say that all of their weights are being altered at random. Well, they would be altered at random if the variables they represent are not correlated with the US. Sometimes they would act on the Excitory Post-Synaptic Potential (EPSP) just before the cell fires, and sometimes just after, and according to the Hebbian rule (and experimental results), their synaptic strengths would not be increased, and would often be decreased. But some of them would systematically increase just before the US started, and those synaptic strengths would be increase continually until they became normal synaptic connections.
I think we have almost an E. coli reorganizing system here. If some of the input quantities from the environment correlate with the input quantities affected by the US, they will eventually be connected to the same perceptual input function and will be just as capable of disturbing the control system as the US is. And since there is only one action that the control system can take (as defined so far) to counteract the effects on the input quantity, the CS will result in the same action that is used by the control system to counteract the effects of the US. Voila: classical conditioning.
I'm cc-ing this to CSGnet, Henry. Who wants the glory on this one? Anybody who modifies Demo 3-1 to test the above ideas can claim to be the first to show that classical conditioning is really a reorganization effect working on the inputs of a control system. Come on, I'm a slow old geezer with a lot to deal with -- it shouldn't be hard to beat me to it.
The next step, of course, is to add reorganization of the output function, as in Demos 7-2 and 8-1. Now we would have the control system acting on many environmental variables at the same time, and all of these variables would be affecting the input quantity through variable weights. We would reorganize these in the way we already know how to do, altering the weights to minimize the average absolute value or square of the error signal. We wouldn't know what aspect of the nervous system is responsible for doing the alterations of output weights, but I assume it is findable. That's tenure right there, isn't it?.
I'll bet that neuroscientists are already doing something a lot like this with their back-propagation perceptron neural nets, except that they're just using this approach to generate classfication-signals in an SR model. We can make a whole control system with it, with continuous variables. The reorganization just establishes normal neural connections, eliminating those that are ineffective in control of the input quantity/perception in question. Once those connections have been established, we can treat the neural signals in the usual way, as frequency-modulated carriers of information.
This is really strange. I started out not believing in Hebbian learning at all, and now I can see it as an example of a reorganizing system.
Whatsay, Henry, and CSG modelers?
Best,
Bill
*Hmm. We need a terminology to distinguish between excitatory signals that increase antagonist outputs, and inhibitory signals that reduce agonist outputs. They both have the same physical effect on the environment.
How about qo+ for agonist output quantity and qo- for antagonist, the sign perhaps written as superscripts? Or written qo(+) when, as here, superscripts aren't available. Later, later.