[From Bill Powers (950312.2145 MST)]
RE: modeling classical conditioning
Bruce Abbott, Rick Marken, Tom Bourbon, SS Saunders, other modelers:
In the middle of all this programming I've been mulling over just how
this signalling or SD business ought to work. I have a glimmer of an
idea now.
It goes back to Wayne Hershberger's analysis of classical conditioning,
the reference to which escapes me at the moment. Wayne had the insight
that an unconditional response is due to a built-in control system, the
unconditioned stimulus being a disturbance and the unconditional
response being designed to correct the effects of the disturbance (under
normal or average conditions). What is learned in classical conditioning
is that some other input predicts that the effect of the disturbance is
going to happen, so the conditioned stimulus comes to be equivalent to
the disturbance.
This is a basic modeling problem to which we should have some kind of
answer when we come up against it. My preliminary thoughts are summed up
in the following diagram.
> REF SIG
>
perceptual V
sig ----> COMP ----->--
> >
SUM OUTPUT
/ \ FUNCTION
/ \ |
> > >
INPUT INPUT |
FUNCTION FUNCTION |
> > V
> C.V. <------- ACTION
> ^
> >
> DISTURBANCE
> ^
SIGNAL |
^ delay
> >
-<---PRECURSOR
You can discern the usual control loop on the right, including the
rightmost input function. What has been added is a second input function
and a summing junction, so the perceptual signal is the sum of an
external signal input and the usual input representing the controlled
variable (c.v.).
Also added is a "precursor" to the disturbance. This precursor is some
physical event that reliably precedes the actual occurrance of the
change in the disturbing variable. This precursor results immediately
in, or is, a signal that enters the left-hand input function. Thus the
signal departs from zero some time before the disturbance occurs.
We assume that the control loop is initially in equilibrium, the
perceptual signal being just close enough to the reference signal to
maintain the output that is counteracting the initial level of the
disturbance -- for convenience, zero.
First leave out the signal pathway. The precursor occurs, and some time
later the disturbance changes to a new value. This causes an error
signal which produces a change in output that cancels the effect of the
disturbance on the controlled variable (c.v.). If the response of the
system to the disturbance is slow, there will be an appreciable amount
of error signal for a while, until the action can change enough to
restore the error to zero.
Now connect the signal pathway. The precursor causes a signal that is
summed with the perceptual signal from the cv to yield a net perception.
This perception will depart from the reference level because of the
signal, and cause an error signal to appear. The output action will
begin to build up as if toward the level that would counteract a
disturbance of some particular magnitude and sign.
If the precursor occurs at just the right time in advance of the ensuing
disturbance, when the disturbance occurs it will find the action already
beginning to increase in the appropriate direction and with the
appropriate sign. The error-cancelling action will therefore occur
sooner than it would without the signal from the precursor, and the
integrated error will be smaller.
However, if the external signal is still present, the action will be too
small to correct the error because some of the input is being supplied
by the external signal instead of the cv. Therefore the perceptual
signal resulting from the external signal should disappear as the action
builds up. This suggests that the input function on the left should take
a time-derivative of the external signal, responding only to its onset.
An opposite response to the offset might also be useful when the
disturbance is removed.
Now the question of how this response to the external signal -- the
"conditioned stimulus" -- is acquired.
The criterion should clearly be minimizing the integrated or average
error signal. Two parameters of the input function can be reorganized:
the amplitude response and the time constant of decay. The former is
adjusted so that action rises at the proper rate to be effective when
the disturbance begins to rise. The latter is adjusted so that the decay
constant of the input function matches the rise time of the action.
If the precursor occurs at just the right time prior to the start of the
disturbance, the reorganization process will match the amplitude and
time-constant of the left input function to the output action, and the
error signal will remain exactly at zero (an idealization, of course).
If the precursor occurs too long before the disturbance begins to rise,
the action will start too soon, and as soon as the input time constant
decays will actually cause an increase in the error instead of
correcting it. This will lead to reorganization and lowering of the
amplitude response to the signal and an increase in the input time
constant. Similarly, if the precursor occurs too short a time before the
disturbance, the signal will not produce an error sufficiently in
advance of the actual disturbance to achieve the optimum timing of the
opposing action. The error due to the disturbance will add to the error
due to the external signal, resulting again in a net increase in error.
So there will be an optimal time for the precursor to precede the actual
start of the disturbance, to allow the system to produce an immediate
and properly-sized action by the time the disturbance actually starts.
When many possible precursor events exist, we can expect the system to
reorganize so as to include the optimally-timed precursor in the signal
loop.
Finally, the conditioned response and extinction.
If, after reorganization is complete, the external signal is caused to
occur without the disturbance occurring, the result will be an error and
an action, the same action that would have been produced to cancel the
disturbance. The perception due to the external signal will decay at a
rate calculated to let the action rise and control the cv. However,
there is no disturbance, so the action will cause a change in the cv
instead of correcting one. This will lead to a net increase in error
signal. Eventually, reorganization will alter the input function on the
left until the signal no longer causes an error by this mechanism, and
the conditioned response will no longer occur.
···
--------------------------------------------------
So I think we have a plausible story to test, and that we can design
some simple experiments to test it. This may be a very simple case, but
once we are able to model it successfully perhaps we can go on to handle
more complicated cases. This should not be hard to set up as an
experiment with a human subject.
-----------------------------------------------------------------------
Best,
Bill P.