# Modeling classical conditioning

**URL:** <http://discourse.iapct.org/t/modeling-classical-conditioning/7116>\
**Category:** CSG1995\
**Created:** [March 13, 1995, 5:46am UTC](http://discourse.iapct.org/t/modeling-classical-conditioning/7116 "1995-03-13T05:46:57Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![William\_T\_Powers2](http://discourse.iapct.org/letter_avatar_proxy/v4/letter/w/c67d28/32.png) [@William\_T\_Powers2](http://discourse.iapct.org/u/William_T_Powers2)\
**Post date:** [March 13, 1995, 5:46am UTC](http://discourse.iapct.org/t/modeling-classical-conditioning/7116/1 "1995-03-13T05:46:57Z")

</div>

[From Bill Powers (950312.2145 MST)]  
RE: modeling classical conditioning

Bruce Abbott, Rick Marken, Tom Bourbon, SS Saunders, other modelers:

In the middle of all this programming I've been mulling over just how  
this signalling or SD business ought to work. I have a glimmer of an  
idea now.

It goes back to Wayne Hershberger's analysis of classical conditioning,  
the reference to which escapes me at the moment. Wayne had the insight  
that an unconditional response is due to a built-in control system, the  
unconditioned stimulus being a disturbance and the unconditional  
response being designed to correct the effects of the disturbance (under  
normal or average conditions). What is learned in classical conditioning  
is that some other input predicts that the effect of the disturbance is  
going to happen, so the conditioned stimulus comes to be equivalent to  
the disturbance.

This is a basic modeling problem to which we should have some kind of  
answer when we come up against it. My preliminary thoughts are summed up  
in the following diagram.

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> REF SIG  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\>  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;perceptual V  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;sig ----\> COMP -----\>--  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> \>  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;SUM OUTPUT  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;/ \ FUNCTION  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;/ \ |  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> \> \>  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;INPUT INPUT |  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;FUNCTION FUNCTION |  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> \> V  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> C.V. \<------- ACTION  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> ^  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> \>  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> DISTURBANCE  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> ^  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;SIGNAL |  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;^ delay  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> \>  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;-\<---PRECURSOR

You can discern the usual control loop on the right, including the  
rightmost input function. What has been added is a second input function  
and a summing junction, so the perceptual signal is the sum of an  
external signal input and the usual input representing the controlled  
variable (c.v.).

Also added is a "precursor" to the disturbance. This precursor is some  
physical event that reliably precedes the actual occurrance of the  
change in the disturbing variable. This precursor results immediately  
in, or is, a signal that enters the left-hand input function. Thus the  
signal departs from zero some time before the disturbance occurs.

We assume that the control loop is initially in equilibrium, the  
perceptual signal being just close enough to the reference signal to  
maintain the output that is counteracting the initial level of the  
disturbance -- for convenience, zero.

First leave out the signal pathway. The precursor occurs, and some time  
later the disturbance changes to a new value. This causes an error  
signal which produces a change in output that cancels the effect of the  
disturbance on the controlled variable (c.v.). If the response of the  
system to the disturbance is slow, there will be an appreciable amount  
of error signal for a while, until the action can change enough to  
restore the error to zero.

Now connect the signal pathway. The precursor causes a signal that is  
summed with the perceptual signal from the cv to yield a net perception.  
This perception will depart from the reference level because of the  
signal, and cause an error signal to appear. The output action will  
begin to build up as if toward the level that would counteract a  
disturbance of some particular magnitude and sign.

If the precursor occurs at just the right time in advance of the ensuing  
disturbance, when the disturbance occurs it will find the action already  
beginning to increase in the appropriate direction and with the  
appropriate sign. The error-cancelling action will therefore occur  
sooner than it would without the signal from the precursor, and the  
integrated error will be smaller.

However, if the external signal is still present, the action will be too  
small to correct the error because some of the input is being supplied  
by the external signal instead of the cv. Therefore the perceptual  
signal resulting from the external signal should disappear as the action  
builds up. This suggests that the input function on the left should take  
a time-derivative of the external signal, responding only to its onset.  
An opposite response to the offset might also be useful when the  
disturbance is removed.

Now the question of how this response to the external signal -- the  
"conditioned stimulus" -- is acquired.

The criterion should clearly be minimizing the integrated or average  
error signal. Two parameters of the input function can be reorganized:  
the amplitude response and the time constant of decay. The former is  
adjusted so that action rises at the proper rate to be effective when  
the disturbance begins to rise. The latter is adjusted so that the decay  
constant of the input function matches the rise time of the action.

If the precursor occurs at just the right time prior to the start of the  
disturbance, the reorganization process will match the amplitude and  
time-constant of the left input function to the output action, and the  
error signal will remain exactly at zero (an idealization, of course).

If the precursor occurs too long before the disturbance begins to rise,  
the action will start too soon, and as soon as the input time constant  
decays will actually cause an increase in the error instead of  
correcting it. This will lead to reorganization and lowering of the  
amplitude response to the signal and an increase in the input time  
constant. Similarly, if the precursor occurs too short a time before the  
disturbance, the signal will not produce an error sufficiently in  
advance of the actual disturbance to achieve the optimum timing of the  
opposing action. The error due to the disturbance will add to the error  
due to the external signal, resulting again in a net increase in error.

So there will be an optimal time for the precursor to precede the actual  
start of the disturbance, to allow the system to produce an immediate  
and properly-sized action by the time the disturbance actually starts.  
When many possible precursor events exist, we can expect the system to  
reorganize so as to include the optimally-timed precursor in the signal  
loop.

Finally, the conditioned response and extinction.

If, after reorganization is complete, the external signal is caused to  
occur without the disturbance occurring, the result will be an error and  
an action, the same action that would have been produced to cancel the  
disturbance. The perception due to the external signal will decay at a  
rate calculated to let the action rise and control the cv. However,  
there is no disturbance, so the action will cause a change in the cv  
instead of correcting one. This will lead to a net increase in error  
signal. Eventually, reorganization will alter the input function on the  
left until the signal no longer causes an error by this mechanism, and  
the conditioned response will no longer occur.

> **···**
>
> --------------------------------------------------  
> So I think we have a plausible story to test, and that we can design  
> some simple experiments to test it. This may be a very simple case, but  
> once we are able to model it successfully perhaps we can go on to handle  
> more complicated cases. This should not be hard to set up as an  
> experiment with a human subject.  
> -----------------------------------------------------------------------  
> Best,
> 
> Bill P.