Killeen: memory and coupling

[From Bruce Abbott (960118.1110 EST)]

In this post I continue my analysis of Killeen's "behavioral mechanics" as
presented in his 1994 _Behavioral & Brain Sciences_ paper. Last time I
described an equation for M, a variable in Killeen's system representing the
animal's memory of some attribute of its most recent responses. Killeen
refers to those responses that contribute toward the completion of the
schedule requirement as "target" responses. "Nontarget" responses may also
occur and are taken into account in the model. For now, however, I will
assume that all responses being produced are target responses (e.g., the rat
is busily lever-pressing). The animal's short-term memory of the relevant
response attribute after the nth response since reinforcement, M(n) is
computed iteratively as

(1) M(n) = beta*y(n) + (1 - beta)*M(n-1), 0 <= beta <= 1

Beta is the "currency parameter"; when beta is 1.0 only the most recent
response "counts." When beta is, e.g., 0.25, then the most recent response
gets weighted in memory 25% and residual memory of previous responses 75%.
Y(n) is the value of the "relevant attribute" of response n. Killeen uses
interresponse time (IRT) as an example of a relevant attribute. Each time a
new response occurs, equation 1 is iterated and the value of previous
responses [M(n-1)] gets multiplied by (1- beta). If beta = 0.25 then this
means that with each new response the old value in memory becomes only 75%
of its former value before the contribution of the new response is added in.
Equation 1 represents a discrete version of a "leaky integrator"; the
integration takes place over responses and NOT over time. Thus if the rat
makes one response and then just sits there, the value of that response as
represented in memory does not decay. However, each subsequent response
will reduce the representation of that first response in memory to the
proportionate amount given by (1 - beta). The current value in memory, M,
is the exponentially weighted average of all responses since the first, with
the most recent receiving the highest weight. Killeen describes a procedure
for estimating the value of beta for a given animal.

One application of equation 1 in Killeen's model involves setting y(n) equal
to 1.0 for each target response and to 0.0 for each nontarget response. If
all responses are target responses, then equation 1 reduces to a simpler
form which may be approximated by the formula

(2) M(n) = 1 - exp(-lamda*N),

where lamda is the measured rate of decay from short term memory. Later in
the paper (p. 114), Killeen notes that lamda covaries strongly with the
estimated response duration, delta (r = 0.80). Because longer-duration
responses would occupy a greater proportion of memory, Killeen proposes
making lamda a function of delta as follows:

(3) lamda = lamda'*delta/rho,

where lamda' is the "intrinsic rate of decay from memory per second," delta
is the response duration, and rho is the proportion of memory occupied by
target responses. Table 2 in the paper gives examples of lamda' and delta
for several experiments; for Powell (1968), lamda' is 0.38 and delta is
0.28. Thus for this study,

     lamda = 0.38 * 0.28 / 1.0 = 0.11, (0.1064 before rounding)

if rho is fixed at 1.0.

Killeen gives the following equation for average response rates on fixed
ratio schedules:

         zeta N
(4) B = ----- - -, where
         delta a

(5) zeta = rho(1 - exp(-lamda*N).

Zeta is the "coupling coefficient" that takes account of the degree of
"coupling" of incentives to responses; its formula differs depending on the
schedule of reinforcement. Note also that if rho = 1.0, then zeta = M(n) as
given in equation 2 above, with n = N, the ratio requirement.

In the Powell (1968) study, a was given as 123 sec/reinf, and we calculated
lamda above. For an FR-25 schedule,

    zeta = 1.0[1 - exp(-0.1064*25)] = 1 - 1/exp(2.66) = 1 - 1/14.3

         = 0.93.

Plugging the numbers into equation 4 gives

    B = (0.93/0.28) - (25/123) = 3.32 - 0.20 = 3.2 resp/sec,

which is close to the observed value in the top panel of Figure 7.

Coupling (zeta) on fixed ratio schedules increases with N and approaches
1.0. For small N's it can have a large effect on the predicted rates. For
example, if N = 5, then

    zeta = 1.0[1 - exp(-0.1064*5)] = 1 - 1/exp(0.532) = 1 - 0.59 = 0.41.

This reduces the predicted response rate in Powell's (1968) experiment to

    B = (0.41/0.28) - (25/123) = 1.46 - 0.20 = 1.26 resp/sec.

So how is the formula for zeta derived for a given schedule? That's a bit
complicated, so I'll simply note that Killeen illustrates the procedure for
several schedules in Appendix C.

At this point we have enough eccentrics and epicycles to, as one reviewer
(Wearden) put it, "account for a respectable proportion of the variance in a
relief map of the Himalayas" (p. 155). What I plan to do with all this is
develop a simulation program that embodies Killeen's model for fixed ratio
schedules, if I can figure out how to represent it in real time. Several of
the formulas he presents are meant for application to available data in the
form of session averages, whereas the simulation will require the iterative
formulas whose effects over time give the results provided by Killeen's
equations.

Regards,

Bruce