[From Bruce Abbott (960116.1100 EST)]
Bill Powers pointed out to me yesterday (private post) that the modification
of the output function used to introduce slowing in our digital computations
is basically a low-pass filter or r-c circuit, and this reminded me that
Killeen had said the same about his equation for representing short-term
memory of responses (equation 2, p. 106 in BBS). The equation is
M(n) = beta*y(n) + (1 - beta)*M(n-1), 0 <= beta <= 1.0
Multiplying out and rearranging:
M(n) = beta*y(n) + M(n-1) - beta*M(n-1)
= M(n-1) + beta*[y(n) - M(n-1)]
which is our usual iterative output function for digitally simulating a
control system with output o = M and a slowing factor of beta.
Killeen calls beta the "currency parameter." y(n) is set to some
"attribute" of the response (Killeen uses the time since previous response,
or IRT in an example). If M is initialized at zero, then the first response
following reinforcement would have an IRT of y(n); the equation would be
iterated and M would now have a value of beta*y(n). When the next response
occurred, its IRT would be entered as y(n) and the equation iterated again,
and so on. If beta is set to 0.25, then after each response,
M := 0.25*y + 0.75*M;
Killeen would say that memory weights the current response 0.25 and the sum
of the previous responses 0.75, and that the value of M is the current
"representation" of the response attribute in memory. He refers to the
equation as embodying an "exponentially-weighted moving average" (EWM). For
IRT, M at any given point would be the exponentially-weighted average IRT.
In Killeen's system, this memory representation is involved in the learning
process that associates responses with incentives. [Up to now in my
description of Killeen's system, I have dealt only with equations describing
asymptotic performance.]
Another equation of interest in Killeen's system is the one he uses to
adjust predicted response rates to take "ceiling constraints" into account
-- the fact that the minimum interresponse time, delta, imposes a maximum of
1/delta responses per second on the response rate. The equation is:
B' = B/(1 + delta*B),
where B' is the adjusted value of B.
At first I was a bit puzzled by this equation, so I began to fiddle with it
to see what might materialize:
B B/B 1
B' = ------------- = ---------------- = -------------
(1 + delta*B) (1/B + delta*B/B) (1/B + delta)
But 1/B' is just the observed IRT (interresponse time) and delta is just the
minimum IRT. So
1 1/B + delta
IRT' = - = ----------- = IRT + delta
B' 1
In other words, the observed IRT equals the theoretical IRT plus the minimum
response duration. Killeen is assuming that his basic equation for
predicting response rate from reinforcement rate counts time from the end of
one response to the start of the next, so that the minimumum response
duration (delta) must be added to the predicted IRT to get the actual one.
To my mind, this is a clumsy way to deal with the limitations imposed by the
fact that responses have finite duration; it seems odd that the basic
equation for response rate would not predict the actual IRT directly and
impose any minimum IRT limit by taking account of the actual dynamics
involved in responding.
Killeen plugs this equation into his basic equation, B = [a/delta]*R, to get
R*delta
B = -------
R + 1/a
However, there is another way to look at this. Inverting the basic equation
gives:
delta
IRT = -----
a*R
Substuting my IRT-equivalent of the response-constraint equation gives
delta
IRT' - delta = -----;
a*R
delta
IRT' = delta + ----- = delta*[1 + 1/a*R]
a*R
But 1/R is the inter-incentive time (IIT), so
1
IRT' = delta*[1 + -*IIT]
a
This has the form of a linear function with an intercept of delta and a
slope of delta/a:
IRT' = delta + (delta/a)*IIT
Delta is the minumum IRT; what is delta/a? Well, a is the number of seconds
of responding incited by each incentive delivery; a/delta = B, the number of
responses per second of duration delta in a seconds of responding, so
delta/a is just 1/B, the predicted IRT prior to the adjustment for minimum IRT:
IRT' = delta + IRT*IIT
where IRT = delta/a
So Killeen's rather complicated-looking hyperbolic function relating
response rate B to reinforcement rate R can be reexpressed to use IRT and
IIT instead, making IRT' a simple linear function of inter-incentive time,
and a simple inverse function of the parameter a.
If we move delta over to the left side of the equation we get
IRT'/delta = 1 + (1/a)*IIT
The term on the left tells us how many times larger than delta the observed
IRT is predicted to be. For example, if it is 2.0 then the observed IRT
will be twice delta. It is a linear function of inter-incentive time, with
an intercept of 1.0 and slope 1/a.
Regards,
Bruce