[From Bill Powers (960106.1340 MST)]
Bruce Abbott (960106.1400 EST) --
I see that I recieved your 14:25 post before this one.
You are right that with the proper interpretations, Killeen's model can
be made to match a control model, at least in the outer loop. But his
development is defective. Whenever I see this sort of hole in a
development, I suspect that the final result was intuited (or observed!)
first, and then an attempt was made to find a path from the initial
assumptions to the right answer. This isn't necessarily a bad approach,
but the result shouldn't be published until the mathematics is cleaned
up -- which means in this case fixing the initial assumptions.
(1) A = a*R, where A = arousal level,
a = specific activation
R = rate of incitement
Arousal level is defined as "the amount of responding elicited by a
schedule of incentives in the absence of competition from other
responses." Specific activation is the number of seconds of
behavior incited by one incentive; experimentally it was determined
that, under constant conditions, one incentive would generate a
total of "a" seconds of behavior.
The confusion begins here. If _a_ is seconds of responding per incentive
(reinforcement?), and R is in incentives (inc) per second, then a*R has
units of sec/inc * inc/sec -- it is dimensioness. But arousal level is
defined in units of "amount" of responding (which remains undefined),
which ought to be at least in units of responses, or perhaps responses
per second. So something is fishy in this equation.
(2) B = A/s,
where s (delta in Killeen's paper; my keyboard doesn't seem to have
a delta) is the minimum inter-response time in seconds. s converts
A seconds of responding per second into the rate of discrete
responses under the assumption that each s seconds of responding is
equivalent to one response. Because s is a constant, in most of his
derivations Killeen sets it to 1.0 and thus writes equation 1 as
(3) B = a*R
This is Killeen's "first principle" of behavioral mechanics.
Basically, it says that each delivery of a reinforcer "incites" "a"
seconds of activity, which through learning (not treated here) gets
translated into the instrumental behavior, here represented as
discrete responses (e.g., lever pressing).
The problem here is that a is defined in seconds per incentive and R is
defined in incentives, so the units of behavior are seconds (of
behavior). But this leaves the total number of behaviors undefined,
because the behavior _rate_ is undefined. 10 seconds of behavior at 1
press per second is not the same amount of behavior as 10 seconds of
behavior at 5 presses per second.
Now look at the next equation:
(4) R = B/n, where n is the ratio of responses to incentives.
What is B now? R is number of incentives, n is number of behaviors per
incentive, so B is now in units of number of behaviors. We have quietly
switched from B meaning seconds of behavior (at an undefined rate) to B
meaning number of behaviors (in an unspecified time). Or if R is in
incentives per second, B is in behaviors per second. So B now designates
a different quantity from what was designated by B in the beginning.
If this isn't confusing enough, let's skip to equation 7:
d in this model is viewed as the amount of food in grams below
stomach capacity; Killeen notes that this is a simplification
compared to the variables probably actually contributing to hunger,
but adopts this definition of deprivation for simplicity. Killeen
computes d as follows:
(7) d = d0 + (M - m*R)*t
Here deprivation d starts at d0. It increases with the loss of stomach
contents at the rate M grams/sec, and decreases as food is added at a
rate (m grams/incentive * R incentives per second). Now R is in units of
incentives per second, not number of incentives!
Killeen's concept of "specific activation" is the culprit here. If one
incentive produces 10 seconds of behavior, the number of presses
produced is undefined, because this could mean
>
>
> **********************
Rate |
ยทยทยท
>
>
>
-----------------------*************
time
or
>
>
> *
Rate | *
> * *
> * *
> * *
>*
----------------------**************
time
...or any other pattern of behavior rates that lasts 10 seconds. This is
particularly problematic when ratio schedules are used, because the
number of behaviors is always exactly n times the number of
reinforcements, no matter how much time elapses. So the idea of "seconds
of behavior" is completely gratuitous and has no connection to the later
equations.
The upshot of all this is that Killeen's equation 7 is completely
unjustified. There is no legitimate sequence of mathematical steps that
will get from equation 1 to equation 7. I haven't even mentioned the
blunder in equation 2, where the minimum response interval is thrown in
as a divisor for no discernible reason (leading to still another
definition of B), and then is ignored, leaving equation 3 orphaned!
What I am tempted to suspect is that Killeen _started_ with equation 7,
and then tried to dream up some basic principles of "mechanics of
behavior" that would lead to it. But in doing so, he simply pushed
symbols around without thinking about what they mean, so their
definitions kept shifting in just the way that would allow the final
equation to be "deduced." The term "specific activation" sounds very
scientific, and it should since it is lifted from chemistry, but in fact
it is nonsense.
Finally, you will notice that the final equations in the table you
present define B and R both as rates: behaviors per second and responses
per second. But this brings us right back to the Abbott Effect: for all
we know, rate of responding is not a function of reinforcement rate or
incentive rate in ANY experiment. The apparent dependence comes at least
in part and perhaps totally from the effect of collection time. Since
collection time does not appear in these equations, any fit to the data
is spurious.
Back to the drawing board.
The effect of reinforcements on loop gain is similar to what I assumed
in trying to build a model of the Motheral curves. If you assume that
the gain of the system increases with reinforcement rate, the
reinforcement rate shouldn't produce runaway behavior, because the limit
of the behavior rate for infinite gain should simply be the rate that
makes the food loading exactly match the reference level. In a
simulation, you have to watch out for computational oscillations when
the gain gets very large; you may have to reduce the size of dt to
prevent them. Computational oscillations can create the appearance of a
sudden runaway condition. The giveaway is that the period of the
oscillations is exactly 2 iterations. When that occurs, you know that
the simulation isn't working right.
-----------------------------------------------------------------------
Best,
Bill P.