Killeen's model

CSGnet Archive CSG1996
1

[From Bruce Abbott (960107.1200 EST)]

Bill Powers (960106.2230 MST) [Private Post]

Bill and I have been discussing the Killeen model privately for a couple of
posts trying to work out Killeen's units of measure to see whether they make
sense, but Bill raises an issue in his last post to me that should be
discussed on CSGNET so everyone can be in on the developments.

OK, let me work through this. I'll try not to add any new mistakes in
correcting the old ones. Actually as I review the bidding, I find a lot
more to complain about, even though some of my comments about units were
not right as stated. You can post this if you want.

    B responses/sec = (a response-sec/inc)(R inc/sec)/s resp-sec/resp
    A response-sec is the number of seconds of responding.

I don't know where you got the units for s; s is defined as the minimum
time per response, so its units should be seconds per response, not, as
you wrote, response-sec per response which is equivalent to just sec. We
have, in terms of the units alone,

resp/sec = resp * sec / inc * inc / sec / sec * resp = resp^2/sec.

Check that out. These are not the right units for B. The only way to fix
this is to change the units of a to sec/incentive.

A "response-second" would be the number of seconds of responding generated
by one incentive. Going back to my gasoline engine analogy, it would be the
number of seconds the engine would run after a spurt of gasoline from the
pump, which would be proportional to the amount of gasoline in the spurt.

Killeen is evidently taking the minimum interresponse time as an estimate of
the number of response-seconds required to make one response (at maximum
rate of responding). Dividing a by s gives:

      a response-seconds/incentive
      ---------------------------- = b responses/incentive
      s response-seconds/response

Multiplying b*R, we get

            responses incentives
      B = b --------- * R ---------- = B responses/second
            incentive second

You may not agree with Killeen's way of converting response-seconds to
number of responses, but the units come out right.

But let's pass over these problems and take a different look at the
equation

(3) B = a*R/s, (correct form; B = a*R is incorrect).

From (4) we have

    (4) R = B/n, where n is the ratio of responses to incentives.

From this, we have

a/s = n.

Plugging this result into (1) we get

B = n*R, which is simply a permutation of equation (4).

So this whole exercise up to this point has been nothing but wheeling
and dealing with the equation that describes the ratio.

Yep.

What's wrong here is excruciatingly hard to explain. Look at the
sequence of steps in Killeen's column:

. . . .

g = p*R
B = (p*R)*e

which is exactly equivalent to Killeen's equation, with p = k, and e =
d.

Yes. (See my revised table.)

So if we just ignore everything Killeen says about "specific activation"
and "minimum time per response" and "seconds of responding per
incentive" we can show that his model is a control system model in which
the gain of the output function is assumed to be proportional to the
reinforcement rate, with the rest of the loop being a normal control
system.

So we agree that Killeen's model involving deprivation-level is a
control-system model. Good! There doesn't seem to be anything "wrong" with
that part.

Words like "arousal" and "incentive" and "value in response-
seconds" and "specific activation" are superflous, not to mention
pretentious, being only names for unnecessary intervening variables
which can't be independently observed or measured.

This is what is wrong that is "excruciatingly hard to explain"?

Killeen's does get empirical estimates of a, and claims to have shown that a
given incentive produces a constant a-seconds of "activity." I haven't seen
the paper in which he explored this. He states that not all the
activity-time incited by the delivery of an incentive is necessarily
expended producing instrumental responses, an issue he presumably develops
further in his earlier paper. Thus a is not always redundant with b. In
the current paper Killeen is focusing on expanding his 1994 model to take
account of changes in deprivation level during the session, so he does not
show us here why he thinks these other intervening variables are necessary
for the more general account. In this simple case, Killeen notes that
several of them can be "absorbed" into a single parameter; thus Killeen
agrees they are superflous in this context.

By the way, don't take me to be agreeing with Killeen's constructs. I don't
much care for "specific activation," "arousal," and his conversion from
arousal to responses either. I'm only pointing out that he has given them
mathematical definitions from which their values can be computed, and has
asserted that the key value (specific activation) can be (and has been)
estimated empirically.

I don't think the reinforcement loop is a positive feedback loop, since
even with infinite gain, the level of behavior and reinforcement would
not go to infinity. As I indicated earlier, if you seem to see a runaway
effect, it is probably due to computational oscillations, an artifact of
trying to do analog computations on a digital computer. But I'll check
this out with your program.

This is the part of my analysis with which I am most uncomfortable. My
original simulation (using B and R directly) didn't go into computational
oscillations, it just flat out crashed, due to numerical overflow. But
perhaps that was due to a poor choice of initial parameters, I don't know.
As I noted, the posted program (FOOD2.PAS) converts Killeen's rates to
amounts in order to keep the number of response-seconds generated in an
iteration no greater than the number of seconds available in which to expend
them (i.e., dt) in an iteration. I'm uncertain about this approach, and I
don't have Killeen's more complete 1994 description of his model yet to help
me decide whether it is what Killeen has in mind. If using the rates
directly will work (as your analysis of Killeen's model seems to suggest),
this would certainly be the preferable approach to the simulation. But see
below.

I am unclear why you think that B and R are not related in a positive
feedback loop. In Killeen's earlier model, he does not deal with the effect
of hunger on responding, so the negative feedback loop through food-level is
absent. If you run the model with this loop open, you have

B = k*R, because e is now constant. Substituting B/n for R, we have

               k
B = k*B/n = B*---
               n

for each dt. If k/n > 1 and B0 > 0, doesn't B grow exponentially?

Obviously one of us isn't seeing the problem correctly. I think you may
have been misled by an apparent similarity in the effect of R on gain in
your model and in Killeen's. In your control model that made g proportional
to the rate of reinforcement, the output of the system was rate of behavior,
the controlled variable was rate of reinforcement, the reference was a rate
of reinforcement, and so was the error. They were all rates. In Killeen's
model, the output is a rate of behavior, which through the environment
function produces a rate of reinforcement, but this then contributes to an
_amount_ of food, which is compared to a reference _amount_ of food to
produce an error in _amount_ of food, all integrals of the rate of
reinforcement over time.

Your analysis of the effect of R on the loop gain assumes that R is the
controlled varible. In Killeen's model it is not. In the open food-loop
mode, B and R increase exponentially with time if k/n > 1. In the
closed-loop mode, the reduction in error due to input of food is very small
relative to the positive feedback of R on B, leading to a net positive
feedback in most cases in the inner loop (but not in the outer one, where
higher R only produces faster convergence of F on Fr).

Regards,

Bruce

2

[From Bill Powers (960107.1230 MST)]

Bruce Abbott (960107.1200 EST) --

Good idea to keep this public.

     A "response-second" would be the number of seconds of responding
     generated by one incentive. Going back to my gasoline engine
     analogy, it would be the number of seconds the engine would run
     after a spurt of gasoline from the pump, which would be
     proportional to the amount of gasoline in the spurt.

What this leaves out is how _fast_ the engine would be running during
the response time. After the spurt of gasoline, the engine might go
chug-chug ---chug --------chug -----------------chug and quit after 10
seconds of running. Or it might whee up to 6000 RPM and then slow down
to a final chug--chug and quit, also in 10 second of running. So a
running-second doesn't mean much in terms of what the engine actually
does.

     Killeen is evidently taking the minimum interresponse time as an
     estimate of the number of response-seconds required to make one
     response (at maximum rate of responding).Dividing a by s gives:

           a response-seconds/incentive
           ---------------------------- = b responses/incentive
           s response-seconds/response

Why "number of response-seconds" rather than just "number of seconds?"
You get the same result if you just say

      a seconds/incentive
      -------------------- = b responses/incentive
      s seconds/response

Note that responses/incentive is simply n, the scheduled ratio. b = n.

There's an issue about how units are defined. The minimum interresponse
time is just that -- a time. We refer to it as a response time only to
identify _which_ time we are measuring; whatever it is, we measure it in
seconds. A response-second is like a foot-pound: it is the number of
responses TIMES the number of seconds of responding. That is not what is
intended here. The only units needed are seconds.

     Multiplying b*R, we get

                 responses incentives
           B = b --------- * R ---------- = B responses/second
                 incentive second

     You may not agree with Killeen's way of converting response-seconds
     to number of responses, but the units come out right.

Since b = n, b*R = n*R, and of course n*R = B (or R = B/n). This simply
repeats the definition of the schedule.

The units also come out right if a and s both employ seconds rather than
response-seconds. Look, if we say that the area of a rectangle is height
times width in inches, we don't have to say that it's height-inches
times width-inches. That does not change the fact that the units of
measurement are inches. In fact, saying height-inches would imply that
height-inches are different from width-inches; we are multiplying height
units by inch units, as in talking about a drawing with different x and
y scalings. So saying response-seconds implies that seconds are
different in duration when we are talking about responses. But they
aren't: we're just talking about the same old time units, seconds.

     Killeen is evidently taking the minimum interresponse time as an
     estimate of the number of response-seconds required to make one
     response (at maximum rate of responding).

I agree, but the tautology goes deeper than that. Suppose that the ratio
is n and the minimum time per response is s. Then the minimum time
required to execute n responses is Tmin = n*s. But from B = a*R/s and R
= B/n, we can see that a = n*s. The specific activation factor _a_ is
simply Tmin, the minimum time taken to produce n responses at the
minimum interresponse interval s. It is not an "estimate" of Tmin; given
s, it is _exactly_ Tmin. That is, it is exactly n*s. Since every n
responses produce one incentive, the specific activation factor is
therefore also the minimum time between incentives. When you divide the
minimum time for n responses by the minimum time for 1 response (pair of
responses), you end up with n*s/s, or simply n. And of course n*R = B.

So all of this is just a tight little circle of computations that proves
nothing but a = a, or 0 = 0. But because of the snarl of computations
and definitions he gets into, Killeen never recognizes that his so-
called specific activation factor is nothing but the minimum time
required to execute the scheduled number of responses at the maximum
possible behavior rate. Dividing this by the minimum interresponse
interval recovers the value n, and that is all that is going on here.
The only function of dividing by the minimum interresponse interval is
to convert a into n and thus to remove the minimum interresponse
interval from having any effect in the equations. He assumed a dummy
variable and then canceled it out.

Obviously, Killeen has it all worked out from the behaviorist point of
view, in which reinforcements cause responses. A jolt of incentive
occurs, and results (as he sees it) in behavior that goes on for some
time. He is separating a sequence of events that goes

   RbbbbbRbbbbbRbbbb...

into chunks like this:

   Rbbbbb Rbbbbb Rbbbbb

Of course the apparatus is actually working this way:

  bbbbbR bbbbbR bbbbbR

Whatever the behavior rate is (and the equations do not reveal that),
the time interval taken to execute n responses is simply n/B, and the
rate of reinforcement is simply R = B/n. We have never left the single
equation that describes the apparatus.

···

---------------------------------------------
     So we agree that Killeen's model involving deprivation-level is a
     control-system model. Good! There doesn't seem to be anything
     "wrong" with that part.

You mean, despite the fact that it doesn't fit the data? But you're
right, Killeen did take one giant step when he dropped all the guff
about specific activation factors and started trying to guess at
processes going on inside the organism. With respect to PCT, that brings
him up to about 1956, or at best, 1973. However, you're the bridge
builder, so you can certainly say that Killeen ended up with a control-
system model even if he didn't know what it was. But I hope you will
find some way to remark that his "mechanics of behavior" model can be
disregarded.
-----------------------------------------------------------------------
Best,

Bill P.

3

[From Bruce Abbott (960108.0900 EST)]

Bill Powers (960107.1230 MST) --

    Bruce Abbott (960107.1200 EST)

    A "response-second" would be the number of seconds of responding
    generated by one incentive. Going back to my gasoline engine
    analogy, it would be the number of seconds the engine would run
    after a spurt of gasoline from the pump, which would be
    proportional to the amount of gasoline in the spurt.

What this leaves out is how _fast_ the engine would be running during
the response time. After the spurt of gasoline, the engine might go
chug-chug ---chug --------chug -----------------chug and quit after 10
seconds of running. Or it might whee up to 6000 RPM and then slow down
to a final chug--chug and quit, also in 10 second of running. So a
running-second doesn't mean much in terms of what the engine actually
does.

Yes, I agree.

    Killeen is evidently taking the minimum interresponse time as an
    estimate of the number of response-seconds required to make one
    response (at maximum rate of responding).Dividing a by s gives:

          a response-seconds/incentive
          ---------------------------- = b responses/incentive
          s response-seconds/response

Why "number of response-seconds" rather than just "number of seconds?"
You get the same result if you just say

     a seconds/incentive
     -------------------- = b responses/incentive
     s seconds/response

Well, it has to do with the fact that you said that response-seconds/second
would be unitless, because the seconds in the numerator and denominator
cancel. This may be technically true but is misleading: it is a rate (at
which seconds of potential responding accumulate) whose denominator is
expressed in seconds.

Note that responses/incentive is simply n, the scheduled ratio. b = n.

Yes, that is an implication of Killeen's mechanics when applied to ratio
schedules. (Ouch!)

There's an issue about how units are defined. The minimum interresponse
time is just that -- a time. We refer to it as a response time only to
identify _which_ time we are measuring; whatever it is, we measure it in
seconds. A response-second is like a foot-pound: it is the number of
responses TIMES the number of seconds of responding. That is not what is
intended here. The only units needed are seconds.

Yes, I know. It's Killeen's term, not mine. However, I took care to define
it so that any possible confusion would be avoided: I said that the number
of response-seconds was the number of seconds of responding. The only units
needed are seconds, but it is still important to keep seconds of (potential)
responding logically distinct from seconds of real time.

    Multiplying b*R, we get

                responses incentives
          B = b --------- * R ---------- = B responses/second
                incentive second

    You may not agree with Killeen's way of converting response-seconds
    to number of responses, but the units come out right.

Since b = n, b*R = n*R, and of course n*R = B (or R = B/n). This simply
repeats the definition of the schedule.

In the appendix to Killeen's paper, Killeen presents the following as "a
fundamental equation of motion" for ratio schedules:

     v*h = s*n

But v*h = a and a/s = b, so b = n. s*n is described as "the number of
response-seconds [v*h] is required to sustain." I presume this means that
v*h must produce at least n responses if schedule performance is to be
maintained on a ratio schedule. If v*h/s = fewer than n responses, then the
ratio is not completed, the next incentive is not delivered, and responding
ceases (unless the missing response is supplied via the baseline rate of
behavior).

In Killeen's hunger model, h declines as responding supplies food and thus
decreases the level of deprivation. Since v and s are fixed quantities, it
is certainly possible that v*h/s will not equal n.

The units also come out right if a and s both employ seconds rather than
response-seconds. Look, if we say that the area of a rectangle is height
times width in inches, we don't have to say that it's height-inches
times width-inches. That does not change the fact that the units of
measurement are inches. In fact, saying height-inches would imply that
height-inches are different from width-inches; we are multiplying height
units by inch units, as in talking about a drawing with different x and
y scalings. So saying response-seconds implies that seconds are
different in duration when we are talking about responses. But they
aren't: we're just talking about the same old time units, seconds.

In my engine analogy, imagine that the pump is spurting the gasoline into
the carburetor float bowl rather than directly into the engine. As the
engine runs, the gasoline gets pumped into the bowl faster than it is being
consumed. If we measure the gasoline in terms of number of seconds of
engine-running, then gasoline is being added at a rate of x seconds of
(potential) running per second. After t seconds, you have x*t seconds of
running-time in the bowl. It is true that rate at which x increases is in
seconds/second, which is unitless, but retaining seconds/second helps keep
what is going on conceptually clear. By labeling the top quantity
"response-seconds," Killeen is just trying to help the reader to understand
what the model is saying: seconds of responding accumulating at some rate.

    Killeen is evidently taking the minimum interresponse time as an
    estimate of the number of response-seconds required to make one
    response (at maximum rate of responding).

I agree, but the tautology goes deeper than that. Suppose that the ratio
is n and the minimum time per response is s. Then the minimum time
required to execute n responses is Tmin = n*s. But from B = a*R/s and R
= B/n, we can see that a = n*s. The specific activation factor _a_ is
simply Tmin, the minimum time taken to produce n responses at the
minimum interresponse interval s. It is not an "estimate" of Tmin; given
s, it is _exactly_ Tmin. That is, it is exactly n*s. Since every n
responses produce one incentive, the specific activation factor is
therefore also the minimum time between incentives. When you divide the
minimum time for n responses by the minimum time for 1 response (pair of
responses), you end up with n*s/s, or simply n. And of course n*R = B.

This would be true if a were computed from n*s, but I take it that it is
derived empirically. I'll have to see how Killeen arrives at this empirical
value before I know whether I can agree with you or not. If the method
always yields a/s = n then I will agree. If it yields other values, then I
will have to ask how Killeen's model deals with the excess or shortfall.

    So we agree that Killeen's model involving deprivation-level is a
    control-system model. Good! There doesn't seem to be anything
    "wrong" with that part.

You mean, despite the fact that it doesn't fit the data?

By "nothing wrong" I mean that it does indeed constitute a control system.
I recall your having said in a post a few weeks ago that if Killeen's model
contained a control system, it was pretty well hidden. Whether it fits any
data is another matter.

But you're
right, Killeen did take one giant step when he dropped all the guff
about specific activation factors and started trying to guess at
processes going on inside the organism. With respect to PCT, that brings
him up to about 1956, or at best, 1973. However, you're the bridge
builder, so you can certainly say that Killeen ended up with a control-
system model even if he didn't know what it was.

Killeen may just be opening the way for us in EAB. He is a well-established
and respected member of the EAB research community, so his proposals will be
taken seriously. He is arguing for the kind of approach you have been
championing for lo these many years: generative models whose structures,
when coupled with particular environment functions and given starting
parameters, produce behavior.

But I hope you will
find some way to remark that his "mechanics of behavior" model can be
disregarded.

If Killeen's model is taken seriously in EAB then I can't just disregard it.
My strategy is first to understand it, then to expose its problems and
Killeen's mistakes in analyzing his own model, then to develop an
alternative PCT model which can be directly compared to Killeen's. My
food-level control model represents the first stage in the development of
that alternative model.

By the way, have you worked out whether the reinforcement loop in Killeen's
model has positive or negative feedback? I note that your reply to my post
ended before you dealt with that issue.

Regards,

Bruce

4

[From Bruce Abbott (950109.0930 EST)]

Bill Powers (950609.1545 MST) --

    Bruce Abbott (960108.0900 EST)

>Why "number of response-seconds"...?

    Well, it has to do with the fact that you said that response-
    seconds/second would be unitless, because the seconds in the
    numerator and denominator cancel. This may be technically true but
    is misleading: it is a rate (at which seconds of potential
    responding accumulate) whose denominator is expressed in seconds.

I get the idea, but this is not in the mathematics. If you want to
propose this idea as part of the model, then you have to express it
explicitly in terms of variables and functions relating them.
"Technically" true means mathematically true, doesn't it? I presume we
want the mathematics to be correct.

Ah, but it IS in the mathematics; that's what I've been trying to show you:

a = seconds (of responding) incited per incentive = sec/inc
            sec (of responding) incentives sec (of responding)
A = a*R = a ------------------- * R ---------- = A -------------------
                 incentive second second
          A sec/sec
B = A/s = ---------- = B resp/sec
          s sec/resp

The problem with this set of equations is that Killeen doesn't provide a
mechanism in them for dealing with the cumulating seconds of responding. I
suspect he provided such a mechanism in his 1994 paper but omitted
discussing it in the 1995 paper to simplify the presentation, which focused
on expanding the model to handle changes in deprivation over the
experimental session. I won't know until I see the 1994 paper.

Perhaps what you want to say that there is a quantity that builds up
with every incentive, and that it can produce responses for some time
after the incentive (God, these words!). We can say that the amount of
the quantity is Q, and that it increases by m with each incentive. Q
also decays at some rate, which we can call c. So with R and B in terms
of rates,

dQ = (m*R - c)*dt, and

B = Kb*Q
where Kb is an arbitrary constant expressing responses/sec per unit of
Q.

Now if m is large enough, a single jolt of reinforcer will produce
enough Q so that the behavior rate does not fall to zero for some time
afterward, and multiple responses occur.

I believe this is what Killeen had in mind when he proposed that each
incentive produces a-sec of "activity." Whether something like your
suggested mechanism is contained in his more detailed 1994 description
remains to be seen.

Obviously, introducing this hypothesis is going to change the model and
the predicted behavior. That's OK if you want to do it; the important
test is whether the resulting model matches observations. If you want to
propose a different meaning for what you say, go ahead -- but put it
into the model rather than alluding to it as a background idea. If
"Potential responding" has any signficance, it belongs in the model.
There is no "potential responding" in Killeen's model as it stands.

Now hold on. _I_ don't want to put ANYTHING into Killeen's model that isn't
there. It's his model, not mine, and I'm not out to modify it. If it turns
out that the detailed model presented in 1994 lacks such a mechanism, I
might want to point out that it needs something like this if it is to work
at all, but it is not the model I wish to pursue.

    In Killeen's hunger model, h declines as responding supplies food
    and thus decreases the level of deprivation. Since v and s are
    fixed quantities, it is certainly possible that v*h/s will not
    equal n.

If so, there is a mistake in the mathematics, such as a definition that
contradicts another definition (subtly). The equations as presented
propose that v*h/s = n. Since we also have already that a = n*s, it
follows that v*h = a. From other equations we have

(6) h = Y*d, where Y (gamma in Killeen's paper) is just a constant of
              proportionality.

so now _a_ is being proposed to depend on the level of deprivation.

Yes, that was the whole point of Killeen's 1995 paper: to "expand" a from
the fixed parameter it was represented to be in 1994 to a variable dependent
on the level of deprivation.

The "subtle contradiction" in Killeen's model (or at least in that part
presented in the 1995 paper) is that the amount of responding generated by
each incentive is assumed to be just that amount required to keep responding
going, which on ratio schedules is n. But in the 1995 paper, Killeen
alludes to the notion (apparently given mathematical treatment in the 1994
paper) that not all activity generated by an incentive is "expended" (my
word) on the instrumental behavior; we're not looking at the full model, but
only a simplified treatment of it. "Excess" resonding could be "vented" in
other activity, for all I know, in Killeen's model. We'll have to wait and see.

Bruce, this is just too much of a mess. It's exactly the sort of tangle
one gets into when trying to make sense of a mathematical development
that is flawed from the start. Juggling quantities around and giving
different combinations of them different names is just wishful thinking;
nothing new is being introduced.

Messy or not, the relevant question is whether the model Killeen produced
actually "works" in simulation. If it does, the simulation results can be
examined relative to extant data (do they fit?) and to the results from a
PCT alternative.

Regards,

Bruce

5

[From Bill Powers (960110.1200 MST)]

Bruce Abbott (950110.0850 EST) --

     I understand your point, but to be fair, we'd have to say that
     Killeen believes it is the ignition of fuel in the cylinders that
     is causing the engine to run, not the turning of the crankshaft, if
     by "cause" one means the source of the power from which all the
     action derives. It seems to me that if you have to identify one
     event in the cycle as the cause, it is the burning of the fuel,
     which produces expansion of the gases in the cylinder, forcing the
     piston down, and thus turning the crankshaft.

But what is it that causes the fuel to enter the cylinder and the spark
to occur that ignites it? The rotation of the crankshaft! My point was
that identifying _any_ variable in a closed loop as "the cause" is
arbitrary, and wrong. We can trace small causal steps from one component
of the loop to the next, but we end up where we started. Once the engine
is started, it makes ITSELF run: the explosion of the fuel drives the
crankshaft which cause the fuel to be drawn in and explode. The
operation of the engine, once started, is _spontaneous_; it is the whole
loop that is "behaving." There is no justification for singling out any
one process in the loop and assigning it the role of cause.

The reason the engine has to be started is that it is basically a
positive feedback system with one of its stable states at zero behavior.
In a negative feedback system with a nonzero reference signal, there is
only one stable state and the system does not have to be started
externally to reach it. The system will _spontaneously_ come to that
state as soon as it is turned on, with no external impetus.

     The Killeen theory of engine behavior would hold that each m units
     of gasoline produces enough power to run the engine through b
     revolutions, that these revolutions pump more fuel into the
     carburator, and that the effectiveness of the gasoline in
     increasing the engine's rpm declines as a negative exponential with
     increasing rpm (owing to the "inhibitory" effects of friction and
     other rpm-dependent effects), so that a given rate of fuel delivery
     produces a given asymptotic rate of crankshaft turning.

Not "through b revolutions" but "for b seconds." His theory leaves the
RPM unspecified, which means that even if the number of seconds is
determined, the number of revolutions is not. As to your last sentence,
it is still causally biased: the rate of fuel delivery is not
independent of the rate of crankshaft turning, but depends on it. So you
could just as well say that the rate of crankshaft turning increases on
a negatively-accelerating curve, producing an asymptotic rate of fuel
delivery. In fact, the rate of turning and the rate of fuel delivery
change _together_, each influencing the other all the way to asymptote.

     Now this is not too far-fetched. The PCT theory of engine behavior
     would say that the throttle setting provides a reference level for
     engine speed, that the difference between this reference level and
     the actual rpm determines how much gasoline is delivered to the
     cylinders, and that the rate of rpm increase will decline as a
     negative exponential to yield an asymptotic rate of crankshaft
     turning that is somewhat less than the throttle reference level
     (depending on the loop gain).

No, the throttle merely puts a constriction on the air flow, which
influences the asymptotic rate of fuel consumption _and_ RPM. These two
variables are also influenced by headwinds or tailwinds and by upgrades
of downgrades, as well as other factors such as air density, air
temperature, dragging brakes, and fuel quality. As RPM increases, the
vacuum pulling in the fuel decreases; friction rises as the square (or
higher powers) of linear speed: car speed, engine speed and of fuel flow
speed. The burning becomes less efficient as the power stroke occupies
less time, and on and on. The final equilibrium represents a balance
among all these changing factors.

     Both theories treat the engine as a closed loop system. Killeen's
     system is an equilibrium system; the PCT system is a control
     system.

The engine is not a control system, but a positive feedback system. It
would run away if there were no counteracting effects that increased
faster than linearly with engine RPM and fuel flow rate. It is an
equilibrium system, as you say, however you analyze it. The difference
between PCT and reinforcement theory is not the issue here; the issue is
that Killeen adopts an unjustifiable causal bias throughout his
arguments, when in fact he is dealing with a closed loop. He thinks of
variables as if they were independent variables when in fact they are
dependent variables like ALL the variables in the closed loop. It is the
loop as a whole that has to be considered the behaving unit, whether the
feedback is positive or negative.

     There is a simple test that will distinguish which is correct:
     apply a load to the crankshaft. If Killeen is right, the engine's
     rpm will decline. If PCT is right, the fuel system will react to
     the load by putting more fuel into the cylinders per stroke, and
     the rpm will hardly change (so long as the load is not too great).

The PCT model would fail in this case. Increasing the load (with the
throttle at a constant setting) will just slow the engine and reduce the
fuel flow. The nonlinear frictional terms will decline faster than the
net torque decreases, so a new equilibrium at a lower speed will result.
This is not a control system.

     The difference between these views has little to do with what is
     actually producing the power to turn the crankshaft: for both the
     power gets delivered only when the compressed air-fuel mixture is
     ignited in the cylinders. The difference is in the nature of the
     systems, whether equilibrium or control.

It is not a difference in views that makes the difference: it is the
organization of the system. A control-system model would be rejected by
the Test, because the RPM and fuel flow can be accounted for completely
on the basis of forces coming into equilibrium for normal physical
reasons. Nothing is sensing the RPM, comparing it with a reference
level, and adjusting the throttle to keep the RPM constant despite
disturbances. That would happen only if there were an RPM sensor and so
forth, operating the throttle. Every disturbance has exactly the effect
you would calculate on physical principles alone. Thus, no control
system. You could just as well say that the power gets delivered only
when the crankshaft triggers the spark; the mere presence of fuel in the
cylinder is not enough.

The argument between reinforcement theory and PCT is not a dispute over
which part of the loop is really the cause of behavior. It is an
argument about whether ANY variable in the loop should be considered as
a cause of the loop's behavior. In the PCT model, there are only two
causal -- i.e., independent -- variables: the reference signal and the
disturbance. All variables in the loop are functions of these two
independent variables. Even saying "behavior controls perception" is a
mistake if you interpret this to mean that behavior is an independent
variable. To understand how behavior controls perception you have to
understand the whole loop and the roles of the reference signal and
disturbance.

     Killeen's BBS article finally arrived yesterday afternoon; I'll
     have some comments on it after I've had a chance to give it a good
     look. As I suspected, the model Killeen presents there has arousal
     being incremented with each successive incentive delivery and
     declining as a negative exponential function of time between
     deliveries. For incentives delivered at regular intervals, arousal
     cumulates to an asymptotic value that depends on the rate of
     incentive delivery.

And on what does the rate of incentive delivery depend? The incentives
do not cause the behavior; they are as much effects as causes. All of
these causal terms should be eliminated. You can't understand the system
if you insist on identifying causes of the system's behavior. Closed-
loop systems behave all by themselves. The language Killeen keeps using
seems designed to deny this basic fact.

···

-----------------------------------------------------------------------
Best,

Bill P.

6

[From Bruce Abbott (950111.1100 EST)]

Bill Powers (960110.1200 MST) --

   Bruce Abbott (950110.0850 EST)

    I understand your point, but to be fair, we'd have to say that
    Killeen believes it is the ignition of fuel in the cylinders that
    is causing the engine to run, not the turning of the crankshaft, if
    by "cause" one means the source of the power from which all the
    action derives. It seems to me that if you have to identify one
    event in the cycle as the cause, it is the burning of the fuel,
    which produces expansion of the gases in the cylinder, forcing the
    piston down, and thus turning the crankshaft.

But what is it that causes the fuel to enter the cylinder and the spark
to occur that ignites it? The rotation of the crankshaft! My point was
that identifying _any_ variable in a closed loop as "the cause" is
arbitrary, and wrong.

When I said I understood your point, I meant that I understood your point:
no need to repeat it. But I think you missed mine. Remove any part of the
loop: the delivery of the fuel, the spark, the rotation of the camshaft --
any part -- and the machine quits. But what is the source of the power? It
is stored in the gasoline and is released when the gasoline is mixed with
air in the right ratio, compressed, and ignited. In Killeen's model, the
energy for the incited activity does not, of course, derive directly from
the food pellet that supposedly induces the activity, but in S-R fashion
simply triggers the neural activity that somehow gets translated into
physical activity, some of which produces the instrumental responses. But
the effect is the same. "Energy" is added to the system only at that point
in the cycle when the incentive is delivered, and this is what drives the
cycle, nevermind the fact that it is a closed loop.

In the PCT model the "energy" is again stored in the body and only released
(in the form of behavior) as a function of a neural signal. However, in
this case the signal represents the error between some quantity and its
reference level; the "power" is delivered not via

    Sr ------>[v*h]------> B,

where Sr is the incentive (reinforcing stimulus), v is the amount of
activity induced by the appearance of Sr, h is the hunger, or level of
deprivation, and B is the behavior, but through

    [r - p] ------>[g*e]------> B,

where r is the reference, p is the current perception, r - p = e, and g*e
translates error into behavior. Both systems include a negative feedback
loop (not shown above), but Killeen's system is not a control system because
its output is not a function of error.

The reason the engine has to be started is that it is basically a
positive feedback system with one of its stable states at zero behavior.
In a negative feedback system with a nonzero reference signal, there is
only one stable state and the system does not have to be started
externally to reach it. The system will _spontaneously_ come to that
state as soon as it is turned on, with no external impetus.

Yes. I've described the Killeen system/engine analogy just as you did, so
we agree there. And what you say about the control system is true, too, of
course.

    The Killeen theory of engine behavior would hold that each m units
    of gasoline produces enough power to run the engine through b
    revolutions, that these revolutions pump more fuel into the
    carburator, and that the effectiveness of the gasoline in
    increasing the engine's rpm declines as a negative exponential with
    increasing rpm (owing to the "inhibitory" effects of friction and
    other rpm-dependent effects), so that a given rate of fuel delivery
    produces a given asymptotic rate of crankshaft turning.

Not "through b revolutions" but "for b seconds." His theory leaves the
RPM unspecified, which means that even if the number of seconds is
determined, the number of revolutions is not. As to your last sentence,
it is still causally biased: the rate of fuel delivery is not
independent of the rate of crankshaft turning, but depends on it. So you
could just as well say that the rate of crankshaft turning increases on
a negatively-accelerating curve, producing an asymptotic rate of fuel
delivery. In fact, the rate of turning and the rate of fuel delivery
change _together_, each influencing the other all the way to asymptote.

I think the problem here is that I am talking about a single event in the
loop that provides the "power" for the system, whereas you are talking about
rates.

    Now this is not too far-fetched. The PCT theory of engine behavior
    would say that the throttle setting provides a reference level for
    engine speed, that the difference between this reference level and
    the actual rpm determines how much gasoline is delivered to the
    cylinders, and that the rate of rpm increase will decline as a
    negative exponential to yield an asymptotic rate of crankshaft
    turning that is somewhat less than the throttle reference level
    (depending on the loop gain).

No, the throttle merely puts a constriction on the air flow, which
influences the asymptotic rate of fuel consumption _and_ RPM. These two
variables are also influenced by headwinds or tailwinds and by upgrades
of downgrades, as well as other factors such as air density, air
temperature, dragging brakes, and fuel quality. As RPM increases, the
vacuum pulling in the fuel decreases; friction rises as the square (or
higher powers) of linear speed: car speed, engine speed and of fuel flow
speed. The burning becomes less efficient as the power stroke occupies
less time, and on and on. The final equilibrium represents a balance
among all these changing factors.

This is a hypothetical: we're supposed to be pretending we don't really know
how an engine is organized and are developing different theories to account
for the engine's behavior. What you've described is the actual
(equilibrium) system.

    Both theories treat the engine as a closed loop system. Killeen's
    system is an equilibrium system; the PCT system is a control
    system.

The engine is not a control system, but a positive feedback system. It
would run away if there were no counteracting effects that increased
faster than linearly with engine RPM and fuel flow rate. It is an
equilibrium system, as you say, however you analyze it. The difference
between PCT and reinforcement theory is not the issue here; the issue is
that Killeen adopts an unjustifiable causal bias throughout his
arguments, when in fact he is dealing with a closed loop. He thinks of
variables as if they were independent variables when in fact they are
dependent variables like ALL the variables in the closed loop. It is the
loop as a whole that has to be considered the behaving unit, whether the
feedback is positive or negative.

But the difference between PCT and reinforcement theory is _precisely_ the
issue I am trying to address, and it has to do with that causal bias. One
can certainly talk about g*e as causing B while recognizing that B in turn
has a feedback effect on e. If I open the loop just after B, g*e will still
affect the value of B as it usually does. If I talk about the effect of e
on B, does this mean that I am necessarily thinking of "variables as if they
were independent variables when in fact they are dependent variables"? To
answer that question, you must examine my analysis. Killeen's analysis
certainly takes feedback into account and thus does not treat the variables
in the loop as independent variables. This is clearly demonstrated in his
1994 BBS paper.

    There is a simple test that will distinguish which is correct:
    apply a load to the crankshaft. If Killeen is right, the engine's
    rpm will decline. If PCT is right, the fuel system will react to
    the load by putting more fuel into the cylinders per stroke, and
    the rpm will hardly change (so long as the load is not too great).

The PCT model would fail in this case. Increasing the load (with the
throttle at a constant setting) will just slow the engine and reduce the
fuel flow. The nonlinear frictional terms will decline faster than the
net torque decreases, so a new equilibrium at a lower speed will result.
This is not a control system.

No, this is not a control system. And neither is Killeen's 1994 model
(which does not include the food-level control system found in the 1995
deprivation model). But it is a closed-loop, equilibrium model. The
theoretical issue to be addressed is whether the system whose behavior is
observed in these operant studies is an equilibrium system, as Killeen
proposes, or a control system, as Powers proposes.

    The difference between these views has little to do with what is
    actually producing the power to turn the crankshaft. . . . The
    difference is in the nature of the systems, whether equilibrium or
    control.

It is not a difference in views that makes the difference: it is the
organization of the system.

Excuse me, but aren't you merely repeating what I had said?

    Killeen's BBS article finally arrived yesterday afternoon; I'll
    have some comments on it after I've had a chance to give it a good
    look. As I suspected, the model Killeen presents there has arousal
    being incremented with each successive incentive delivery and
    declining as a negative exponential function of time between
    deliveries. For incentives delivered at regular intervals, arousal
    cumulates to an asymptotic value that depends on the rate of
    incentive delivery.

And on what does the rate of incentive delivery depend? The incentives
do not cause the behavior; they are as much effects as causes. All of
these causal terms should be eliminated. You can't understand the system
if you insist on identifying causes of the system's behavior. Closed-
loop systems behave all by themselves. The language Killeen keeps using
seems designed to deny this basic fact.

Incentives "cause" behavior if, when you open the loop, delivering an
incentive is still followed by a regular pattern of activity. Error in a
control system "causes" behavior in this sense. If Killeen wants to
theorize that each incentive delivery provides the impetus for the
subsequent output, as the burning of compressed air-fuel mixture does in the
engine, this does not automatically render his account incorrect. The
important thing is whether the model as a whole treats the closed loop
properly as an equilibrium system. It does.

Regards,

Bruce

7

[From Bill Powers (960113.1600 MST)]

I'm not really keeping up with all the discussions on the net, because
I've found something more interesting than PCT. Day before yesterday I
finished laying out the posts and pier for a little backyard observatory
building and helped (watched) a guy dig the holes and pour the concrete.
My garage is full of lumber and I just cleaned up the steel-pipe mount I
had welded together a couple of weeks ago, getting ready to prime and
paint it. The weather has been disastrous (for farmers) -- warm and
clear every day. Over the next month I hope to get the little house
built and roofed (removable) and the 10-inch Schmidt-Cassegrain mounted
permanently, so I can go out whenever my rage at human obstinacy becomes
too large and look at some more permanent aspects of the Universe than
theoretical fantasies. It's more tranquilizing to watch the heavens go
around and around in big circles than to watch verbal arguments go
around and around in little ones.

···

-----------------------------------------------------------------------

From Rick Marken (960112.1400) --

Bruce Abbott (960112.1040 EST) --

From Bruce's summary of Killeen's article, I deduce that the basic model

makes output behaviour B a function of F0 - F (where F is food in the
stomach and F0 is the reference amount in the stomach) AND ALSO makes
the output gain proportional to reinforcement rate (so this is a
nonlinear system). Rick Please Note.

The integration that stabilizes the loop occurs in the stomach, because
F is the integral of food intake rate R. This is a nonlinear negative
feedback system; I haven't yet examined Bruce's Pascal code (my eyes are
on the stars) to see what else is involved, but at the moment I don't
think it's a positive feedback system.

Of course this may change, but right now it looks as if we have a
negative feedback system in which behavior produces food and keeps the
stomach full at the level F0, given enough reinforcement. At the same
time, the gain in the output function can be varied. We know that
raising the gain will result in quicker and more accurate control of the
controlled variable, so the effect of making gain depend on
reinforcement rate should be that control is better at higher
reinforcement rates. If the relationship between reinforcement rate and
output gain is linear and zero-based (zero gain at zero reinforcement
rate), I believe that the result will be like the right side of the
Motherall curves, but on the left side there will be a leveling off
instead of a downturn. To get the downturn, it's necessary to use a
nonlinear relation between reinforcement rate and gain, or make it non-
zero-based (i.e., the effect on lowering gain doesn't come until
reinforcement rate has falen to a critical level). The nonlinear "zeta"
effect in Killeen's elaborations of his basic model may take care of
this; there are certainly enough parameters to do it. I haven't tried to
understand them yet.

I will sooner or later get to the Pascal code and check all this out. My
enthusiasm for doing this is limited, because unlike Bruce I really
don't care how Killeen's model works, having seen enough of it. I
suspect that the real explanation of the downturn on the left side of
the Motheral curves is that the animal simply goes away from the key or
lever and is doing something else most of the time, when pressing the
key hardly ever produces any reinforcers or incentives or behavior
controllers or whatever imaginative name you want to give to the things
the animal eats.

We can mess around with a control model, adding features to it until it
matches the curves, under the assumption that the apparent behavior
rates are really the rates at which the animal pecks or presses. We can
ignore the high probability that the apparent rates have little to do
with the actual rates. But what will we get out of that? Some curves
that pass near the data points. So what? We will still be ignoring the
fact that our data may be illusory. I really have no interest in
carrying this on until we can see some real data and find out if our
suspicions are well-founded.

As to trying to understand Killeen's model, Bruce, I applaud your
scientific attitude and your desire to know what you're talking about
before you criticize. It must be very hard not to get sucked into
Killeen's facile reasoning. What you're doing is probably a necessary
preliminary to writing the paper you have in mind. I am glad, however,
to leave this process mostly in your hands. I have difficulty telling
when you are expressing your own interpretation and when you are
describing Killeen's, so I had better stop arguing against whomever I am
talking to at the moment. It's too confusing.
--------------------------

One point, Bruce:

     I would say that it is the error signal that is the prime mover in
     this loop, not the reference, but perhaps we have different
     definitions of "prime mover."

The error signal is determined by the current value of the perceptual
signal and by the reference signal. The reference signal is the only
independent variable in this loop (since no disturbances are
considered). A "prime mover" in a loop can only be an independent
variable, which is not what the error signal is. In terms of Killeen's
equations, the prime mover is F0.
-----------------------------------------------------------------------
Best,

Bill P.