[From Bill Powers (960111.0945 MST)]
Bruce Abbott (950111.1100 EST) --
But what is the source of the power? It is stored in the gasoline
and is released when the gasoline is mixed with air in the right
ratio, compressed, and ignited. In Killeen's model, the energy for
the incited activity does not, of course, derive directly from the
food pellet that supposedly induces the activity, but in S-R
fashion simply triggers the neural activity that somehow gets
translated into physical activity, some of which produces the
instrumental responses. But the effect is the same. "Energy" is
added to the system only at that point in the cycle when the
incentive is delivered, and this is what drives the cycle,
nevermind the fact that it is a closed loop.
I set up my engine analysis deliberately so that the event corresponding
to organismic motor behavior was the burning of the gasoline in the
cylinder, corresponding to the expenditure of energy by muscles drawing
from the body's "fuel tank." The _consequence_ of this burning was
rotation of the crankshaft. I then presented an elaborate argument that
the cause of the motor behavior was its consequence, the turning of the
crankshaft. I was intending to show the parallel between saying that
behavior is controlled by its consequences and saying that the engine is
controlled by the rotation of the crankshaft.
Your claim that energy is added at the point where the incentive is
added would correspond to a claim that the fuel pump is the cause of the
behavior. But note that this energy is simply stored; it is not used
until the spark plug fires, which is caused by the turning of the
crankshaft to the correct angle to trigger it. Similarly, in the body
the discriminative stimulus triggers the expenditure of energy in the
muscles.
The energy input in the form of food pellets makes behavior _possible_,
but in itself it produces no behavior. Gasoline in the car's tank or
existing as vapor in a cylinder does not cause any behavior. The energy
taken into an organism is not specific to any particular behavior; all
behavior uses the same stores of energy. It is the actual use of energy,
triggered by neural signals, that produces motor outputs. All the high-
energy processes take place in the output function.
The overall point I was trying to make was that attributing causality to
reinforcers is simply a mistake, because delivery of reinforcements
depends just as much on behavior as behavior depends on delivery of
reinforcements. No matter what the contingency, the rate of
reinforcement depends on the rate of behavior. This is a closed loop.
My point was not to compare a PCT analysis with a reinforcement-theory
analysis; the system I picked as an example was not a control system, so
I thought that issue could be put aside. It was to show that the
behaviorist bias toward attributing causality to one element of a closed
loop is a mistake -- regardless of the kind of closed loop involved.
···
-----------------------------------------------------------------------
So you
could just as well say that the rate of crankshaft turning increases
on a negatively-accelerating curve, producing an asymptotic rate of
fuel delivery. In fact, the rate of turning and the rate of fuel
delivery change _together_, each influencing the other all the way to
asymptote.
I think the problem here is that I am talking about a single event
in the loop that provides the "power" for the system, whereas you
are talking about rates.
The single event of one incentive does not provide the power for the
response that follows. It is more like the triggering of a sparkplug.
The power is drawn from an internal store and enters the loop at its
output function, not its input function. That internal store is not tied
to any one input event, or events occurring only in the present. It's
just a tank of gas. Food incentives have two effects: one is to fill the
tank, the other is to produce sensory experiences.
What am I disturbing here? I'm sure you're aware that Killeen and other
behaviorists choose their language so it never appears that any behavior
is initiated by the organism. I'm sure you can see that reinforcements
are often spoken of as if they were independent causes of behavior.
Behavior is never spoken of as having a similar controlling role with
respect to reinforcements. I was merely trying to show how arbitrary it
is to pick one element of a closed loop and speak of it as a cause. This
has nothing to do with control theory, which I tried to show by picking
a non-control system for my example. You still seem to be looking for a
causal variable in the loop. Why?
But the difference between PCT and reinforcement theory is
_precisely_ the issue I am trying to address, and it has to do with
that causal bias. One can certainly talk about g*e as causing B
while recognizing that B in turn has a feedback effect on e. If I
open the loop just after B, g*e will still affect the value of B as
it usually does. If I talk about the effect of e on B, does this
mean that I am necessarily thinking of "variables as if they were
independent variables when in fact they are dependent variables"?
To answer that question, you must examine my analysis. Killeen's
analysis certainly takes feedback into account and thus does not
treat the variables in the loop as independent variables. This is
clearly demonstrated in his 1994 BBS paper.
Here is how Killeen talks: (from "Mathematical principles"):
Reinforcement controls aspects of the response along multiple
dimensions. Experimental contingencies ... may shape these aspects
either in concert with or in opposition to the fundamental
excitatory property of reinforcement.
A reinforcer's effects are not limited just to the response that
immediately preceded it. ... It follows that the reinforcement must
reach back toward the penultimate response.
Consider an alternative approach: assume that reinforcement
increases the probability of those events that are in a subject's
recent memory.
... behavior's time is ... driven by those external pacemakers and
external stimuli that capture attention.
Response rates changed in the direction which the percentile
schedule forced them, and the force was more effective at some
values than at others.
The experiment showed that we have much better control of
organisms' behavior insofar as we ...
... two counterpoised forces affect the control of behavior under
schedules of reinforcement: as we increase the reinforcement, we
activate more behavior, but at the same time we decrease the number
of responses that each reinforcer can influence.
To see how biased this is, all you have to do is to remember that the
occurance of a reinforcer is entirely a function of behavior. Every
reference to a reinforcement could be a reference to the behavior that
produced it. Every attribution of a causal role to a reinforcement could
equally well be carried back one step in the loop to produce an
attribution of a causal role to the behavior without which the
reinforcement could not have occurred. To choose the reinforcer as the
cause is evidently a deliberate policy, in support of an ideological
position.
I am not saying that Killeen never breaks free of this framework. When
he sets up a system of equations with a loop in it, he treats the
relationships mathematically, which involves no causal statements. But
whenever he lapses back into English, he resolutely ignores the equal
claims to causality by behavior. He always treats reinforcement as the
driving force behind behavior, never asking what drives the driving
force.
Incentives "cause" behavior if, when you open the loop, delivering
an incentive is still followed by a regular pattern of activity.
When you open the loop by cutting the link from behavior through the
environment to the incentives, the incentives cease to appear. So
clearly, incentives do not "cause" behavior. And you know what happens
if, with the loop broken, you start admininstering incentives
independently of behavior. "Non-contingent reinforcement reduces
behavior." Eventually, it reduces any specific behavior to the
background level.
If Killeen wants to theorize that each incentive delivery provides
the impetus for the subsequent output, as the burning of compressed
air-fuel mixture does in the engine, this does not automatically
render his account incorrect.
It does not render his mathematical account incorrect, but it does
render his theory incorrect. He could equally well say that each
delivery of N responses provides the impetus for the next reinforcement.
The biggest problem of Killeen's theoretical story is not in the
mathematical script, but in the background music. Although the script
seems to have certain artistic influences in it.
-----------------------------------------------------------------------
Bruce Abbott (960110.2330 EST) --
And even though responding is rate-limited by this constraint,
Killeen's model would cumulate seconds of responding so long as
v*h/s is greater than the ratio requirement of the schedule, n,
where v = the value of the incentive and h is proportional to the
level of deprivation, as I noted in my post of 960108.0900 EST.
...
When incentives occurred at a periodic rate, arousal accumulated
with each incentive delivery and drained off between them according
to the negative exponential. Because of this relationship, a
stable asymptotic level of arousal (and thus of activity) would
develop which would be proportional to the rate of incentive
delivery: A = k*R, where k is the constant of proportionality. I
suggested then that this is probably the mechanism Killeen adopts
to limit the cumulation of arousal over time.
At least Killeen recognized the problem. Of course this still leaves the
problem that theoretically, behavior increases with arousal and arousal
increases with incentive rate, which is observed only at the largest
ratios. This means that some assumption has to be made to cause the
curve to turn over and start down again as the ratios become small
enough, because that is what the real data do.
In other words, modeling response rates as a simple proportion of
incentive rates, and taking into account the constraint imposed by
response time, one gets a result for ratio schedules that simply
does not work. However, there are two additional aspects of
Killeen's model that have not yet been brought into play here.
As is to be expected: we need more assumptions to fix the basically
faulty model.
These considerations involve the organism's memory of the responses
it has recently made and are fairly complicated, so I'll skip the
details here. What it ultimately comes down to is a "coupling
coefficient" (zeta) which basically determines the effectiveness of
the incentive in inciting the "target" (i.e., instrumental)
response (as opposed to, say, general activity). In ratio
schedules zeta turns out to be proportional to the number of
responses in the ratio, meaning that coupling gets better as the
ratio increases. Rewriting equation 3 and using c = 1/a, Killeen
comes up with
zeta N
(4) B = ---- - ---, N =< a/s;
s a
B = 0, N > a/s,
where zeta = p[1 - exp(-lamda*N)].
Ignoring the English explanation of this set of modifications, the point
is clearly that we now have enough parameters and mathematical forms so
that the resulting curve can be passed through practically any set of
reasonably consistent data points.
This is where I cease to take the model seriously. The original model
hardly shows through the patches. It has become so complicated that it
can explain just about any observations, even erroneous ones or
observations that leave out critical factors, like collection time. Note
that the purpose of the modification is to explain why, at low ratios,
an increase in reinforcment rate appears to go with a decrease in
behavior rate (which is a contradiction of the initial assumption that
response rates increase with arousal and arousal increases with
reinforcement). But you have shown that at least a good part of this
decrease, if not all of it, is due to averaging the response rate over
times that include collection time, and that it is quite possible that
behavior rate does not change at all. So what happens to Killeen's
inspired explanation for the downturn in behavior rate if behavior rate
does not actually change, or if it changes in a way other than the
apparent way? Obviously, another patch will be required, and will be
found along with a suitable verbal explanation.
Killeen introduces this explanation for the downturn without apparently
realizing that he has another, completely different, explanation for it.
In the equation for the effect of deprivation that you posted a while
back, you showed that as deprivation decreases, behavior decreases. But
deprivation is F0 - F, where F is the delivery rate of incentives.
Deprivation decreases when incentive rate increases, so as the incentive
rate increases, behavior decreases. Killeen's analysis accounts for the
decline in behavior rate with increases in incentive rate without ever
uttering the word "zeta." So why does he have to explain it again?
Because the assumption about deprivation doesn't provide for any region
in which behavior rate _increases_ with incentive rate?
If you plug the new relationship that includes zeta into the equations
relating to deprivation, what happens?
I admire your persistence and patience in going through Killeen's
reasoning. If you really think it's worthwhile go ahead, but I really
can't see much return for the effort. Killeen will never be at a loss
for rebuttals of criticisms; see his replies to the commentaries in the
BBS article. I didn't find any remarks like "Gosh, I never thought of
that, you're right!" I think that if you come up with a rigorous model
that improves on Killeen's model, you will simply acquire a verbally
facile and mathematically adept enemy.
----------------------------------------------------------------------
Here a final thought to chew on.
If you pick any variable in a closed loop, you can write the loop
equations, by successive substitutions, in such a way as to eliminate
all other variables that are in the loop. So if incentives are a
function of behavior, behavior is a function of deprivation or arousal
or whatever, and deprivation or arousal is a function of incentives, you
can end up with an equation in which behavior is the only loop variable
left, other than time, as a function of system parameters, constants,
and independent variables outside the loop.
This is how closed-loop systems behave: any variable around the loop can
represent the behavior of the entire loop. This is because there are no
independent variables in the loop. You could express the behavior of the
loop in terms of reinforcement rate, or behavior rate, or arousal level,
or any other _single_ variable that you like. The independent variables
would be any external influences (like disturbances, or in a PCT model,
a reference signal, or in general any arbitrary additive constants).
-----------------------------------------------------------------------
Best,
Bill P.