[From Bruce Abbott (951216.1220 EST)]
Rick Marken (951215.1900) --
Bruce Abbott (951215.1420 EST)
The first step in model-development is to construct a model that
includes reasonable assumptions (given what is known about the
situation) and fits the data.Done. Here they are:
Reinforcement: dB/dt = k1*C - k2*B
PCT: dB/dt = k1*(Co - C)
Done? These make a minimum number of not unreasonable assumptions. But do
they fit the data? The first model runs away. The second goes to max rate
and stays there if extinction is imposed.
Let's see whether we can develop some reasonable-sounding models
that actually _work_ under a given set of conditions,
Done. We have used a simple control model to account for the
phenomenon of "reinforcement" (behavior and contingent reinforcement
rate rise to an asymptotic value over time);
I wouldn't say we have "accounted for" the phenomenon of reinforcement in
this model. Imposing the "reinforcement" contingency is modeled as a sudden
change in the reference level for this system from zero reinforcement rate
to some positive value, and if imposing the "extinction" contingency is
modeled as a sudden loss of feedback, followed rather quickly by a resetting
of the reference level to zero reinforcement rate. Why should the reference
for reinforcement rate change in this way? The model doesn't say. What it
does say is that, if the reference _were_ to change in this way, the model's
rate of behavior would change in a way that resembles the observations.
As presently constructed, both the "reinforcement" model you developed
and the PCT model are unsatisfactory, although for different reasons.
Well, I can see that the reinforcement model is unsatisfactory; it doesn't
predict what we actually see in experiments. What is unsatisfactory about
the control model? The fact that it works?
No. The model has some good things going for it. First, the behavior of
the system following imposition of the reinforcement contingency does follow
a negatively accelerated, increasing curve that levels off at some
steady-state rate, as is often observed in the data. Second, as Bill
pointed out, a transient increase in response rate following transition to
extinction is commonly observed in the data, and this model accounts for it.
Third, in extinction the model gives a negatively-accelerated, decreasing
curve that again resembles the data. So, the increase in responding
following imposition of the contingency (reinforcement phenomenon), decrease
following removal of the contingency (extinction phenomenon) and the
extinction transient can be understood as the natural consequences of
certain changes in control-system parameters.
So what is wrong with the model? Several things. First, it does not
explain why behavior should stay at zero if the initial baseline is zero.
The reinforcement model does: if the behavior rate is zero, reinforcement
never occurs, response strength remains at zero, and behavior rate remains
at zero. Second, it does not explain the initial slow acceleration of the
system during acquisition (tending to produce an S-shaped, rather than
negative exponential curve). Third, it leaves unanswered the question of
why simply imposing the reinforcement contingency should have the effect (as
it is assumed to do) of raising the reference level for reinforcement rate,
and why simply breaking that contingency should reset the reference level to
zero. Isn't the animal still hungry? Why should it's reference for food go
to zero? Bill's very simple reinforcement model basically says that the
behavior will be zero unless maintained at higher rates by a reinforcement
contingency, so withdrawal of the contingency leads naturally to extinction.
Withdrawal of the contingency in the control model leads naturally to an
infinite behavior rate.
I do not believe that these are insurmountable problems for control theory,
but it is clear that the simple control model presented here is not adequate
to the task set for it. That is why I want to develop it further.
Bruce Abbott (951215.1820 EST) to Joel Judd --
These simpleminded programs don't _employ_ proper operant
technology, they ignore it.Now I see the problem, Bruce. You actually believe in behavior
modification.
I believe that if we are going to criticize someone else's methods, we ought
at least to give them a fair presentation. When we construct caricatures to
attack, then we are indeed doing battle with straw men and ought not to be
too surprised -- and offended -- when the accusation is leveled against us.
If you were attacking the Ptolemaic system, it wouldn't matter a whit
whether I believed in it or not; I would still insist that you at least
describe it correctly before launching your attack.
Regards,
Bruce