# Bill's model of operant behavior in EAB terms

**URL:** <http://discourse.iapct.org/t/bills-model-of-operant-behavior-in-eab-terms/1802>\
**Category:** CSG1995\
**Created:** [July 4, 1995, 10:46am UTC](http://discourse.iapct.org/t/bills-model-of-operant-behavior-in-eab-terms/1802 "1995-07-04T10:46:11Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![William\_T\_Powers4](http://discourse.iapct.org/letter_avatar_proxy/v4/letter/w/977dab/32.png) [@William\_T\_Powers4](http://discourse.iapct.org/u/William_T_Powers4)\
**Post date:** [July 4, 1995, 10:46am UTC](http://discourse.iapct.org/t/bills-model-of-operant-behavior-in-eab-terms/1802/1 "1995-07-04T10:46:11Z")

</div>

[From Bill Powers (950704.0100 MDT)]

Bruce Abbott --

Here is the operant conditioning model I developed and posted last year,  
or perhaps it was two years ago, to fit the Motherall data (and, I hope,  
other data as well). The following development expresses the  
relationships entirely in terms of observable variables in EAB terms,  
without any model of the internal organization of the animal. I have  
added a "value" parameter and a "relative time" parameter; in the model  
I posted last fall, I lumped all the gain parameters into one, but the  
effect is the same.

> **···**
>
> ----------------------------------------  
> The satiation level of reinforcement rs can be defined as that level of  
> average obtained reinforcement ra at which the average behavior ba will  
> just go to zero. When ra is less than rs, average behavior ba will occur  
> at a rate proportional by km to to the difference (this is the  
> "motivation due to deprivation"):
> 
> ba = km\*(rs - ra)
> 
> The average obtained reinforcement is the average of the feedback  
> function ff of the behavior measure plus the noncontingent reinforcement  
> rn (in my model, rn was zero):
> 
> ra = average(ff(ba) + rn)
> 
> The "value" kv of the reinforcer to the organism is measured by an  
> increase in behavior due to increasing some aspect of the reinforcer  
> such as its size. This requires kv to be used in a position where  
> increasing kv will increase average behavior:
> 
> ba = kv\*km\*(rs - ra)
> 
> The measured average amount of behavior depends on the fraction of the  
> time kt that the animal actually spends performing the behavior b, as  
> opposed to some other behavior, where kt varies from 1 (full-time  
> behavior) to 0 (none of the time spend on the behavior). As kt decreases  
> from 1, the apparent average behavior rate also decreases. Thus kt must  
> also appear in the expression for b:
> 
> ba = kt\*kv\*km\*(rs - ra)
> 
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If the average behavior ba consists of continuous behavior bc for a  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;time tc alternating with zero behavior for a time tz, the  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;continuous behavior bc can be obtained from ba by
> 
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;bc = [(tc + tz)/tc]\*ba, tc \> 0.
> 
> This gives us the main two system equations:
> 
> (1) ba = kt\*kv\*km\*(rs - ra)
> 
> (2) ra = average(ff(ba) + rn)
> 
> Note that rn will tend to reduce behavior, by increasing ra and thus  
> reducing the motivation.  
> ---------------------------------------------  
> These equations apply to the right-hand side of the Motherall curve. As  
> we move left on that curve, the observed behavior rate begins to fall  
> below the straight line predicted by the above equations, with the curve  
> turning downward. One possible explanation is that as the average  
> motivation ma increases beyond some critical value, the animal begins to  
> spend more time on other behaviors, thus reducing the value of kt.
> 
> Average motivation ma is defined as
> 
> ma = (rs - ra)
> 
> A linear point-slope model for the effect of ma on kt can be defined as
> 
> (3) kt = kt0 - k1\*(ma - mc), ma \>= mc
> 
> where mc is a critical amount of motivation.
> 
> When this expression is used for kt in equation (1) and the constants  
> are properly adjusted, the model fits the Motherall data (for body  
> weight equal to 80% of normal) over the entire range of ratios. Other  
> mathematical forms may give nearly the same results.  
> --------------------------------------  
> The loop gain of this control system is kt\*kv\*km\*(partial of ra with  
> respect to ba). The output sensitivity is kt\*kv\*km, which I treated as a  
> single gain constant in the posted model.
> 
> This model makes average behavior a two-valued function of average  
> reinforcement, with a maximum point. Left of the maximum point, average  
> behavior increases as average reinforcement increases. Right of the  
> maximum point, average behavior decreases as average reinforcement  
> increases, reaching zero when the average reinforcement reaches the  
> satiation level. The effects of costs of behavior can be absorbed into  
> km if they are assumed linear.
> 
> The position of the maximum of the curve depends on the output  
> sensitivity kt\*kv\*km, the critical motivation value mc, and the  
> parameter k1.
> 
> For any range of schedules of reinforcement and constant values of k1  
> and mc, the output sensitivity can be adjusted so that behavior rises  
> with reinforcement over the whole range, decreases with reinforcement  
> over the whole range, or first rises and then decreases with  
> reinforcement as in the Motherall data. In general, decreasing any  
> output sensitivity factor (kt, kv, or km) will move the operating region  
> toward the condition of increased reinforcement going with increased  
> behavior; the peak of the curve will move to the right.
> 
> So, for example, by decreasing the "value" of the reinforcer (as by  
> decreasing its size), the behavior can be made to rise monotonically  
> with increases in reinforcement over the whole range of schedules. This,  
> I propose, is how the general rule of "more reinforcement, more  
> behavior" was initially established, and how, in fact, the concept of  
> "reinforcement" gained credence. As long as the product kt\*kv\*km is kept  
> low enough, this rule will apply. It is always possible, therefore, to  
> set up an experiment to prove that an increment of reinforcement will  
> cause an increment of behavior. All that is required is to keep the  
> product kt\*kv\*km small enough, which can be done by manipulating kv as  
> by reducing the size of the reinforcer.
> 
> When this relationship is not the primary subject, a large reinforcer  
> can be used (large value of kv), moving the peak of the curve far to the  
> left. Then as the schedule of reinforcements, starting with an easy  
> schedule such as FR-1, is changed to produce less and less average  
> reinforcement per unit behavior, the behavior will rise as the  
> reinforcment decreases, leading to very large values of the behavior at  
> low rates of obtained reinforcement. This is how Skinner demonstrated  
> shaping the pecking behavior of pigeons to very high rates. He did not  
> seem to notice that the "normal" relationship of reinforcement rate to  
> behavior rate had reversed.  
> -----------------------------------------  
> In this way of developing the model, we accept certain relationships  
> without explaining why they hold -- for example, why excessive levels of  
> motivation lead to spending less of the total time on the behavior in  
> question. We accept the satiation level as given and fixed. In a more  
> complete model, we would try to relate a variable satiation level to,  
> for example, weight gain and loss, and to relate excessive motivation to  
> the commencement of a trial-and-error search for other sources of  
> reinforcements. The latter consideration, starting with the search in  
> progress, would come to explain the apparent "selective" effects of  
> reinforcement.  
> -----------------------------------------  
> All of this, as you know, is pure PCT. Several phenomena that are new  
> may be explained by this model, particularly the shift of the peak of  
> the Motherall curve with changes in certain parameters. The apparent  
> effect of reinforcement on behavior is explained in terms of the model,  
> with the "standard" effect appearing only over a certain range of  
> parameters. The reinforcement variable does not play any special role in  
> behavior other than the apparent one; it does no "maintaining" of  
> behavior, although the observed relationships can be interpreted in that  
> way. Causation is completely circular, with the only independent  
> variables being rs and rn: the satiation level of reinforcement and the  
> noncontingent reinforcers.
> 
> Does this bear any resemblance to the model you're working on?  
> -----------------------------------------------------------------------  
> Best,
> 
> Bill P.
