# Reorganization vs reinforcement

**URL:** <http://discourse.iapct.org/t/reorganization-vs-reinforcement/9786>\
**Category:** CSG1994\
**Created:** [December 6, 1994, 8:11pm UTC](http://discourse.iapct.org/t/reorganization-vs-reinforcement/9786 "1994-12-06T20:11:31Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![William\_T\_Powers2](http://discourse.iapct.org/letter_avatar_proxy/v4/letter/w/c67d28/32.png) [@William\_T\_Powers2](http://discourse.iapct.org/u/William_T_Powers2)\
**Post date:** [December 6, 1994, 8:11pm UTC](http://discourse.iapct.org/t/reorganization-vs-reinforcement/9786/1 "1994-12-06T20:11:31Z")

</div>

[From Bill Powers (941206.1205 MST)]

Rick Marken (941205.1230) --

Rick, this is an important point we've been leaving out; thanks for  
reminding us all of it:

> So a random structural change in the perceptual control hierarchy will  
> tend to be preserved (not reorganized away) if (as a side effect) the  
> change results in the reduction of intrinsic error.... Learning (from a  
> PCT perspective) is not a matter of "strenthening" responses that  
> "work" (produce a particular result).

Reinforcement theory and reorganization theory are not similar; they are  
essentially opposite explanations of the phenomenon of change of  
behavioral organization. Under reinforcement theory, it is the  
consequence of the behavior that brings the behavior into being and  
"strengthens" it. But this explanation contains some conceptual  
problems. Before the behavior that is to be strengthened occurs, some  
other behavior is occurring, with other consequences. Why do those  
consequences not strengthen the behavior that leads to them? How does a  
different consequence take over, and guide the behavior toward creating  
more of itself? And exactly what effect is the consequence supposed to  
have, and via what mechanism, in creating a change in the behavior?

Reorganization theory does not have these problems. Reorganization is  
basically a process of random changes in organization. It is LACK of  
certain beneficial effects (or presence of damaging effects) that drives  
reorganization. If the organism is sufficiently hungry, or thirsty, or  
hurting, or cold, it begins to alter the forms of its behavioral  
systems. This process of change continues as long as the underlying  
error condition persists. When a result of these changes is to  
ameliorate the error condition, the interval between reorganizations is  
increased; when the error condition is worsened, the interval decreases.  
Only when the error condition is brought below some minimum value does  
the process of change cease.

In reorganization theory there is no special effect of the consequences  
of behavior on the behavior that produces them. The behavior is not  
"strengthened" or made "more probable" by reorganization. Instead, the  
final behavioral organization is simply the one that existed at the time  
that the errors driving reorganization went to zero. If the errors had  
not dropped to zero, the organization would have continued to change and  
behavior would have continued to produce new and different consequences.

Grasping the difference between reinforcement theory and reorganization  
theory is like a figure-ground Gestalt switch, like turning the two  
faces into a vase simply by seeing them differently. Under reinforcement  
theory you say that a piece of food produced by an accidental press of a  
bar acts on the organism to make it press the bar again, and ultimately  
again and again. Under reorganization theory, you say that as the hungry  
animal roams the cage its organization is continually changing (because  
of the hunger) so that it is trying to control all sorts of variables by  
all sorts of means. It is not trying to get food; it is simply trying  
different behaviors because of being hungry. One of the new  
organizations leads it to try pressing the bar down, and the result of  
this is a piece of food which decreases the error that is causing the  
changes in organization. This decrease in error postpones the next  
change of organization. The behavior in effect at that point will then  
persist for a little while, and if it produces more food the error will  
decline further and the rate of change of reorganization will become  
still slower. So organizations that lead to an increase in hunger \_by  
any means\_ lead to immediate reorganization, while those that lead to a  
decrease \_by any means\_ postpone the next reorganization. If the  
situation is favorable, the result will be fewer and fewer  
reorganizations, and eventually none at all. The behavioral organization  
then in effect will be the one that prevents the error that was causing  
reorganization. It has to be, because any other organization would not  
correct the error and thus would not prevent further reorganizations.

Note that there doesn't need to be any rational connection between the  
immediate consequence of a behavior and the state of the organism. If  
there is a rule that says pressing a bar to turn a green light into a  
red one will result in a little jolt of intravenous food, the animal  
will learn the behavior that has the consequence of turning the green  
light into a red one.

This is why we were so impressed by the E. coli phenomenon: it is a  
direct demonstration of the principle of reorganization. When E. coli  
goes in the right direction, it does NOTHING to change the direction.  
Only while it is going in the wrong direction is there any process that  
alters its direction. The right direction is defined by default: it is  
that direction that leads to no change in direction.

Bruce's model actually illustrated that point, more or less by accident.  
The effective logical path that resulted in the correct changes in  
probabilities was the one that "punished" swimming in the wrong  
direction. The "pure" model, with fixed probabilities, is the basic  
model of reorganization, which says that if something is wrong, tumble  
right away; otherwise postpone the next tumble. So it is basically error  
that gets E. coli going the right way, and keeps re-selecting the right  
way every time it goes wrong. But there is nothing specific that says it  
is going the right way: going the right way is simply the condition that  
calls for no change. E. coli doesn't even know which way it is going.

The concept of reorganization gives us a new understanding of "reward."  
It is not that getting a reward causes the behavior that leads to the  
reward; it is that lack of the rewarding thing creates a state of  
reorganization that will cease only when the lack is corrected.

> **···**
>
> -----------------------------------------------------------------------  
> Best,
> 
> Bill P.