[From Bill Powers (941210.0735 MST)]
Martin Taylor (941209.1210) --
Appreciate your support re reinforcement as an explanatory construct.
However, there must be _something_ I can argue with you about...
Your discussion of reorganization and its relationship to apparent
reinforcement was inspirational. In fact I can't find anything to argue
with, so let me add some arguments on your side.
In a perceptual control UNIT (a single ECU with one scalar perceptual
signal and one scalar output signal) there CANNOT be different acts to
bring about the same perceptual consequence. All there can be is a
greater or lesser degree of the same act.
This is true, but it involves a rather advanced concept of what
"different acts" means. In operant conditioning, I believe it is thought
that different degrees of reinforcement are what cause different degrees
of the same act: the rate of behavior is said to be maintained by the
rate of reinforcement. Thus when the scheduled ratio is changed, the
result is said to be reinforcment of a different rate of behaving, and
that is why the behavior rate is different under different schedules.
Just what do we mean by "the same act?" Suppose a horse has been trained
to pull a cart at a steady speed. The Law of Effect would say that the
horse has learned that pulling at a certain speed when commanded to walk
has produced rewards in the past, and so when commanded to walk again it
should pull at the same speed to get the same reward. The consequence of
getting the reward has selected the response of walking when the command
"walk" is heard (or a slight tug of the reins is felt, etc.). Have I
stated the three-term contingency correctly?
But now the horse trained on level ground is taken out into the country,
where it is commanded to walk while the cart is going uphill. Now the
"act" of walking at a certain speed requires a considerable increase in
the muscle efforts that propel the horse forward. Nevertheless, the
well-trained horse emits the actions that result in moving at the
prescribed speed up the hill. It increases the muscle forces just enough
to counteract the disturbance due to the tendency of the cart to roll
back down the hill. It has never been rewarded for doing this, but it
does it anyway.
And now the horse reaches the top of the hill and starts down the other
side. Being well-trained, it continues to pull the cart at the same
speed when told to walk. In order to do this, however, it must
completely reverse all its muscle forces, because now the cart is
tending to go down the hill faster and faster and must be held back. So
the horse proceeds down the hill at the same pace, while exerting forces
in the reverse direction.
So -- is the horse producing "the same act" throughout this scenario? If
we view walking speed as the measure of the act, then the answer is yes.
If we look at the means by which this walking speed is maintained, the
answer has to be no, because surely pulling forward and pushing backward
are different acts, shifting the greatest efforts to different muscles
and reversing the effect on the environment.
This is an elaboration of your comments on microscopic and macroscopic
ways of seeing behavior. But I think that we need to focus even on the
behavior of a single ECU, because as far as I can see, different degrees
and directions of the same output effect are treated as different acts
under the Law of Effect, as I said above with respect to schedules of
reinforcement.
Behind the law of effect is a concept of causality in the environment:
1. Behavior -->physical laws--> Consequence
But as a general model, this is incorrect. A more correct model would be
Behavior -->physical laws--> Consequence
^
2. |
Variable Disturbances
Consequences are not produced by behavior alone. They are a joint
function of influences from behavior and influences from independent
processes in the environment. An even more correct general model would
be
Behavior -->physical laws--> Consequence
^ ^
> >
3. | Variable Disturbances
>
Variable disturbances
With model 3 in mind, we can see the mistake in supposing that an
organism can -- or should -- learn a particular behavior that will
always produce a particular consequence. If the disturbances arise from
invisible sources, as they most often do, they become known only because
the SAME degree of behavior produces, or seems to produce, a DIFFERENT
degree of the consequence.
Under conventional concepts of causality in behavior, it is impossible
to explain how a constant consequence can be created under the above
conditions. The Law of Effect can't apply, because if the same
consequence is to be repeated, a different degree of behavior must occur
every time, and it must be exactly that degree that will offset, moment
by moment, the effects of the variable disturbances. There can be no
association between a given amount of behavior and a given amount of the
consequence.
I have noticed, incidentally, that in some engineering treatments of
optimal control, the above situation is only partially recognized. The
variable disturbances are treated as statistical variables, in terms of
their means, variances, and spectral distributions, but there seems to
be no direct treatment of the case in which the behavior actually does
vary so as to offset the detailed changes in the variable disturbances,
leaving the consequence, in fact, essentially constant.
So persuasive is the concept of the first model above that
experimenters, without even thinking much about it, almost always set up
their experiments to exclude the variable disturbances shown in the
second and third models. At the very least they try to make sure that
nothing extraneous can directly alter the consequence independently of
behavior and the physical laws connecting the behavior to the
consequence, as in the second model. So the experimental conditions are
normally chosen to support the concept that behavior is directly
connected to its consequences.
When variable disturbances are successfully excluded, so that Model 1
correctly describes the situation, it is possible to proposed models
that explain the observations. The Law of Effect is such a model. It
proposes that the consequence has a causal effect on the way the
organism behaves in response to stimuli. The result of this causal
effect is to increase the probability of the behavior that previously
produced the same consequence. Since nothing but behavior can influence
the consequence in Model 1 above, this explanation will appear to be
correct -- that is, it will fit the observations.
But models must be tested under more than one condition to see if they
continue to work. When we introduce variable disturbances, it is no
longer true that the organism can produce the same consequence by
repeating the same behavior that previously accompanied it. Under the
Law of Effect, we would have to predict that with the disturbances
acting, the best the organism could learn to do would be to produce the
behavior that had the right effect _on the average_. But of course that
behavior would sometimes add to the effects of large disturbances and
sometimes small disturbances, sometimes disturbances in one direction
and other times disturbances in the opposite direction. So the
prediction would be that the consequence would vary according to the
size and direction of the disturbances, with only its mean value, over
many trials, being reliably related to the behavior.
What actually happens when disturbances are introduced is that the
consequence remains essentially the same, but the behavior that seems to
produce it becomes highly variable. That result is so unexpected, so
impossible from the point of view of Model 1 and the explanations of
behavior built on Model 1, that behaviorists seem unable to accept that
it actually happens.
The fact that it DOES happen tells us that we need a different model of
behavior. The Law of Effect, while plausible under certain conditions,
is not a general law, and in fact predicts incorrectly when disturbances
are allowed back into the picture.
ยทยทยท
-----------------------------------------------------------------------
Best.
Bill P.