The Law of Effect

[From Bruce Abbott (941121.1600 EST)]

In my previous post I promised to explain why I have an affection for
Thorndike's "law of effect." What I hope to demonstrate is that the law of
effect, as conceived by Thorndike in 1911 (and not necessarily by those who
came after) contains some ideas that are very close to certain concepts of
PCT. I'll start by repeating Thorndike's criteria for satisfying and annoying
states of affairs:

     By a satisfying state of affairs is meant one which the animal does
     nothing to avoid, often doing such things as attain and preserve it. By
     a discomforting or annoying state of affairs is meant one which the
     animal commonly avoids and abandons.

Thorndike is here defining how you will know these states of affairs when you
find them. It is important to note that the word "stimulus" appears nowhere
in this description. I hold that these statements constitute a crude version
of The Test. You will know a satisfying state of affairs because the animal
will voluntarily approach it (reduce error between present situation and
reference) and will do such things as to preserve it (i.e., oppose
disturbances). You will know an annoying state of affairs because the animal
will resist being forced into contact with it (i.e., will oppose such
disturbances).

These states of affairs are thus not, for Thorndike, "things" out there in the
environment with properties independent of the animal's state of being. They
are "states" of the organism which behavior serves to attain or avoid.

Having established that a particular state of affairs is, in fact, a goal for
the animal under the test conditions, Thorndike now shows that, among a
variety of behaviors being activated within a "situation," those that are
accompanied or followed by such a state of affairs will become more likely.
That is, those responses that yield approach toward the satisfying state of
affairs (i.e., reduce error) will become "more firmly connected to the
situation, so that, when the situation recurs, the response will be more
likely to occur."

For Thorndike the mechanism by which responses became "more firmly connected
to the situation" was associative: the complex of stimuli (including internal
states of the animal) that constituted the "situation" for the animal became
more strongly associated with responses that turned out to move the animal
toward the reference state. In today's vocabulary, the situation served as a
complex discriminative stimulus affecting the probabilities of the various
behaviors that occur in its presence, in the same way that a "hint" may affect
the probability that a person can recall the correct answer to a question.
Given the state of physiological knowledge of the day, Thorndike wisely left
out any speculation about how these response probabilities are represented in
the nervous system.

It is useful to distinguish two "laws of effect." The first is what has been
called the "empirical law of effect," which is simply descriptive. It says
that when you follow a response with some consequence that can be shown to
function as a goal for the animal, that the response becomes more likely on
subsequent exposures to the situation in which that response occurred (true,
by the way, only so long as the consequence of the response continues to
function as a goal). It is a Baconian generalization. The second law of
effect is theoretical. The theoretical law of effect specifies an associative
mechanism by which behaviors and situations become "connected" in such a way
as to alter the probabilities of the behaviors that occur in those situations.
The theoretical law of effect can be disproven without affecting the truth or
falsity of the empirical law of effect.

Now it seems to me that PCT and the empirical law of effect are in no way
contradictory; in fact, the principle it embodies (Thorndike called it
"selecting and connecting") describes one method by which animals develop
behaviors that produce the appropriate negative feedback within a perceptual
control system: (a) vary the behavior, (b) observe the consequences of those
behaviors on the controlled perceptions, (c) select those behaviors that
appear to correct the perceptual errors and suppress those that appear to
induce them.

This is the principle I built into my successful ECOLI4a model. ECOLI4a tries
out tumbling under different conditions. If tumbling makes the rate of
nutrient change more positive (a satisfying state of affairs for e. coli)
under a given condition (situation), the probability of tumbling in that
situation increases, so that, when the situation recurs, the response is more
likely to occur. If tumbling makes the rate of nutrient change less positive
(or more negative), the probability of tumbling in that situation decreases,
so that, when the situation recurs, the response is less likely to occur. The
result is that ECOLI4a quickly establishes the correct error-reducing
relationship between behavior and the error, i.e., negative feedback.

I believe I have established that Thorndike's conception of satisfiers and
annoyers is congenial to PCT: indeed PCT explains WHY, HOW, and WHEN these
states of affairs have their observed effects on the organism's behavior. I
have shown that the law of effect represents a simple principle whereby
responses that successfully counter error in a perceptual control system get
selected. It is not only compatible with PCT, should be an important element
of it, for it is an ubiquitous method by which organisms reorganize their
perceptual systems.

While there is plenty wrong with traditional reinforcement theory, the law of
effect as Thorndike conceived of it is not the problem. In attacking the law
of effect, you are, I believe, throwing out the baby with the bath water. As
described here it is not an ALTERNATIVE to perceptual control theory, it is a
primary means by which effective control systems are created and perfected.

Regards,

Bruce