The Power of Reinforcement Theory

[From Bill Powers (950617.IHateToSay)]

Bruce Abbott (950616.1230) --

Some added thoughts after sufficient reorganizing time has passed.

     Reinforcement theory describes the process of acquisition and then
     assumes that the factors leading to acquisition continue to
     maintain the behavior in the steady-state. What could be simpler?
     This is what you're competing against.

Actually, I think I'm beginning to get the hang of explaining behavior
with the powerful competing method of reinforcement theory. Tell me if
I'm getting this right.

Reinforcer = R
Discriminative stimulus = S
Behaviors = Bn

···

------------------------------------------------
1. Learning to call a bottle "bah"

R+: mother smiles
R-: mother frowns
S+: bottle appears
S-: ball appears
B1: utter "bah", B2 = utter "dih", B3 = cry

If mother smiles only when the ball appears, the probability of uttering
"bah" when the ball appears increases. Eventually, every time the ball
appears, "bah" will be uttered.
------------------------------------------------
2. Learning to move a cursor to a target.

R+: Finger-target separation decreases
R-: Finger-target separation increases
S+: Finger left of target
S-: Finger right of target
B: Move finger left, right

If separation decreases
if finger left of target
  increase Pr{ move finger right }
else if finger right of target
  increase Pr{ move finger left }

If separation increases
if finger left of target
  decrease Pr{ move finger left }
else if finger right of target
  decrease Pr{ move finger right }

Eventually, when the finger is to the left of the target, the
probability of moving the finger to the right will be 1, and when the
finger is to the right of the target the probability of moving it left
will be 1. When the finger is neither left nor right, there will be no
discriminative stimulus and the finger will move neither left nor right.
-----------------------------------------------
Hey, this is a lot simpler than control theory! Let's get more
ambitious.
----------------------------------------------
R: remaining balanced and standing upright on deck of ship.

S: waves tilt ship to North, South, East or West

B: lean more to North, South, East, or West relative to deck

I don't need to repeat the analysis: eventually a tilt to the North will
result in a lean to the South, and so forth around the compass.
----------------------------------------------
It seems pretty obvious that reinforcement theory can explain the
learning of any behavior. Let's go all-out:
----------------------------------------------
R: Mathematicians say you're getting closer to or farther from
   proving Fermat's Last Theorem.

S: Mathematicians say current step of proof is correct or incorrect

B: execute any of a list of legal next steps

If mathematicians say you are getting closer to the proof
  If they say step of proof is correct
    increase Pr{ executing that step of the proof }
  else if they say step of proof is incorrect
    increase Pr{ changing that step of the proof }
else If mathematicians say you are getting farther from the proof
  if they say step of proof is correct
    decrease Pr{ executing that step of the proof }
  else if they say step of proof is incorrect
    decrease Pr{ changing that step of the proof }

So eventually you will cease to execute steps that lead mathematicians
to say you are farther from the proof, and will execute only those steps
that lead mathematicians to say you are closer to the proof. When
mathematicians cease to say that a step is either correct or incorrect,
there will be no discriminative stimulus and no further action. This
explains how the proof of Fermat's Last Theorem was completed.
---------------------------------------------
Not only is reinforcement theory powerful enough to explain the learning
of any behavior whatsoever, it isn't even necessary for one to
understand anything about brain function, physiology, physics,
chemistry, control theory, mathematics, linguistics, and so forth.
Reinforcement theory is the Grand Master Theory of behavior; all else is
mere detail.
-----------------------------------------------------------------------
Best,

Bill P.