What's It All About?

[From Bruce Abbott (941203.1930 EST)]

[From Rick Marken (941203.1330)] -- What's it all about?

Rick, during your absence I was able to prove the PCT is all wrong and that
TRT is right. Behavior IS selected by its consequences, after all.

Just kidding.

Despite Bill's nice attempt to rescue Bruce from the jaws of ignominy,
those of us who have been awake during Bruce's "class" (apparently
only Tom and myself) know that this "learning" thing is a red herring.
The law of effect has no rules for "shutting off" when "learning" is
complete; if the law of effect is "true" then responses are _always_
selected by their consequences. When the environment changes so that
consequences are no longer polite enough to select the "right"
responses (the ones that result in control) then there is no more
control.

No, Bill was awake and you and Tom were asleep--or at least didn't follow the
lecture. (Bill wasn't just trying to be polite.) If you will take the
trouble to review what I have said about the law of effect, you will find that
I have nowhere claimed that the consequences are selecting the behavior in the
sense that you take it. The selecting takes place within the organism,
according to its own, internal criteria. (If there's a "red herring" here,
it's this business about selection BY consequences.)

The whole point of the E. coli modelling (and experiments) was to
show that control cannot be viewed as selection BY consequences.

The whole point of my effort at e. coli modelling was different: to
demonstrate how a set of appropriate behaviors could be acquired and
maintained as a function of an organism's experience with their consequences.
There has been flat assertions made that this was impossible in the e. coli
situation, as I recall.

The
law of effect model illustrates selection BY consequences with a
vengence. In the law of effect model of E. coli, responses (tumbles) are
always selected by their consequences; the probability of a tumble is
always changing but it will converge to a steady state.

Again, not really BY their consequences, per se. The consequences would have
no meaning to e. coli without its implicit internal reference. ECOLI4a's
learning mechanism selects the appropriate relationships between tumbling rate
and nutrient concentration according to this internal criterion, which in
effect defines what is "good" and "bad" from e. coli's perspective.

This steady state
probability can be called the "learned" state but nothing about the
model has changed; the model is no different (structurally) than it was.

How else can you represent learning? I assume that when I learn something
that some parameters of my system (brain/spinal cord) have changed (e.g.,
levels of certain chemicals in some of my neurons). I take it that you would
represent learning in some other way. How?

The current values of some model parameters are hanging around
values that produce interesting results (control). However, when
consequences start selecting the "wrong" tumble probabilities, the
wrong tumble probabilities are "learned". The law of effect model rolls
with the consequences; the control model _controls_ them.

You are only making my point here, which is that ECOLI4a learns what to do
from experience. If a response that led to a mild shock on the last three
occasions now produces a bit of food, will this not tend to increase
responding on the lever? Even though food occurs on only 25% of lever
presses? (I'm assuming that the shock is not so aversive that it completely
suppresses the response.)

The fact that the control model of E. coli does not learn is irrelevant to
the main point of the whole modelling exercise. The law of effect
model is a pure case of selection BY consequences and, under special
circumstances, this model seems to work (control); the probabilities of
response will converge to values that result in control -- but this only
lasts as long as the special circumstances remain in effect. Once these
special circumstances are eliminated (using my technique or Bill's) the
law of effect model results in a random walk rather than control.

The control model does not control them, either, unless its parameters have
been set up by the programmer so that the proper (negative) feedback occurs.
However, the "law of effect" model will learn such control, given that the
consequences its learning system monitors are such as to permit it. The
result is a lower-level control system whose parameters have been determined
by experience.

What the E. coli modelling shows is that control cannot by viewed as
selection BY consequences; neither the acquisition of the ability to
control nor the process of control itself can be viewed this way.

The e. coli modelling that you did had nothing to do with acquisition and
therefore has nothing whatever to say about it.

The point of this whole modelling exercise, from my point of view, is
not to "disprove" reinforcement theory or the law of effect (as Bruce
seems to think). The point is to show how the control model works.

I had a good laugh with that one. I seem to recall some very strong (and
satirical) statements on your part which came down to the idea that only fools
and idiots believe that there is any value in the law of effect.

It is my experience that one has a better chance of learning PCT by
taking classes in it than by giving them -- awake or not;-) The last
person I know of who tried to learn PCT by teaching it was William
Glasser of Reality Therapy fame. Need I say more?

Ouch. I think my classes have been quite instructive. However, since I am
evidently not, in your opinion, qualified to teach, I'll be happy to let you
take over. We were just discussing Thorkdike's law of effect and the behavior
of his famous cats in the puzzle box. Here's the scenario I was about to
explain:

Imagine Thorndike's cat in the puzzle box. It would like to get out of the
box and have a bit of that fish that's in the plate just outside. But what to
do? So the cat experiments. It tries this and that until, while taking a
swipe at a string that dangles from the ceiling, viola! the door opens. The
cat immediately rushes out and begins to devour the bit of fish.

Trial 2. The cat is back in the box again. What does it do? Open the door?
No, it repeats the whole random assortment of activities until it pulls on the
string again. The door opens and the cat rushes out.

Trial 3. Back in the box. Do this 'n that, then pull the string. The door
opens and the cat rushes out, a minute sooner than on the previous trials.

Trial 10. In the box again. As the door is latched, the cat pulls the string
and opens the door again. The cat rushes out. Elapsed time: 5 seconds.

Questions:

1. What brought about the changes in the cat's behavior? Why does it now
     pull the string immediately upon being placed in the box, rather than
     engaging in those other behaviors that were a characteristic of its
     earlier encounters with the interior of the box?

2. Do these changes have anything to do with the consequences of pulling
     the string? Explain.

3. What would happen if you disabled the cat's sensory apparatus so that it
     could no longer tell that its string-pulling had in fact unlatched the
     door?

Now I'll go sit in the back with the rest of the class; you come up front here
and fully answer these questions from a PCT perspective. After all, you're
the expert. Enlighten us. (;->

Regards,

Bruce