Autoshaping

[From Bruce Abbott (951203.2005 EST)]

Bill Powers (951202.2350 MST) --

So, once you have accepted an error-correcting mechanism you have,
whether you realize it or not, accepted that the environment does NOT
determine where the bird will peck, or even if it will peck at all. At
that point you have to stop talking about an "unconditional stimulus,"
because you have just denied that it is the stimulus that directs the
pecking toward itself.

In EAB, to say that some environmental stimulus "determines where the bird
will peck" is only to say that there is an empirical relationship between
where the stimulus is and where the peck will be directed. The stimulus is
not assumed to "direct the peck toward itself" (there you go with animism
again) but rather, to innately "release" a peck which the bird's brain
mechanisms direct toward the releasor. The source of energy is not in the
target, but in the bird; the target only acts to "pull the trigger," as it
were, of a system already primed to act. The "S-R" mechanism assumed to be
at work here would be the ethologist's "innate releasing mechanism (IRM),
which acts as the trigger; what follows CAN be open loop OR an organized,
feedback-directed pattern of behavior (the "fixed action pattern").

Since the bird can direct its pecks at things, it would be assumed that the
bird's brain is so organized as to perform the necessary computations and
corrections. How it does this would be left unspecified, although as I
suggested, I don't think anyone in EAB would have a problem with the notion
that a negative feedback error correction system is as work. However, this
would be _described_ in terms of external variables, as though a direct
causal link existed among the relevant variables.

The only way you can go on speaking as if there
is an S-R connection is to have missed the point of agreeing that there
is an error-correcting mechanism involved.

Well, you are the only one talking about an "S-R connection" as a simple
open loop system here, and that is where the problem lies.

What does "signalling food" have to do with classical conditioning? As I
understood it, you begin with a US -> UR connection, an S-R connection
that links an environmental stimulus to a response that it always
produces. In the present case, the US is (correct me if I'm wrong) the
sight of something that might be edible, and the UR is pecking at the
thing. So before conditioning, the bird simply wanders around pecking at
things that might be edible. These pecks are not aimed anywhere in
particular; that is, any object that appears edible might be pecked at.
The key would not be particularly favored; indeed, according to what you
have said, it is difficult to get the bird to peck at the key to get
operant conditioning started.

Again, you are employing an overly simple conception of what is said to be
going on. Certain propeties of things, when perceived by a bird in the
right state (e.g., food-deprived in this case), serve to release a pattern
of directed pecking. For classical conditioning to occur, a stimulus must
serve as a signal for some event, and that event must serve to release a
pattern of behavior in reflex-like fashion. Supposedly, the ability to
release that same pattern is confired through the process of classical
conditioning to the formerly neutral stimulus. After a number of pairings
between key illumination and the subsequent delivery of grain, the
illuminated key would begin to serve as sufficient stimulus to trigger the
IRM, which would then set into motion a directed peck at the illuminated key.

Compare what I just said with your own description:

So if the CS is presented just before a peck at
something that might be edible, the bird will gradually come to peck at
the edible object when the CS appears, without waiting to see the object
(a neat trick). In order for this edible object to be the key, the CS
must be presented just before the key is pecked. Then the key will be
pecked when the CS appears, without waiting to see the key.

I think you can see that this is an incorrect description. The CS is
becoming a substitute for the natural releasing stimulus; what is released
is a peck toward the (spatially localized) releasing stimulus, which in this
case is the illuminated key.

I confess that I'm already completely confused. What exactly are the US,
the CS, the UR, and the CR? And what is the role of the food delivered
when the key is illuminated, with or without a peck?

The US is presentation of grain in the hopper, the UR is pecking directed
toward grain in the hopper, the CS is the illuminated key (which reliably
preceded the presentation of the grain) and the CR is pecking directed
toward the illuminated key.

When the grain is presented without a keypeck, the reliable presentation of
the illuminated key just prior to grain delivery serves to convert the
illuminated key from a neutral stimulus to one capable of releasing a
keypeck directed toward the releasing stimulus (i.e., the illuminated key).
When a _keypeck_ in the presence of the illuminated key is immediately
followed by contingent presentation of the grain, this event reinforces the
keypeck, i.e., pecking the illuminated key becomes more probable.

The former is classical conditioning; the latter is operant conditioning.
In classical conditioning, the CS functions as a substitute for the US,
releasing the response. In operant conditioning, it doesn't matter what the
pigeon's innate response to grain is; whatever response produces grain
access will become conditioned. In autoshaping, both the classically and
operantly conditioned responses happen to be keypecks.

In Pavlov's experiment, the US was the puff of meat powder into the
mouth, and the UR was salivation which dissolved the dry powder and made
it ingestible. The CS was a bell rung just before the puff of meat
powder. If we use a parallel construction, the US in the autoshaping
experiment ought to be the delivery of food, and the UR the act of
pecking at the food and eating it. The CS should be the combination of
illumination of the key and a peck at the key (the peck being considered
not only an action but a stimulus), since these events immediately
precede the food stimulus. But that doesn't seem to be how you're
interpreting the elements of the experiment.

No, in Pavlov's experiment, the dog is not required to salivate in order to
get fed; food is delivered regardless of what the dog actually does.
Likewise, in autoshaping, the pigeon is not required to peck at the key in
order to get fed; grain is delivered regardless. So the CS is only the
illumination of the key, not key illumination plus peck.

It seems to me that the success of autoshaping depends on just one
factor: a bird is much more likely to peck at an illuminated key than at
anything else. This, if anything, is the US-UR pair. Since this
unconditional response by itself would be sufficient to get operant
conditioning going, I don't see why any reference to classical
conditioning has to be made.

No, pigeons have very little tendency to peck at illuminated keys, as I said
before. This is why the success of the autoshaping procedure created such a
stir.

What happens during the time that the key is
illuminated but before the noncontingent delivery of food occurs?

    Nothing is _programmed_ to occur.

But isn't the apparatus programmed so that if a keypeck occurs while the
key is illuminated, the food will immediately be delivered? I asked

Does the food appear immediately upon a keypeck when the key is
illuminated?

And you said Yes. I guess I don't know what you mean by saying that
nothing is programmed to occur. Just to make sure, I asked

is not
the delivery of food contingent on the keypeck just as in operant
conditioning?

And you said Yes.

Apparently I misunderstood what you were asking. When you asked, what
happens during the time the key is illuminated but before noncontingent food
is presented?, I thought you meanht "in the absence of a keypeck." The
answer is nothing. But when there IS a keypeck during this period (your
second question), food is delivered immediately.

Is it part of operant or classical conditioning theory to assume that
pigeons do not like to waste their efforts? If that's true, why do we
observe superstitious behavior?

No, it has nothing to do with it. I mentioned it only to point out that
this result would be contrary to what one might expect on the assumption
that the pigeon is trying to maximize the ratio of benefit to cost.

Doesn't all this sound a bit ad-hoc even to you? If keypecking is an
innate response sequence (pretty hard to imagine; pecking, sure, but
KEYpecking?) then why is it so hard to get the pigeon to peck the key in
ordinary operant conditioning?

Boy, you really ARE confused. Who said KEYpecking was innate? I hope
things make better sense to you now that I've tried to clairify it!

If pecking is released by certain
perceptions under the right conditions, then sure, what happens would
happen. Obviously the perceptions are the right ones and the conditions
are right; you just said they are. But aren't those assumptions being
made out of thin air specifically to explain what happens?

It's purely descriptive. One observes that, as you say, pecking is released
by certain perceptions under the right conditions, and is directed toward
the releasing stimulus. In the past, one has observed that when a neutral
stimulus is paired in a certain way with one capable of releasing a certain
pattern of output, the neutral stimulus gains a parallel ability to do so.
One observes that the conditions present in autoshaping are those required
for classical conditioning (once it is recognized that pecks can be
"released" by stimuli and thus may qualify as URs). Empirical tests rule
out other explanations. It that thin air?

Isn't that
what "ad hoc" means? What's this "releasing" business? Is that different
from "causing?" From "triggering?" From "eliciting?" From "setting the
conditions for?"

One CAN think of any disturbance as A (not THE) cause of a control system's
action, yes? Why can't biological systems be so constructed that certain
perceptions automatically set into motion a series of actions? These
actions could be produced by control systems, couldn't they? As seen from
the outside, it would appear that these perceptual inputs were "causing" or
"triggering" or "releasing" or "eliciting" a response or even a pattern of
activity, defined not in terms of muscle contractions but in terms of the
effect on environmental variables (e.g., the key gets struck).

"Setting the conditions for" is different. You may more prone to eat when
you are hungry, or when you pass a freezer containing your favorite
icecream. Neither your hunger nor the sight of the freezer triggers or
releases eating behavior; but the likihood of your doing just that is in
fact higher when those conditions are present.

In a discriminated operant, the illumination of the key signals that pecking
at the key will be followed by the presentation of grain (not true when the
key is dark). The pigeon's peck is not released or "forced" in this case;
nevertheless, the pigeon may peck the key more often when the key is lit
then when it is not.

Bruce, this is a horrible, sloppy, shifty way of constructing an
explanation. Why do I have to ask all these questions? Can't you just
lay out the definitions and the logic of the explanation and show how it
works? Every time I raise a new question, you seem to have a new answer.
I feel like John Cleese being sold a parrot by Michael Palin.

I believe I have offered the same explanation each time; only your
misinterpretation of it has changed over the repetitions. I hope I've
succeeded in getting it across to you this time.

By the way, I am getting the strong impression that you (and Rick) believe
that these reinforcement-theory explanations are the ones I subscribe to.
While taking potshots at the theory, I hope you don't misdirect them at the
messenger. I'm trying to show that these explanations, so far as they go,
are internally consistent and sensible, given the assumptions of the theory
and the observations. I'm not arguing that they are correct, but I think it
is important that you understand how the theory would be applied here.

I'm overdue for dinner, so I'll address your last point later.

Regards,

Bruce