[From Bruce Abbott (950303.2100 EST)]
Bill Powers (950302.1200 MST)
Thanks for the details on programming the e. coli-style reorganization
process; we had talked of this before but it was nice to see it laid out
concisely. In fact, I found the entire post extremely interesting, and the
one following it.
We need a superordinate system that can
turn off the outputs from one control system and connect another control
system that is already organized to use the same lower-order locomotion
systems to go get the food (from any starting position).
Yes, and I can visualize how that might happen, by inhibiting and
disinhibiting appropriate neural inputs. It's quite clear to me that we
construct new control systems all the time (isn't that what we mean when we
say that we've learned how to DO something?) and then just call them as
needed, like subroutines. When these systems become automatic enough we
call them "habits" or "skills." Of course, the "subroutine calls" are just
outputs of higher-level control systems.
An event would be defined as a set of such vectors existing between two
times, t1 and tm. I suppose there would be a way to represent such an
event as a scalar perceptual signal, but we can leave that problem for
the future (the only reason for doing so would be theoretical, anyway).
The question then becomes what picks out an event from the continuous
stream of variable values. We might guess that events are separated by
zero entries -- nothing happening. Or perhaps they are marked off by
particular perceptions occurring. You can see why I'm reluctant to get
very far into this -- about two steps of this sort of conjecture define
a year's worth of experimentation. And we haven't even asked how an
event would be _recognized_.
There may be innate "rules" to help identify which perceptual events should
be associated. Garcia's learned taste-aversion studies showed, for example,
that a rat that had been given "bright, noisy, tasty water" (sips of water
with an unfamiliar flavor that was delivered along with a flashing light and
beeping tone) and then made ill by injection with lithium chloride
subsequently avoided water of that flavor but not unflavored water delivered
with the flashes and beeps. But "bright, noisy, tasty water" followed by
footshock produced avoidance of "bright, noisy water" but not of the
flavored water. In other words, rats associated flavor with illness, light
and sound with footshock. Also, a traumatic, single event may become
associated with a wide range of perceptual inputs: not just a simple,
discrete stimulus that immediately preceded it but general visual,
olfactory, and auditory inputs surrounding the event, which help to identify
the entire context in which the event occurred. Then there are other
"rules" such as sequence (causes before effects) and temporal proximity to
the event which usually play a part. (This by no means exhausts the list.)
This is beginning to look to me like the question of how the
relationship level of control gets organized, just as the previous
comments look like asking how the event-level gets organized. This is
suggesting to me that there may be specific learning processes
associated with the hierarchical levels. Reorganization would come into
the picture only when no fixed method could create a control process. If
we inherit the basic machinery for developing these different levels of
control, it's not unreasonable to suppose that the method of learning
such levels is actually what is inherited. So we have some excuse to
look for possible methods.
This is one of those things I was trying to get across during the "e. coli
wars." Reorganization may be required when all else fails, but it would be
far more efficient to apply more systematic methods when the situation
permits. This is what my "learning" e. coli was doing, and I presume that
living control systems have evolved (and may learn) a set of strategies for
gaining control which may be followed initially.
That fits with the concept of perceiving correlations as one type of
relationship. A single instance means, and should mean, little. Only
when there is a consistent relationship over a period of time should we
believe that it is real. So the mechanism for creating relationship-
perceiving systems should only gradually create an automatic perception
of relationship when there are covariances.
This is what I found with e. coli: if the change in tumble probability
following a single tumble was too great, the system failed to adapt--it just
kept "changing its mind" about what to do and thus never developed an
effective, consistent strategy (i.e., tumble if nutrient decreasing, inhibit
tumble if nutrient increasing).
However, other perceptual systems may have control--the smells, the
cracks and crevices in the corner of the chamber, the view of
things lying out of reach in the pan below the grid floor--and
control may return to these systems after the food has been eaten.
Let's try to keep the language straight: perceptual systems can't
control anything.
Oops. I meant to say "perceptual CONTROL systems" there.
Getting the food may have temporarily reduced the error in the system
involved in the former activity, leaving other errors that are larger
and entail different actions to reduce them. But with enough
observations we ought to be able to get an idea of what the major
control systems are. What does a rat do during its busy day? Not too
many different things, I would think.
Yes, or some new perceptual input may have caused a higher-level control
system to activate a different lower-level control system by producing a
large enough disturbance. As to what rats do with their time, not much in
the sterile environment of an operant chamber (though perhaps surprisingly
more than you might expect), but in a"real life" setting it would take a
very long list indeed to include all the rat's control systems.
But there are still a lot of gaps to fill, particularly how to deal
with what in operant terminology is called "stimulus control."
What mechanism selects which control system will have current
access to the lower-level systems required to produce the necessary
behavioral outputs? On what basis?
A lot of this would simply fall out of a sufficiently complex model, in
which many systems are controlling for many reference conditions at the
same time.
Yes, I can see that; this is, I think, what Rick Marken was proposing.
If you see the whole hierarchy as a collection of control systems all
trying to correct their own errors at the same time, you can get an
inkling of what the final picture will look like. All the interactions
will seek states in which overall error is minimized. As learning takes
place throughout the hierarchy, this minimum will gradually, over months
and years, get smaller and smaller until some irreducible amount of
error remains. If this minimum is too high, you get an anxious screwed-
up organism. If it's exceptionally low, you get a happy effective rat or
person.
Yep, that's what the world needs--happy, effective rats. D'ya ever wonder
what it would be like to study some nice, simple system like rocket
propulsion or nuclear synthesis? Man, those physicists have it made!
Bill Powers (950303.0945 MST)
I think we need to take these questions seriously. We have never tried
to model the situation where one perception appears to work as a signal
rather than as a controlled variable in itself. It's possible, of
course, that this interpretation of the role of the perception is
misleading, but we need to find the "correct" description and show that
it makes at least as much intuitive sense. The problem is similar to
that of showing that some "stimuli" should really be interpreted as
disturbances. Once you can see exactly what is disturbed, and how the
control action counteracts the disturbance and _appears_ to be caused by
the stimulus, the PCT interpretation becomes at least as believable as
the other one. We need to do this for the case of "discriminative
stimuli" or else admit that some stimuli serve as triggers for other
control processes.
I'm very glad to hear this, first because the concept of "discriminative
stimulus" ranks second only to "reinforcement" in importance within
traditional behaviorist theory and thus needs to be at least addressed by
PCT and second because it means you agree with me that it is an empirical
issue within PCT that requires serious attention and--ta da!--research. It
is also the subject of some very confused thinking (as I view it) within the
experimental analysis of behavior. This will require a bit of explanation.
The concept first arose in the context of providing a simple cue (such as a
light) to signal that a reinforcement contingency was now in effect: Light
on, lever-pressing activates feeder on, say, a VR-10 schedule; light off,
lever-pressing fails to activate feeder (extinction schedule). The rats
learned to lever-press when the light was on and to do something else when
the light was off. This observation required that a new term be invented
and defined:
Discriminative stimulus: A stimulus in the presence of which a response is
reinforced.
Along comes a new situation: green light = VR-10, red light = extinction.
The green light is the discriminative stimulus. What's the red light? So
we need a new term invented and defined:
S-delta: A stimulus in the presence of which a response is NOT reinforced.
(The "delta" should be given in superscript as the greek letter that looks
like a triangle.) In contrast to S-delta, we'll call the discriminative
stimulus "S-D" (S superscript D).
But in the first situation, wasn't S-delta the ABSENCE of S-D? Well, yes.
So the ABSENCE of a stimulus is a stimulus? Must be. Hmmmm. And shouldn't
BOTH S-D and S-delta be called discriminative stimuli? Looks like we need a
new definition of descriminative stimulus:
Discriminative stimulus: a stimulus that sets the occasion for a response.
That seems to work better: Green light = press the lever; red light = go do
something else.
More trouble: Green light = VR-25 shock delivery, red light = no shock;
FR-5 food delivery in either case. Responding is suppressed during green
(but still occurs), recovers during red. What's the green light? The red
light? Who's on first? Why didn't I take up something easier like nuclear
physics or fractal geometry?
Well, despite the confusion, everyone seems to know what everyone else MEANS
by these terms, even if they evade precise definition. These stimuli
indicate what contingencies (relationships) are "in effect" at any given
moment, and the well-trained rat's behavior changes instantly (if that is
what is required) when these stimuli change. The changes may be in the
goals being pursued (earning food, investigating the chamber) or in the
methods by which the goals are pursued (lever pressing, turning, both for
the same access to food).
There are two contributing lower-order perceptions under the organism's
control (peck, turn-in-circle) and two that are independently variable
(Red, Green). As I interpret the description, there are two additional
relationships imposed by the environment: red XOR green is true, meaning
that the light is either green or red but never both, and peck XOR turn-
in-circle is true, meaning it is physically impossible to do both at
once.
Correct.
The only way to find out what logical function (if any that we can
understand using Boolean algebra) describes the perceptual function is
to test various hypotheses -- the good old Test again. The actual
behavior involved, pecking or turning in a circle, is of little interest
in itself, however striking it may be to the experimenter. What we would
be investigating would be the logical function of perceptual variables
that is under control by the organism.
To do this it is also necessary to look carefully at the logic of the
experiment. What is the contingency for the case when both lights are
on, or both off, or when a light of a new color is shown? If the animal
turns in circles, pecking at the key once each time around, what is the
contingency? It's very easy to set up the logic of an experiment with a
particular set of relationships in mind, and forget that you have to
cover all combinations of true and false, not just the ones you first
thought of. Bill Leach will no doubt support this observation;
forgetting to cover all logical possibilities is a pitfall of electronic
logic design. Whichever condition you forgot to provide for is almost
certain to be the next one that occurs.
Easy to forget the other relationships? Like overlooking the fact that, if
light-on is the discriminative stimulus, that light-off must be something,
too? Now who would ever do that? (;->
Start with, say, a vertical line projected on a pigeon's response key as S-D
(reinforcement) and a horizontal line as S-delta (extinction), then train
the pigeon until it responds on the key only during S-D. What happens if
you now project a line on the key at a 45 degree angle? Answer: an
intermediate rate of keypecking. In fact, the rate will vary smoothly and
continuously as you vary the line angle smoothly and continuously, from zero
rate at horizontal to max rate at vertical and back to zero when you reach
horizontal again. The plot of this relationship is called a gradient of
generalization. There's a huge literature on this which includes some
surprises, but I think I've said enough for now. Do I sense another
research project coming on? (:->
Regards,
Bruce
