[From Bill Powers (960123.1345 MST)]
Martin Taylor 960123 10:45 --
That was a fascinating pair of comments on reorganization. The first is
the first evidence I have seen that might have something to do with
reorganization in the brain. The second makes intuitive sense -- that up
to a point, reorganization can be effective over a collection of
parameters being changed at the same time, but beyond that point the
advantage turns into a disadvantage.
With respect to the second point, I wonder if this might be an
underlying reason for development of the brain in levels. Or to go even
further, for the appearance of both sensory and motor "nuclei" in the
brain -- small volumes that seem to have uniform structures within them.
These specializations might represent the limits of the number of
similar systems that can be reorganized as a unit, with reorganization
altering all parameters within the unit. And this, of course, hints at
the possibility of specialized intrinsic variables and reorganizing
subsystems.
Good find.
ยทยทยท
-----------------------------------------------------------------------
Bruce Abbott (960123.1230 EST) --
As to what good it does to store remembered responses, Killeen's
theory attempts to account for learning as well as steady-state
responding. I haven't described the learning mechanism yet, but it
supposedly acts on those stored (and averaged) memories of
responses to -- you guessed it -- increase the probability of those
stored response characteristics being reproduced in future behavior
on the schedule.
I think we can agree that one of the problems a model has to deal with
is the way organisms seem to remember what to do to achieve a particular
end. But that isn't the only way to describe the problem. "Remembering
what to do" can equally well be described as "acquiring a connection
between a higher-order error and a lower-order reference signal." This
doesn't necessarily involve memory, unless you want to include the
establishment of a neural output connection in the definition of memory
(which I don't want to do).
This problem arises when the organism has to go to some place and
perform some action in order to produce food, water, or some other
important result of acting. Many of the output resources are pre-empted
in doing this; other results can't be achieved at the same time because
the machinery is busy. So this is definitely a problem in levels of
control, and of becoming organized so systems at the same level don't
conflict. Part of what higher levels of organization accomplish is the
elimination of direct internal conflicts. If one is hungry and thirsty
at the same time, a sequence control system can active the hunger-
satisfying system first, then the thirst-quenching system, or the other
way around. I think that the sequencing of mutually-excusive behaviors
can be studied as a control process.
With respect to the learning problem, the "memory" of how to do
something can be embodied in the way control processes are sequenced.
Once a specific hierarchical connection is established, a higher-order
goal is likely to be achieved by using the same lower-order control
system, because it is connected to that lower-order control system. But
there can be circumstances where different lower-order systems have to
be used to achieve the same higher-order end, in which case we have to
imagine a search process, or some way of varying which lower systems are
used when disturbances prevent the usual one from succeeding. I suppose
that different animals will show different abilities in this regard;
some simple animals will be unable to find a different way.
The basic HPCT model isn't set up to work this way, not explicitly. As
long as we think of communication between levels as being carried by
signals that vary only in magnitude, there's really no way for a control
action to involve sending reference signals to one lower system and then
a different one. I don't know if this process is already implicit in the
sequence and logic levels, or if we need a specific level that works
explicitly in this way, by switching its outputs to different
destinations. It would probably have to be a higher-level system, up
there with categories and sequences and programs, because switching is
inherently a discrete process (unless manipulation of weightings is
involved).
My feeling about this is that we're getting into complications that we
aren't prepared to handle. I prefer to look for experiments that define
small packages of behavior and to try to model them. When we get enough
packages, we can start looking for larger units of organization in which
those packages appear as subsystems. In typical operant-conditioning
experiments, we're looking at a lot of levels of control at the same
time, with the measures of behavior -- contact closures -- being unable
to separate simple variables from complex ones. When we see a low rate
of responding, is it because the animal is having difficulties in
executing the response itself, or because it hasn't yet learned to use
that response instead of another one? I think we have to build up from
the simple motor skills to the more complex relational and logical
skills, so we will know at each stage what we're measuring.
Just consider the Abbott Effect. The problem here might be (we don't
know yet) that rats or pigeons are incapable of producing repetitive
actions at continuously-variable frequencies. It could be that they can
either press repetitively or not press repetitively, without being able
to vary the rate. As long as we don't know that basic fact, our
interpretation of the relationship between rates of pressing and
schedules has no basis. This is why I like a bottom-up approach in small
steps. I think that most operant-conditioning experiments bring in all
kinds of considerations from simple motor skills to the ability to
recognize logical propositions, in one intractable chunk. Either they're
too complicated, or I'm too simple.
-----------------------
What may have been missed is that Killeen's formulation attempts to
account for steady-state average rates on a great variety of
reinforcement schedules, not just FR. He attempts to do so using
the same assumptions about memory and so on, which when applied to
different schedules yield different system equations and coupling
formulas. Merely adopting, say, an exponential function, to "make
the curve bend in the right place," might work for one schedule,
but probably would not provide a good description of performance on
other schedules.
My operant-conditioning models attempt to do the same thing, but without
changing parameters from one kind of schedule to another. In Opcond5,
you will notice that you can switch schedules from FR to VR to FI to VI.
The same control system with the same parameters behaves under all these
schedules. I can't say it behaves correctly, in part because Opcond5
isn't set up to produce handy numerical measures of the behavior, being
graphically oriented. But this is how I would like to continue: trying
to find a model that behaves correctly under all these schedules without
any change of its system parameters. I feel that this should be
possible. There's no reason to think that a rat or a pigeon can tell the
difference between schedules, or that it changes its organization from
one schedule to another. The right model will match real behavior no
matter what contingencies you think up for the environment part of the
loop.
----------------------------------------
An important theoretical question to resolve is how the delivery
of the incentive leads to the establishment of this reference.
(ME: Aw, Bruce!")
I'm not clear why the "aw, Bruce!" Do you think your question
needs to be answered first, that answering it will also answer my
question, or something else?
Delivery of the incentive is done by the organism itself. It is the
organism's own desire for the incentive, and its ability to try
different actions until the incentive appears, that accounts for the
establishment of the lower-order reference signals. The incentive by
itself can take no action, nor can it cause any particular behavior. It
is a dependent variable. It is produced by the organism for the benefit
of the organism. Even the word "incentive" is loaded with spurious
causal meanings. We are talking about food pellets and drops of water.
They have no "abilities." They have only properties. Whatever meaning
they have is given to them by the organism. If the organism didn't want
food, food would not seem to be an incentive. If the organism didn't do
something that made food appear, food would not appear. It's not the
food that plays a causal role, that acts as a prime mover. It's the
organism's reference level for food, without which there would be no
attempt to get food.
-----------------------------------------------------------------------
Phil Runkel (960123) --
Yes, I got the paper, and yes, I've told myself a thousand times to get
busy and reply to it. It's a great paper, it deserves comment, and I am
a jerk.
-----------------------------------------------------------------------
Best to all,
Bill P.