[From Bruce Abbott (950616.0850 EST)]
Bill Powers (950615.1230 MDT)
This is the opposite of what I meant by "manually" changing the model:
you're "mentally" changing it by changing what you mean by S+, without
making any corresponding change in the model.
Yes, the point was that S+ and S- are NOT what gets "manually" changed to
make the model work with repellants. Perhaps I misunderstood, but you
seemed to be saying they were:
In order to make it adapt as appropriate for an
attractant or a repellent, something would have to detect the kind of
input that is being sensed, and change the definition of S+. You can
manually change the definitions, but that's not "adaptation."
I needed to address that point before I could go on to state what _does_ get
changed.
OK, after all that talk you ended up realizing that you have to
reprogram the model to make it work with a toxin.
No, I had this in mind from the start. But I had to get Point 1 out of the
way before proceeding to Point 2.
Your suggestions about
how to do this are reasonable. When you implement them, you will have a
truly adaptive model, if not the simplest one possible.
Good, we agree.
So is learning NOT "behavior determined by the internal structure of the
organism and the properties of its environment?" When we model anything
that resembles learning, we come down to specialized circuits that have
the effect of modifying parameters in other circuits. What else could
learning be?
Of course it is. But not all "behavior determined by the internal structure
of the organism and the properties of its environment" involves learning,
which is a very different point and the one I'm trying to make here.
The notion of "hard-wired" isn't very useful: if you hard-wire
multipliers into the system, it can change its organization just as
readily as it could do under "software" control. Anything you can do
with a program can be done with hard-wired components. If you stop and
think about it, anything you can do with a program is ALWAYS done with
hard-wired components.
Yes, but consider the control model for e. coli. The model has a certain
structure, and so does the environment. Throughout the simulation the
model's structure never changes. Either it controls or it doesn't. That's
what I mean by "hard-wired": the structure is in place (including parameter
values) and is not modified _on the basis of experience_.
A reinforcement model, in contrast, is built on the assumption that
experience is a crucial factor determining how the organism will behave in a
given environment. Changes take place in the structure (system parameters),
owing to the consequences of the organism's own behavior, and these changes
alter the way the organism will behave in that environment in the future.
Steady-state behavior emerges from these changes acting in combination with
current environmental conditions.
Thus, if there is no basis for learning (e.g., responses have no systematic
consequences), there is no reason to expect that a learning-based model
should apply.
If we remove, one at a time, all the examples in which control theory
works as well as or better than reinforcement theory, sooner or later
there will be no pure examples of reinforcement left. That is, there
will be no case in which ONLY reinforcement theory can explain the data.
That sounds like one long, tedious research project to me. Be my guest.
It may be true that "the failure of the reinforcement model in this
situation would say nothing at all about its validity", but that is also
true of SUCCESS of the reinforcement model. All that success shows is
that the reinforcement model fits the observations, not that it explains
them correctly in terms of the actual mechanisms involved.
True enough, but the same can be said for any model (see Popper).
Wherever both PCT and reinforcement theory can explain the same
observations, choosing between the models has to depend on something
other than the fit of predictions to data.
Yes, such as parsimony and "elegance." Initially those were the only
criteria that favored Copernicus over Ptolemy. But eventually other, more
direct ways to test the two views were developed, based on structural
differences in the models which led to different implications (e.g.,
crystalline spheres were inconsistent with the orbits of comets). The same
will be true, I think, for PCT versus reinforcement theory.
From what Bruce says, if reinforcement theory doesn't explain how we
learn to do the task in your new program, that will only show that
reinforcement theory isn't _expected_ to work in that situation.
Not so fast. If you can show that behavior is maintained under conditions
in which the consequences of behavior are unsystematic (i.e., there is no
consistent reinforcement or punishment of responses), you have demonstrated
that reinforcement is not a _necessary_ condition for behavior. But is it a
_sufficient_ condition? Your experiment won't say.
A serious problem for PCT in its battle against reinforcement theory is that
PCT as currently developed does not offer a really well-developed
explanation for how the observed performance under given conditions is
acquired. Where do all these control systems come from? How do they get
established? Reinforcement theory describes the process of acquisition and
then assumes that the factors leading to acquisition continue to maintain
the behavior in the steady-state. What could be simpler? This is what
you're competing against.
Regards,
Bruce