[From Bruce Abbott (941121.1830 EST)]
Bill Powers (941119.0700 MST)
[Kahn hails the Enterprise and demands surrender, along with immediate
transmission of the plans to the Genesis machine. Thinking quickly, Kirk
offers to send the plans, but actually has Spock retrieve the Federation code
what will lower the shields of the Reliant....]
Bruce's model was very clever.
But . . . [notice the use of past tense.]
So this was a successful model of some organism. Some questions,
however, remain as to whether it is actually a reinforcement model and,
regardless of that, whether it is a model of a real organism.
Kahn: Your time is up for sending those plans.
Kirk: Here it comes...
The word reinforcement has been around for so long that it has become a
thing in its own right and we tend not to think of what it means. A
reinforcer is an observable physical something that is supposed to
reinforce a given behavior: that is, it has an effect on the organism
that a non-reinforcer like a piece of gravel would not have; to increase
the probability that there will be a given response to a given stimulus.
But what property does a piece of matter or an external situation have
to have in order to be able to create this effect? Reinforcement theory
is silent on this question. In fact, reinforcement is a dormitive
principle: a sleeping powder makes you sleepy because it contains a
Dormitive Principle; a piece of food increases the probability of
behavior because it contains a Reinforcing Principle. A piece of gravel
does not change behavior because it lacks the Reinforcing Principle.
Where on Earth do you get these ideas? Dormative principle? I don't think I
can name a single person doing work in the experimental analysis of behavior
who would subscribe to your conception of THEIR conception of reinforcement.
You are, as I and, apparently, quite a number of others have tried to tell
you, constructing a straw-person to demolish. The reasons that certain
consequences of behavior serve to increase the probability of those behaviors
are to be found jointly in the consequence and in the organism. Those
consequences that function as reinforcers must have certain properties, it is
true, but those properties are important only in relation to the organism, as
determined by its needs, structure, evolutionary history, and history of
experience. What IS true is that radical behaviorists (Skinnerians) believe
that it is better to avoid speculating on what those (unobservable) internal
conditions are and focus instead on identifying "establishing" conditions,
such as food deprivation. (It is a view, by the way, with which I strongly
disagree.)
In Bruce's model, the change in dNut across a tumble is said to have
reinforcing properties. However, no such properties have been given to
that change. Instead, we find inside the organism a complex mechanism
Well how else is reinforcement to exert effects on the organism if not through
mechanisms inside the organism? Shall we do without cause and effect? The
mechanisms specified by traditional reinforcement theory are admittedly vague
(association by contiguity somehow increases response probability) and thus
difficult to relate to specific physiological mechanisms. However, my
representation of learning in ECOLI4a is precisely as specified by the law of
effect: alteration of response probabilities as a function of experience. The
physical mechanism is unknown, so its input-output relationships are modeled
instead (i.e., functional relationship).
Bruce's model is not an example of reinforcement theory. The changes in
behavior that are seen are not due to reinforcing properties of dNut or
its changes, but to mechanisms inside the modelled organism without
which neither dNut nor any of its derivatives would have any effect at
all.
Horse puckey. Increases in dNut following a tumble increase the probability
of a tumble and decreases in dNut following a tumble decrease the probability
of a tumble. The model literally EMBODIES the law of effect, the very core of
reinforcement theory. But it is only a functional model: it captures the
essence of what happens without specifying the actual mechanism within the
organism by which the function is carried out.
Which leads us to the other question: is Bruce's model a plausible model
of a real organism? There is no question that it works, but is there a
simpler way, a simpler mechanism that we can propose for the insides of
the organism, that will create the same effect? The answer is clearly
yes. We can do completely without the calculation of the change in dNut
across a tumble. We need no probability calculations.
Sure. But such a model will not LEARN appropriate behavior to control its
perceptions. You (or evolution in the case of e. coli) must program the
correct relationships into it. This is what your model of human performance
on the e. coli simulation did--it skipped over the very rapid learning phase
the participants passed through and then modeled the terminal performance.
You then papered over this omission with phrases like "learning was almost
nonexistent." You have yet to produce a model that does what your human
participants did. ECOLI4a may not model the physical organism correctly, but
I believe it captures the correct functional relationships that the physical
mechanisms underlying the learning process in real organisms establish.
[Hans Blom, 941121]
Learning is, I think, not the same as control; a better analogy of
learning is the adaptation of the parameters of the control system in such
a way that an "optimal" control system results.
Things get slightly more complex when parameters are adjusted continuous-
ly, and not in a batch-like fashion, but the principle is the same. There-
fore, in my opinion, conditioning ought not be compared with control, but
with reorganization. In the process of reorganization, the consequences DO
select the resulting behavior.
Thank you, Hans! My point exactly. And it's really irrelevant whether you
phrase it as selection BY consequences or selection OF consequences. It all
depends on whether you prefer to be accused by your critics of animism or
mentalism: take your choice. The important fact is that the perceived
consequences of behavior determine which behaviors are retained and which are
eliminated.
Regards,
Bruce