[Hans Blom, 950713]
(Bill Powers (950712.0825 MDT)
Rick Marken (950711.1615) --
So reinforcement stops "strengthening" behavior when responses
have been learned and are being "maintained". How does the
reinforcement know when to stop strengthening and start
maintaining? ...
Bill:
Good questions. It's obvious to me that two quite different
mechanisms are involved; calling them both "reinforcement" is a
mistake.
Yes, very confusing. If we view the organism as an adaptive
control system (as is my hobby), one mechanism is learning /
model building / model tuning / model parameter estimation, all
basically an attempt to establish the relationship between what I
do and what I perceive as a result of that doing, i.e. an attempt
to learn the laws that govern the "behavior" of the world out
there. This mechanism never stops, but as long as the world (the
effect of actions) does not change, the uncertainty of the model
parameters will steadily decrease, until so much certainty has
been accumulated that -- in a practical sense -- the mechanism
might as well have stopped. In an adaptive control system living
in a constant world, once the parameters have sufficiently con-
verged, further observations are not required anymore; learning
is, for all practical purposes, complete.
The second mechanism is control. As long as the world-model is
still inaccurate, the quality of the control might be bad. But
control quality will improve as the model's parameters converge.
When learning is complete -- i.e. the model could not be better
tuned -- control will be at its most accurate.
Being aware of both these very different mechanisms with their
very different goals helps to understand the behavior of orga-
nisms, in so far as they resemble adaptive control systems.
Greetings,
Hans