[From Bill Powers (950629.1205 MDT)]
John Anderson (950629.0630 EDT) --
... if some process strengthens the synaptic connections in the
control systems whose activity produced the action that led to
those "good" results, then when those control systems become active
again, a similar neural response will be likely to occur.
This idea of "strengthening" synaptic connections when there is a
favorable result is, I think, too limited if taken literally. Suppose
your synaptic connections are set up so that when you perceive yourself
160 yards from the green, you reach your arm out toward the 8-iron and
grasp it. The ball, when you swing, then lands on the green. Now,
according to the synaptic-strength idea, the synaptic connections that
caused the arm to reach out to the 8-iron are strengthened. Supposed the
synaptic strengths are doubled, so the same input signals produce twice
as much output signal. The next time you find yourself 160 yards from
the green, therefore, you will reach out twice as far as before to pick
up the 8-iron. Of course your hand will now go right past the 8-iron and
grasp whatever is at this new distance -- your caddy's arm, perhaps.
In adaptive neural networks, if the only change in a synaptic connection
that could occur were to increase the weighting of that connection, the
only possible result would be that all connections would go to maximum
strength, some getting there before others but all reaching the limit
eventually. What is actually required is that the weightings be adjusted
upward or downward as needed -- a lowering of "synaptic strength" is
just as important to success as an increase.
What you really want is for the right action to become more likely than
the wrong action. The whole question in trying to model learning is to
figure out how that could come about. To say that it happens because of
strengthening synaptic connections is really just to say that it
happens; how strengthening synaptic connections could do that is not
explained. In fact, strengthening synaptic connection would probably NOT
achieve that end. Altering relative synaptic weightings both upward and
downward is probably closer to what is required for a workable model.
···
-----------------------------------------------------------------------
Best,
Bill P.