goose-eggs; reinforcement theory; the Test

[From Bill Powers (951023.1633 MDT)]

Back safely.

···

------------------------------------------------------------------------
A note on a previous post, in the light of Bruce Abbott's report.

Gary Cziko (Sat, 15 Jun 91 22:24:13 -0500)--

     Concerning THE TEST of the controlled variable, let's say that the
     goose is controlling for a given amount of push (force) against the
     egg in a certain direction (back toward the test). We have already
     seen how the lateral movement of the egg seemed to be controlled by
     the goose (Lorenz calls this taxis). Now, if I wanted to make a
     robot arm that could do the same as the goose, I can't see how I
     could use anything but a control system for lateral movement, but
     it still seems to me I could use an open-loop system to bring the
     egg closer. I would just have it set up to provide a given amount
     of force to the egg, no more, no less and let it go. So what can I
     do with the goose to show that the far-to-near movement is
     controlled? What would be THE TEST that would make it clear to us,
     and hopefully to an ethologist as well, that it is not a fixed
     motor pattern?

The way Bruce Abbott reported this, the goose may be controlling for no
contact of an object with the lateral edges of its bill, and at the same
time is controlling (kinesthetically?) for a gradual sweep of the bill
toward the goose. As reported, there is no evidence that the goose is
controlling anything about the position of an egg.

The control for zero contact with the edges of the bill is done by
moving the bill laterally; if the left edge is contacted, move right; if
the right edge is contacted, move left. When the egg is centered under
the bill, it seems that the goose receives no perception from it,
because removing the egg does not alter the actions. I believe it was
Tinbergen who substituted a darning egg (made of porcelain or some such
material), and found that the goose's behavior was the same as for a
real egg. So the goose's behavior has nothing to do specifically with
eggs.

When I first heard of this, I wondered about the geometry of a goose's
field of vision in relation to its bill. Can it see an egg that is
directly under its bill? When the egg is off-center, is it the feel, the
sound of contact, or the sight of the egg that is the critical
perception? Try using an upside-down shovel blade to pull a ball of
cotton toward yourself. If you put the shovel over the ball, and the
ball drops into a hole in the ground, wouldn't you keep pulling the
shovel toward yourself anyway and then look to see if the ball was
closer?

Bruce made no mention of vision in his report. I've always wondered how
you remove an egg from under a goose's bill without the goose noticing
what you're doing. And how does the goose locate the egg when it notices
that it has rolled out of the nest? Vision must be involved somehow.

     Can it be that simple? Just add some mass to the systems and if it
     is still successful it is clearly not open loop?

All this would prove is that the position of the neck, bill, and head is
under control. I think Bruce's evidence shows that the objective
position of a real egg is NOT under control. Whatever perception is
being controlled by the goose, it's obviously not the one that a human
being would pick as the logical one. What the goose is going brings the
egg back to the nest as a side-effect, but there are no data (reported
so far) to show what the goose was actually controlling.

Re-reading my earlier comments and the original report by Lorenz, it
would seem that the force-control system is still a viable hypothesis.

It's interesting how many possibilities for controlled variables come up
when a PCTer considers the problem -- and how few the ethologists
considered.
----------------------------------------------------------------------
Hans Blom, 951023 --

     Where differences arise, is that PCT mostly studies control after
     learning has been finished (i.e with a steady state, fully
     converged model), whereas RT [reinforcement theory] mostly studies
     learning (i.e. convergence of the model) -- necessarily while
     control goes on.

Actually, reinforcement theorists don't seem to spend a lot of time
actually studying the process of acquisition of a learned control
behavior. To do that in, for example, a Skinner Box, you would have to
record all the behavior of a rat as it gradually changed from randomly
poking around in the cage to systematically pressing the lever. What is
actually studied, I think, is only the final systematic behavior, a kind
of projection of all the rat's learning behavior onto a single axis, the
pressing of the lever. Perhaps some researchers do study the process of
acquisition, but all the data Bruce and I have discussed is taken after
the basic learning of WHICH behavior to use is complete. Even in cases
where variations of behavior within a session are reported, much too
little data are described to allow testing a model of the adaptation
process. Basically, all you can get from the data is the change in mean
squared error with time.
-----------------------------------------------------------------------
Bruce Abbott (951023.1235 EST) --
RE: reinforcement theory vs. control theory

     What I meant was that reinforcement theory and PCT offer different
     mechanisms to explain the observations. RT says that certain
     consequences of behavior "strengthen" that behavior, making it more
     probable in the presence of environmental stimuli present during
     conditioning. These stimuli then "control" the probablility of the
     reinforced response. This is, as you are well aware, a different
     mechanism from the one proposed by PCT.

Somehow in our simplications we never did get around to re-introducing
disturbances into this discussion. Reinforcement experiments don't use
any independently applied disturbances, which is why it seems that the
organism is learning to produce particular _responses_. If varying
disturbances of the controlled variable were used, we would quickly see
that what is learned is not any particular response, but an internal
organization that can produce _whatever_ response is required to keep
the controlled variable (the reinforcement rate) at its reference level.

When you run control-systemn experiments without any disturbance of the
controlled variable, the same action will produce the same effect every
time. As a result, the observer can conclude incorrectly that it is the
action that is being learned. When you introduce disturbances, the
actions become decoupled from their consequences, and then you can see
that it is the consequence, not the action, that is being learned and
being brought under control.

This is a FUNDAMENTAL difference between reinforcement theory and PCT. I
wish you would discuss this point so I can be sure you see what I'm
talking about. The following makes me suspect that you don't see it:

     ... certain actions are observed to increase when those actions
     produce certain consequences. One can call such consequences
     "reinforcers" because of their apparent effect on the behaviors
     that produce them, and demonstrate that such consequences may have
     a similar apparent effect on other behaviors if those behaviors are
     allowed to produce them.

This would be true only as long as the same action produced the same
consequence each time -- i.e., when no disturbances are present.
-----------------------------------------------------------------------
Peter J. Burke (951023) --
[I'm horning in on post to Rick Marken)

     I have found that when I test for a controlled variable in a system
     of a marble in the bottom of a bowl, no matter how I disturb the
     position of the marble, it always comes to rest at the bottom of
     the bowl again. I surmise that the position of the marble is
     controlled. Now, tell me about perceptions and comparators and how
     PCT explains this!

You aren't doing the whole Test if you just disturb the marble and see
that it returns to center. Doing that only shows that there MAY be a
control system acting. To show that the marble is under control, you
have to identify the output of the system that is applying a restoring
force equal and opposite to the disturbing force, and you have to
indentify the means of perception by which the actual position of the
marble is detected by the system.

When you apply the whole test, you find that the restoring force is
created by your own deflection of the marble from the bottom of the
bowl: all of the restoring energy is put into the marble by your finger.
The only other "system" that might be involved is the Earth, which
generates the same force on the marble regardless of its position. Since
this force does not vary in proportion to the "position error", there is
no other system providing the "controlling" output.

If that's not enough, you will also fail to find any sensor such that
interrupting the ability to sense the position of the marble destroys
the ability to control.

So a marble in a bowl fails the Test.

     Certainly the "test for a controlled variable" doesn't necessarily
     tell us very much, even whether or not we have an ECU.

In this case, it tells us we do NOT have an ECU.
-----------------------------------------------------------------------
Best to all,

Bill P.