Approach gradients

[From Rick Marken (2002.05.24.1000)]

Bruce Nevin (2002.05.23 11:17 EDT) --

maybe we should reserve NEC for the phenomenon and
use related terms that refer specifically to the class of mechanisms
being proposed.

I don't think it's a good idea to call the phenomenon NEC because it implies that
we can see the error (E) that is non-monotonically related to output. I think I
like the name used by Dollard and Miller to describe this phenomenon: "approach
gradient". An approach gradient is observed level of output as a function of
distance from a presumed goal. If this gradient is non-monotonic -- if it has a
component where an increase in output is associated with a decrease in distance
from the presumed goal -- then this phenomenon _might_ require an explanation in
terms of a non-linear error curve.

Bruce Nevin (2002.05.23 23:28 EDT)--
Bill Curry (2002.o5.23.1000 EDT)--

Can either of you think of a way to convince yourselves, experimentally, that an
observed non-monotonic approach gradient -- such as the one observed by Dollard
and Miller -- is based on sequence control (your preferred explanation) rather
than a NEC?

Best regards

Rick

···

--
Richard S. Marken, Ph.D.
The RAND Corporation
PO Box 2138
1700 Main Street
Santa Monica, CA 90407-2138
Tel: 310-393-0411 x7971
Fax: 310-451-7018
E-mail: rmarken@rand.org

[From Bruce Nevin (2002.05.24 16:01 EDT)]

Rick Marken (2002.05.24.1000)--

Can either of you think of a way to [show], experimentally, that an
observed non-monotonic approach gradient -- such as the one observed by Dollard and Miller -- is based on sequence control [...] rather than a NEC [output function mechanism]?

Since the NEC output function mechanism by definition produces the output effort gradient that Dollard & Miller report, there is no way to distinguish it from any other mechanism that produces that gradient. It is an untestable hypothesis.

The obvious thing to do is to show that existing mechanisms posited in PCT produce that gradient. If that can be shown, then it follows that the hypothesis of an output function mechanism is unnecessary, redundant, and without evolutionary motivation.

That much can be done entirely with modelling.

There is a problem referring to this as a gradient based on the Dollard & Miller experiment. The gradient they report is already present in the experimental apparatus, which increases its resistance to the rat's movement as the rat approaches the food. The observed gradient is actually imposed, by means of the harness, upon the rat's movement. The rat is controlling "walk over to food"; the harness impedes the movement, a disturbance; the rat resists that disturbance. Because the disturbance is gradient, the rat's efforts to resist the disturbance are gradient. On the face of it, this is straightforward PCT. I assume the resistance of the harness was not the same throughout the rat's progress, first because that is not how I understood the description, and second because, if the rat was speeding up against uniform resistance, I would expect it to speed up without any disturbance, i.e. normal locomotion with no harness attached. After all, there is a certain amount of resistance in the physics of walking from A to B -- if there were not, there would be no need to expend energy to do it.

The rat's stepping actions result in the continuous movement of the rat's body. If disturbance is applied to one discrete step and not to others, there will be no gradient, but PCT predicts that there will be an increase of effort in that step to counter that disturbance.

The hypothesis that I advanced is that the Dollard & Miller experiment involved control of a sequence comprising at least two steps, (1) move mouth to food and (2) eat. The first part of this is in turn made up literally of steps as the rat uses its legs to pull its body toward the food.

A second hypothesis that I advanced is that the gain in control of later steps of a sequence may be greater than the gain in control of earlier steps (or the opposite if the result of the sequence is in itself undesirable but a goal nonetheless under some higher control system -- the proverbial schoolboy lagging on the way to school). This might result in something like a gradient. I have only subjective, anecdotal evidence for this.

A possible experiment would be to substitute some other sequence prior to eating. Set it up so that with each sequence-step completed the rat perceives an increment of progress toward eating. (Perhaps a table turns or a conveyer brings food and rat closer together.) Introduce disturbances to the steps individually (to one step in one run of the experiment, to another step in the next run, and so on). The prediction of this second hypothesis is that the rat will overcome greater resistance to perform the last step before eating than before earlier steps, indicating that it controls sequence-steps (whatever they may be) with higher gain the closer they are to the end of the sequence (assuming that the culmination of the sequence--and maybe the sequence itself
--is controlled with high gain.

         /Bruce

···

At 09:55 AM 5/24/2002 -0500, Richard Marken wrote:

[From Bill Powers (2002.05.25.0823 MDT)]

Bruce Nevin (2002.05.24 16:01 EDT)--

>The obvious thing to do is to show that existing mechanisms posited in PCT

produce that gradient. If that can be shown, then it follows that the
hypothesis of an output function mechanism is unnecessary, redundant, and
without evolutionary motivation.

That depends on how much structure has to be imagined to account for the
gradient. There comes a point when you have to say that the imagined
structure has become less believable than the simpler ad-hoc alternative.
For me, the threshold is rather low, apparently lower than it is for some
others (not just you).

Anyway, what I have proposed is not an output function mechanism, but a
superordinate system capable of changing the gain of the output function as
the error changes. We already know that gain-changing mechanisms are needed
to account for some phenomena (like the "reversal" experiment, where the
gain changes from a positive number to a negative one). Also, it's easy to
track with low gain instead of high gain, and the parameter-matching method
shows that the gain is low. So control via changing the gain is not a
problem. What is conjectural is the idea that the system controlling
the gain does so on the basis of the size of the error signal, and that
the gain change occurs in the output function.

Anyway anyway, after a long and interesting discussion with Mary this
morning, it suddenly occurred to me how the gain in the Dollard and Miller
experiment might increase with decreasing distance to the food dish,
without postulating any sort of change in the output function. I had
forgotten that "the gain" means "the LOOP gain", meaning the product of all
gains encountered in one trip around the control loop. If the feedback
function which makes the input quantity depend on the output quantity is
nonlinear, we will end up with an apparent nonlinear error curve.

Let's say that the effect that is observed is involved with controlling the
retinal area subtended by the food dish (or any objects in the vicinity
where the food is). As the rat moves toward the food dish, the retinal area
increases -- _as the inverse square of the distance_. If the distance is
halved, the retinal area increases fourfold.

The _slope_ of this relationship ( the change in area per unit change in
distance) then increases linearly as the distance decreases. That slope is
part of the loop gain, so as it increases the loop gain increases. And as
the loop gain increases, the rat moves closer to the goal or exerts
increased efforts toward the goal. And that increases the loop gain even
more. At least that's what I imagine now without setting up a simulation.

>There is a problem referring to this as a gradient based on the Dollard &

Miller experiment. The gradient they report is already present in the
experimental apparatus, which increases its resistance to the rat's
movement as the rat approaches the food. The observed gradient is actually
imposed, by means of the harness, upon the rat's movement. The rat is
controlling "walk over to food"; the harness impedes the movement, a
disturbance; the rat resists that disturbance. Because the disturbance is
gradient, the rat's efforts to resist the disturbance are gradient.

Ah, you got hold of the paper. I'd appreciate receiving a scan or a Xerox
of it. I had remembered that they simply stopped the rat at various
distances from the food and measured the pull on the tether. A great deal
depends on the details -- what you're reporting is that the force applied
to the rat changed with distance from the goal. Did they describe just what
the relationship was, or how the apparatus implemented it?

Best,

Bill P.

[From Rick Marken (2002.05.25.1015)]

Bill Powers (2002.05.25.0823 MDT)

Bruce Nevin (2002.05.24 16:01 EDT)--

>There is a problem referring to this as a gradient based on the Dollard &
>Miller experiment. The gradient they report is already present in the
>experimental apparatus, which increases its resistance to the rat's
>movement as the rat approaches the food. The observed gradient is actually
>imposed, by means of the harness, upon the rat's movement. The rat is
>controlling "walk over to food"; the harness impedes the movement, a
>disturbance; the rat resists that disturbance. Because the disturbance is
>gradient, the rat's efforts to resist the disturbance are gradient.

Ah, you got hold of the paper. I'd appreciate receiving a scan or a Xerox
of it. I had remembered that they simply stopped the rat at various
distances from the food and measured the pull on the tether. A great deal
depends on the details -- what you're reporting is that the force applied
to the rat changed with distance from the goal. Did they describe just what
the relationship was, or how the apparatus implemented it?

Your memory is correct Bill (as I hope is relatively clear from my description
of the study). The rats were stopped at two points (on different trials) and
the pull against the same force (spring constant) was measured each time. And,
yes, Brown describes the apparatus in great detail.

Best

Rick
-- -
Richard S. Marken
MindReadings.com
marken@mindreadings.com
310 474-0313

[From Bill Powers (2002.05.25.1136 MDT)]

Rick Marken (2002.05.25.1015)--

Thanks very much for finding the gradient paper. I see that I have chosen
to be worried about nothing. The differences in the force measures are 1./3
to 1/4 of a standard deviation, which means the article contains no facts
at all about the relation of force to distance: only what Dag likes to call
"mush."

I've had this problem too often before: accepting well-known "facts" and
stewing over how to model them, only to discover that they are just
statistical garbage. I would save myself a lot of trouble by looking up the
data first, before believing what I hear. As far as I'm concerned, there is
no change in how hard rats pull against a tether at various distances from
the food, so I can erase that "observation" from my reality.

It's much easier to understand efforts that are _constant_ with increasing
error.
Neurons carrying error signals can fire only so fast.

Thanks again, Rick.

Best,

Bill P.

[From Bruce Nevin (2002.05.25 19:05 EDT)]

Bill Powers (2002.05.25.0823 MDT)–

Ah, you got hold of the paper.

No. Sorry to have misled you. I should have emphasized that what I meant
by ‘the description’ (quoted below) was only [my understanding of] the
description one or another of you had made from memory in prior CSG-List
posts. The rest (quoted in part below) was a questioning of what did not
seem reasonable. But in any case the point is now moot. Or, rather, the
point that is not moot, and indeed you have emphasized it strongly, is
“data first, then model”.

Bruce Nevin (2002.05.24 16:01 EDT)-

I assume the resistance of the harness
was not the same throughout the rat’s

progress, first because that is not how I
understood the description, and

second because, if the rat was speeding up against uniform
resistance, I

would expect it to speed up without any disturbance, i.e. normal
locomotion

with no harness attached. After all, there is a certain amount of
resistance

in the physics of walking from A to B – if there were not, there
would be

no need to expend energy to do it.

    /Bruce
···

At 08:53 AM 5/25/2002 -0600, Bill Powers wrote: