[From Bill Powers (990617.1135 MDT)]
This is a copy of a post to Isaac (Marc) Kurtzer, for his use in a summer
project connected with his degree program at Brandeis.
Hi, Isaac --
Here are some writings about arm movement control. I'm copying them to
CSGnet for those who are interested. In the following discussion, you will
need only algebra to understand the developments. Differential equations
will only be mentioned when relevant, without requiring understanding of
that subject.
Let's consider three main topics: arm dynamics, a muscle model, and the
spinal reflexes.
I. Arm Dynamics.
For a control system model we need to have a test bed in the form of a
model of a physical arm. Richard Kennaway is currently working on a
four-degree-of- freedom model, but a 3-df model is sufficient to get
started. The three degrees of freedom considered here are
1. Yaw at the shoulder (side-to-side swing of the upper arm)
2. Pitch at the shoulder (up-and-down swing of the upper arm)
3. Pitch at the elbow (flexion-extension at the elbow with the forearm and
upper arm remaining in a vertical plane)
The fourth degree of freedom would be
4. Roll about the long axis of the upper arm.
The Little Man Version 2 contains a 3-df dynamic model of the arm. One of
its modules receives three arguments which represent the torque applied to
the arm at each joint. Given these three torques, the angular accelerations
in each d.f. are calculated; these are then integrated once to produce
angular velocity about each joint and again to produce each joint angle. In
the simulation, this model is updated every 0.01 sec. If your engineer
consultant wants to see this model I'll post it to you, or you can download
it off my Web page.
Given the papers you've sent me, I suspect that your engineer will be using
the _inverse_ dynamics model, in which you specify the desired angular
accelerations, velocities, and position, and calculate the torques needed
to achieve them. The Little Man uses only the _forward_ equations, the
torqu,es being generated through feedback processes. Actually, to run a
working model of the arm, even an inverse-dynamics model needs the same
forward-dynamics model to provide a model physical arm for the
control-system model to operate. It may seem redundant to actually model
the physical arm, since presumably the inverse dynamics supposedly being
calculated by the brain are being calculated perfectly, but a proper test
requires actually doing all the computations. I think there would be some
surprises if this were actually attempted.
I'm not enough of a mathematician to add the effect of the spinning room to
this model, although it could easily be done, according to Richard K.
II. The Muscle Model
I've somewhat simplified McMahon's model; adding the omitted details would
make only a small difference in how the arm control model works. One major
omission from the equations is any first-derivative damping within the
muscle (muscle viscosity). This is actually in the Little Man model, as a
slowed response of muscle tension to the driving signal, but it's added
where convenient rather than correctly. The reason we can get away with
this is that the damping introduced by the first-derivative muscle spindle
feedback is adjustable, and can be set to correct any errors in the muscle
viscosity approximation. Fixing this treatment up has been on my "to do
some day" list for a long time.
The muscle itself is represented as a contractile element in series with a
spring element (no dashpot, as noted above). The muscle spans a joint (in
the actual model, part of the biceps and triceps force spans _two_ joints,
for a bit of added realism). Consider first what happens when the arm
segments are clamped at a constant angle to each other. A shortening of the
contractile element will stretch the series spring element (since the ends
of the muscle can't move), with the result of generating a force. The
magnitude of the force depends on the amount of stretch and the spring
constant. A very stiff spring would generate a large force for a small
amount of stretch. In a real muscle, the maximum amount of force (according
to McMahon) is produced by about a 2% stretch of the muscle (you will,
however, want to double-check that figure). In other words, the series
spring is very stiff.
If we maintain a constant length of the contractile element, the force can
also be varied by moving the ends of the muscle apart or together. That
also changes the amount of stretch of the series spring element. So we can
write an equation for the generated force as follows:
(1) T = Ks(x + Ko*e), where
T = force pulling ends of muscle together
Ks = spring constant of series elastic element of muscle
x = length of muscle relative to resting length
e = signal driving muscle contractile element
Ko = amount of shortening per unit driving signal
(subscripts according to usage in Little Man model)
The symbol T is actually used in the model to represent torque, not lineal
force. The lineal force is applied across a joint on a line passing some
distance away from the center of rotation of the joint; the lineal force is
applied at right angles to a moment arm, creating a torque. The conversion
from lineal force to torque is not perfectly proportional because the
moment arm changes somewhat as the joint angle changes, but we assume a
proportional (linear) relationship in the model. The conversion to torque
involves a rather large mechanical disadvantage; at the elbow it is about
8:1 -- an 8-pound lineal force generated by biceps muscle across the elbow
joint produces only about a 1-pound force applied by the hand against an
object. So a 100-pound pull with the hand against something (as in lifting
a weight) entails about 800 pounds (about 360 Kg) of linear force on the
tendon attached to the forearm.
Equation (1) as written actually represents two opposing muscles, one of
which is made to contract as the other is made to relax. A positive signal
e has a negative effect on T on one side, and the same signal has a
positive effect on -T on the other side. If the full equations for each
side are written out and the forces are added together, all common terms
such as the resting length of the muscle and the average value of two
signals +/- e cancel out and we end up with Equation (1).
We could plug in the actual square-law signal-to-force function for a
muscle on each side, and assume a constant non-zero component of the signal
e on each side that maintains muscle tone. We would then find that in the
region where both muscles are active, the force-displacement relationship
is linear. Changing muscle tone can be used as a way of adjusting the
effective series spring constant for the muscle. In the Little Man we
assume a constant muscle tone, and thus a constant value of Ks (which of
course we can adjust). So our use of a linear muscle model is fully
justified, over the range mentioned.
This treatment in terms of balanced pairs of muscles extends also to the
effects of tendon and spindle feedback. There are internuncials that
convert positive signals from one side into negative effects on the other
side, for both tendon and stretch reflexes.
One complication omitted from this model is that these "pairs" of
"opposing" muscles are actually whole sets of muscles that pull in
different directions around the clock. The direction of movement of a limb
about a ball-and-socket joint can thus be in any direction, depending on
where the signals from above enter the collection of motor neurons. In this
model we assume only two orthogonal sets of muscles, working in the X and Y
directions. The results will be the same, within reason, as the more
realistic model would create.
This is actually the muscle model employed by Bizzi, Kelso, and others who
see limb behavior as that of an open-loop "mass-spring" model. Given a
driving signal entering a set of muscles spanning a joint, the joint will
come to some equilibrium angle, and it will do so in a manner describable
by a second-order differential equation. If the damping effect of muscle
viscosity is sufficient. the change of angle will be smooth and will not
involve any overshoots when there is an abrupt change of driving signal
from one steady value to a different steady value. I can tell you right now
that there is not sufficient damping in a real muscle to account for such
smooth stable motions, but proponents of the mass-spring model necessarily
assume that there is sufficient damping since real arms do not oscillate to
a stop after sudden moves. In a full control-system model, the required
extra damping comes from the phasic component of the stretch reflex.
Let's pass on to the model of the reflexes.
III. The tendon reflex.
The tendon reflex is the shortest loop, which we will analyze first. The
tension produced by all spinal motor neurons sending signals to fibers in a
given muscle is summed in the attaching tendons. Golgi receptors in the
tendons report the sensed tension as a set of neural signals returning to
the motor neurons on the same side in the inhibitory sense. The signals
also cross over in the spinal cord via internuncials that change the
inhibitory effect to an excitatory effect for the opposing systems. We will
deal just with one side as if bidirectional action existed.
The amount of feedback inhibition varies continuously down to zero, with
the threshold signal representing less than 0.1 gram of total force in the
tendon (compared with a maximum of 360 kilograms or more). That lower-limit
number is from McMahon, experimentally determined. The old idea that the
tendon reflex serves only to prevent excessive tension is just a bad guess
which should now be discarded. Unfortunately, lots of people still believe
it and cite it as a fact. Ain't science wonderful?
Representing all motor neurons for a given muscle by a single one, and all
signals entering and leaving all those neurons on both sides by single
equivalent signals, we can write the equations approximating the tendon
reflex. Below are given first the verbal description, then the mathematical
description for each element of the closed loop. Note that we use the
programming convention of an asterisk (*) to represent multiplication. This
permits the use of symbols that contain more than one letter, such as Kt
(which does NOT mean K times t).
(2a) The perceptual signal pt representing muscle tension is a constant Kt
times the torque T being generated by a muscle contraction or a change in
joint angle:
pt = Kt*T
(2b) The torque T depends on both the joint angle, which affects x, and the
output from the motor neuron represented by e. The amount of effect of e on
the torque is determined by the output sensitivity factor Ko. The torque
comes from a spring constant Ks times the combined effect of muscle stretch
and muscle contraction.
T = Ks*(x + Ko*e) (from Eq. 1 above)
(2c) The output e of the motor neuron results from summing all excitatory
and inhibitory signals that reach the typical cell body. The effect of the
Golgi feedback signal, pt, is inhibitory; all other inputs to the motor
neuron are represented for the time being as a single excitatory signal ra.
We assume inhibition to be subtractive (rather than acting as a divisor or
a digital gate).
e = ra - pt
We can solve the equations describing the tendon reflex control loop for
any dependent variable by successive substitutions. The _independent_
variables are ra, the net signal entering the motor neuron "from above",
and x, the joint angle established by external forces both direct and
inertial.
Solving this system of equations for T, the torque, we start with equation 2b:
(2b) T = Ks*(x + Ko*e)
The only dependent variable on the right is e, so we substitute the value
of e from (2c):
T = Ks*[x + Ko*(ra - pt)]
Now the only dependent variable on the right is pt, for which we can
substitute the right side of (2a):
T = Ks*[x + Ko*(ra - Kt*T)]
There are now no dependent variables left except T, the one we are solving
for. Note that this variable exists on both sides of the equal sign; this
is typical when solving closed-loop equations. To put this equation in
standard form we must transform it until T appears only on the left. First
we expand by multiplying out all parentheses:
T = Ks*x + Ks*Ko*ra - Ks*Ko*Kt*T
Then we subtract from both sides all terms on the right containing T (that
is, the only term containing T):
T + Ks*Ko*Kt*T = Ks*x + Ks*ko*ra
Extract the common factor on the left side:
T*(1 + Ks*Ko*Kt) = Ks*x + Ks*ko*ra
Divide both sides by the expression in parentheses on the left:
Ks*x + Ks*ko*ra
(3) T = ---------------------
1 + Ks*Ko*Kt
Now we have expressed the torque as a function of system parameters (Ks,
Ko, and Kt) and the independent variables (x and ra). Mechanically
displacing the arm will change x, and changing the signal from above
entering the motor neuron will change ra.
This same strategy of successive substitution can be used to solve for any
of the other independent variables.
Let's look at the meaning of equation (3). First we back up a step,
unsimplifying the equation by separating it into two terms, each divided by
(1+Ks*Ko*K):
Ks*x Ks*ko*ra
T = ------------ + ------------
1 + Ks*Ko*Kt 1 + Ks*Ko*Kt
The torque T is completely determined by the sum of these two terms. Each
term contributes part of the torque. Let's analyze the terms one at a time.
In the first term on the right, the only variable is x, the muscle length,
so this term is telling us how the torque is affected by the muscle length,
which we are treating here as equivalent to the joint angle. In the
numerator we have Ks, which is the spring constant of the elastic part of
the muscle, times x, the muscle length relative to the resting length. That
product is the amount of force that a stretch of the muscle by an amount x
would produce. This number is divided by (1 + Ks*Ko*Kt), where Ks*Ko*Kt is
known as the loop gain. If the loop gain is a very large number (and we
have reason to say that it is on the order of 50 to 200), the effect of the
change in muscle length on the total torque T will be reduced by that
factor. In other words, if the joint angle or muscle length is arbitrarily
changed by an external agency, the torque will scarcely change, so there
will be little resistance to the change (remember that right now we are
only considering the tendon reflex loop, not the effects of any higher
systems).
Obviously, for this to happen something else has to change when the joint
angle or muscle length is changed by an external agency. What changes is
the driving signal coming out of the motor neuron. It is the feedback link
that produces this change; if you want to verify this effect, solve the set
of equations for e. A change of x that would seem to result in increasing
the torque causes the inhibitory tendon feedback signal to increase, which
reduces the output of the motor neuron and allows the muscle to lengthen,
thus preventing most of the stretch of the elastic element that would
create an increase in the torque.
The tendon feedback is making this system very compliant to external forces.
In the second term on the right, the only variable is ra, the net signal
entering the motor neuron. The signal ra, when it becomes nonzero, produces
another contribution to the total torque. To estimate this contribution, it
helps to realize that if Ks*Ko*Kt is a large number, like 50 or 200, adding
1 to it will hardly change its value. Thus we will not be far off if we say
that
Ks*Ko*ra/(1 + Ks*Ko*Kt) is the same as
Ks*Ko*ra/(Ks*Ko*Kt).
The reason for making this handy approximation now becomes obvious, because
we can now cancel factors, leaving
ra/Kt
as the second term on the right.
To see what this means, suppose that there is no external agency producing
a change in muscle length or joint angle. Then only the second term on the
right contributes to the torque, when the signal ra is nonzero.
T = ra/Kt
The torque perceptual signal is pt = Kt*T, and we already have T = ra/Kt,
so we can deduce that the Golgi tendon signal pt is, very nearly,
pt = ra.
In other words, when there is no external force acting to change x, the
torque signal, and thus the torque, is being made nearly equal to the
signal ra, which is a reference signal. This control system controls the
sensed tendon force, which is directly proportional to the torque applied
at the joint.
And since we have already seen that the other term, the contribution to
torque due to arbitrary changes in muscle length or joint angle, is made
very small when the loop gain is high, we can conclude, finally, what the
tendon reflex control system does:
The tendon reflex control system makes the torque applied across a joint
proportional to the net excitatory input to the motor neuron (the net
reference signal), and also makes it independent of joint angle or muscle
length.
In a linear system, the torque is proportional to the angular acceleration
of the arm segments about the joint. Thus setting the signal ra to some
value is the same as determining the angular acceleration at the joint when
the limb segments are free to move. These features of the tendon control
system arise directly from the properties of the negative feedback loop.
I do not believe that anyone else has recognized this function of the
tendon reflex before.
A note concerning differential equations:
The above equations represent the steady-state solutions of a set of
differential equations. Thus they do not describe how the system gets from
one steady state to another. The equations assume that there _is_ a steady
state, meaning that the control systems are stable, but more advanced
methods (or simulations) are need to show what is required to guarantee
stability. This same consideration applies as we go on to expand the model.
IV: The static and phasic stretch reflexes
Paralleling the muscle fibers and embedded in the muscle are sensory organs
called muscle spindles. As the main muscle changes in length, the muscle
spindles also necessarily change in length.
Within each muscle spindle are two small muscles, one at each end or pole.
Wrapped around the equatorial part of the spindle is a spiral sensory organ
that detects a stretching of the equatorial part. This stretching can be
caused in two ways: by a lengthening of the main muscle which lengthens the
whole spindle, and by a contraction of the small muscles at the poles of
the spindle, which lengthens the equatorial part where the sensor is. The
muscle spindle thus acts as a mechanical comparator: the reference signal
induces a stretch of the equatorial part of the spindle, and a shortening
of the main muscle induces a shortening of the equatorial part, so the
spindle's output signal represents the difference in the amount of these
two effects.
If the polar muscles (driven by "gamma efferent" signals) contract, the
equator will be stretched. There will be a signal generated by the spiral
sensory organ; this signal passes to the motor neuron in the excitatory
sense, causing the main muscle to shorten if the segments of the limb are
free to move. This shortening will result in moving the ends of the muscle
spindle closer together, and this shortening in turn will counteract the
stretch of the spindle's equatorial region caused by the gamma efferent
signal and the contraction at the poles of the spindle. So we have a
negative feedback loop in which the output function consists of the entire
tendon reflex loop, and the comparator is the muscle spindle. This control
system controls muscle length, and because muscle length is directly
related to joint angle, it controls the angle at the joint.
Now we are going to re-analyze the spinal loops, this time taking into
account the stretch reflex. The signal we had been calling "ra" now has to
be treated as two signals, one from the stretch reflex loop's mechanical
comparator and the other from higher systems (the alpha efferent signal).
We need to assign some new variables and constants. We will use x as before
for the angle at the joint, which is proportional to the length of the main
muscle. This is also the length of the spindle; we'll just absorb any
proportionality factors into other constants.
The stretch of the equatorial portion of a spindle is proportional to x
plus the amount of contraction of the poles of the spindle, which is
assumed numerically equal to the gamma efferent reference signal, rg. The
amount of signal coming out of the comparator represents the error in the
stretch loop, es. So we have
(5a) es = Kg*(rg + Kl*x)
The constant Kg adjusts the spiral sensor's sensitivity to length error,
determining how much error signal will be generated by an actual physical
length error. The constant Kl (that's a lower-case "el", not a "one")
adjusts the relative sensitivity of the mechanical comparator to changes in
main muscle length. The variable rg is the gamma reference signal. These
parameters allow for all the fitting adjustments that need to be made. For
matching at some later time to the detailed properties of the various
signals and mechanical elements, these parameters can be expanded into the
required more detailed parameters.
The error signal es connects to the motor neuron in the excitatory or
positive sense. This changes equation 2c to
(5b) e = ra + es - pt,
where now ra means "alpha efferent" reference signal -- the signal
descending the spinal cord to the motor neuron. The added signal, es, is
the feedback from the muscle-length (or joint-angle) control system's
mechanical comparator. The remaining term, pt, is as before the Golgi
tension feedback signal.
The tendon feedback signal pt is still
(5c) pt = Kt*T,
and the magnitude of torque T is still
(5d) T = Ks*(x + Ko*e) (from Eq. 1 above)
Equations (5a) through (5d) now constitute the system equations for the
combined tendon and static muscle length reflexes. This is as far as we can
go with algebra; the remaining effect, which is due to the rate of change
of muscle length, takes us into differential equations. Briefly, equation
(5a) must have a term added to it that represents the amount of effect due
to the rate of change of x. Because this effect determines the damping of
the system (which counteracts oscillations), the adjustable constant is
symbolized Kd, meaning the damping constant, and the complete equation is
(5a') es = Kg*(rg - Kl*x - Kd*dx/dt)
where dx/dt is the rate of change of muscle length, hence angle.
These four equations are the model of the spinal kinesthetic control
systems in the Little Man; there is one set of these equations for each
degree of freedom in the arm. If a fourth degree of freedom were added,
roll about the long axis of the upper arm, another set of these equations
would appear. The systems represented by these equations convert the
current joint angle and rate of change of joint angle into neural signals
that determine the torque produced by the muscles for each degree of
freedom. The physical model of the arm converts the torque at each joint
into the angular acceleration (d2x/dt^2), angular velocity (dx/dt), and
angle (x) for each joint, taking into account all interactions and dynamic
effects, thus closing the feedback loop.
V: Higher control systems
In the Little Man, the spinal reflex control systems are under the control
of another kinesthetic level of control (which acts by coactivating the two
reference signals rg and ra, the gamma and alpha efferent signals). This
probably represents a brain-stem level of control. It is conceptualized as
using both muscle length and joint angle information to control joint angle
per se. The reason it is needed is that the spinal systems do not appear to
be able to exert tight control of joint angle; one has to look at the full
differential equations to see why. Adjusting the parameters of the spinal
systems to get the best dynamic performance results in very weak static
(sustained) control of position. Since we do not observe weak control of
static position, we need the second level of kinesthetic control to get
realistic performance, both dynamic and static.
The writeup included with the Little Man program on my web page, and the
forthcoming IJHMS article, go into some added considerations concerning
this second level of kinesthetic control, primarily the way it can be used
to change the coordinate system into one that is easier to control from
higher levels still, such as visual systems.
In the "Artificial Cerebellum" version of the Little Man, referred to as
"Armac" on my web page, the adaptive scheme proposed as the way the
cerebellum works acts in parallel with the second kinesthetic level. For
this version of the Little Man, the parameters of the second level control
systems are set to provide very weak integral or leaky integral control,
with the Artificial Cerebellum providing most of the loop gain and all of
the dynamic stabilization.
All of this is VERY DIFFERENT from what mainstream (i.e., employed)
researchers seem to be proposing, according to the papers you sent me. I
predict that my model will remain a minority view until the current fads in
explaining motor control have run their course. It will then be realized
that my approach is the most direct and least fanciful -- and probably
correct -- one.
I can't help feeling that all these experiments with rotating rooms and
vibrators attached to muscles are premature. What really needs to be
understood first is how a person can reach out and touch something under
normal conditions. Once we have a working model that can accomplish that,
we can add refinements to take care of special circumstances.
But who listens to me?
Best,
Bill P.