[From Bill Powers (970313.0400 MST)]
Now that we're starting to get somewhere with the MCT model, I've started
thinking about how it would look in a hierarchical form like that of the PCT
model. While the HPCT model is still just an approximation to the real
thing, leaving out interactions among different perceptual functions, the
basic principle of controlling simple scalar quantities seems promising as a
step toward the best model of a living system. I won't speak about
engineering models; maybe the one-complex-system approach will turn out to
surpass the way living systems control; maybe not.
In the light of our theodolite example and my experience with the Little Man
model, there seems to be a basic principle of hierarchical control: get the
highest derivative of the system (that matters) under control first.
Suppose we have a mass with damping, and a force is applied to it to bring
the velocity from 0 to v. The force required to maintain a specific velocity
is proportional to the damping, so v(final) = k*f. The length of time
required to reach the final velocity depends on the mass of the object being
moved, which we can take as fixed. The most direct way to achieve the new
velocity is to set the force to the required amount, and then wait for the
velocity to reach asymptote.
Of course the velocity could be increased much faster if the force could be
made many times what is needed for the desired velocity, but, as this
velocity was approached, the force could be adjusted downward until it
became just enough to sustain the new velocity. This could be done by
dynamic filtering of the signal producing the force, but a much simpler way
is to use negative feedback.
By using a velocity reference signal indicating the desired velocity, and a
velocity sensor indicating the actual velocity, the system can produce an
initially large error signal that produces a large force. This force could
be far larger, hundreds of times larger, than needed to produce the desired
velocity. The mass would accelerate rapidly, as if toward a very high velocity.
However, as the mass accelerates, its velocity would increase toward the
reference velocity and the error signal would decrease. The force would thus
also decrease. The result would be that just as the mass reached the desired
velocity, the force would have declined to just the value needed to sustain
that velocity. In fact, there would be an exponential increase in velocity
to the desired limit that would have a time constant equal to the open-loop
time constant divided by the amplification factor in the loop. If the
open-loop time constant were 1 second and the loop gain were 100, the time
constant with the control system acting would be 0.01 second.
The effect would be, relative to the signal that is specifying the new
velocity, just as if the mass of the object had been divided by the loop gain.
Now suppose it turns out that, as in a muscle, the applied force can't be
changed instantly. When a driving signal appears instantaneously, let's say
that the force rises toward a value proportional to the driving signal,
along a curve that to a first approximation can be represented by a
decelerating exponential. We now have a higher derivative to consider: the
rate of change of acceleration, or the third derivative of position. The
presence of this higher derivative will create problems for the velocity
control system; in fact, the maximum permissible loop gain of the velocity
control system will be lower, because there will be a tendency to
oscillation. The driving force can no longer be changed instantly. To allow
a higher gain in the velocity control system, we must somehow reduce the
time constant of the force response to the driving signal.
This can be done in exactly the same way. Instead of the driving signal
producing force directly, it becomes the reference signal of a control
system that can also sense the force being generated. If this system has a
high gain, the rise-time of the force in response to the driving signal can
be made much faster -- as much faster as the loop gain in this new control
system. If the open-loop time constant for force changes is 50 milliseconds,
and the loop gain is 100, the closed-loop time constant will be 0.5
milliseconds.
Without the force-control system, the velocity control system would have
some maximum loop gain that would permit stable operation. With this force
control system having a loop gain of 100, the velocity-control system could
have a gain 100 times larger and still remain stable.
The limit on this process is set by delays in the lowest control system, and
by noise in the sensor and comparator.
The spinal reflexes are organized in exactly this way. The lowest level of
control is the tendon reflex, which directly measures the force generated by
a muscle. The lag between a change in muscle force and a change in the
tendon signal is set physically by the speed of sound in muscle and tendon,
and by the refractory period of the sensors. Altogether it must be one or
two milliseconds. Most of the lag is in signal-transmission, and amounts to
perhaps 5 milliseconds. The muscle time-constant, measured open-loop, is
around 50 milliseconds for a contraction, so in principle the tendon reflex
could reduce it to 5 or 10 milliseconds. With many force-sensors acting in
parallel in the same loop, the delay could be considerably shorter than
that, because some sensors would always be at the end of their refractory
periods.
The second level of control involves the phasic part of the muscle spindle
response. Again, the physical lag time here is only a millisecond or two,
since the phasic component is created by a lag between the muscle stretch
and the response of the interior part of the spindle (embedded in the
muscle) to a stretch at its ends. The velocity control system can have a
relatively high gain, too, given the force control system.
The third level of control involves the tonic part of the stretch reflex,
which gives approximate muscle length control and thus joint angle control.
This control system experiences a limb with an apparent mass that is only a
fraction of the actual mass, and operates via muscles that have an apparent
response time that is far shorter than the open-loop response time. The
result is that a joint angle can be changed from one stable value to another
very different angle in only about 150 to 200 milliseconds. These speeds are
achieved by momentary muscle tensions close to the maximum that can be
produced without physical damage. The maximum speed is high enough that the
eye can't follow the movement; it's a blur.
It seems, therefore, that getting control of the highest significant
derivative is a basic principle of hierarchical control. Note that this
greatly simplifies the problem of stabilization, because once we have
reached the highest derivative, the associated control loop becomes a simple
first-order lag system, which is inherently stable. This is undoubtedly why,
in the Little Man Version 2, no special dynamic filtering was needed to
attain stability. The required filtering is inherent in the hierarchical
arrangement.
Finally, the MCT model. When we have reduced each control system to a simple
one-dimensional system with a first-order lag, what kind of "model of the
environment" would be involved? It would be either a simple constant of
proportionality (if the lag is in the output function as for the force
control system) or a single integration (if the lag is in the environment as
for the velocity control system). In either case, adjusting the system for
optimality involves changing just one parameter: the constant of
proportionality or the rate of integration. The required "world model" has
been reduced to the absolute minimum.
This means that adjusting any given control system for optimal performance
involves changing only one parameter, the output gain. If the output gain at
the lowest level, controlling the highest derivative, is set for stable
control, then each higher system's maximum permissible gain is determined,
and each higher system can also be optimized by adjusting its output gain.
It is impossible to do better than that.
In human systems, there does not appear to be any model of the environment
at the lower levels of control. When the inputs are lost at these levels,
performance simply collapses, and it recovers very slowly if at all. In
fact, recovery from injuries to the afferent paths seems to go at the rate
of nerve regeneration, which is both slow and limited in the adult. There is
no hint of control that uses an internal simulation of the external feedback
path.
At higher levels, it does appear that the imagination connection comes into
play, in cognitive planning most obviously, but perhaps also at other levels
in the higher ranges. However, the HPCT model still treats each control
system at the higher levels as a one-dimensional scalar system, so for any
one system, the "internal simulation" is still just a constant of
proportionality or a simple integration, with the output function being of
the complementary type. Complex behavior involves many of these elementary
control systems operating at once, so the overall effect can be that of a
complex simulation of the environment -- when many systems are operating in
the imagination mode at the same time.
The most obvious difference between the MCT and PCT models is the normal
mode of operation. In the PCT model, the normal mode is the one in which the
perceptual signal arises directly from lower-level perceptions or sensors.
In the MCT model, the normal mode is the one in which the perceptual signal
is derived from the internal simulation. In fact, the MCT model never uses
the real-time information from lower systems or sensors _for control_. That
information is used only to optimize the internal simulation. This means
that the ONLY way the MCT model can deal with disturbances is by calculating
them and using the calculated disturbance in the simulation. With the
hierarchical version, this may not be such a disadvantage, since the
disturbance, too, is a simple scalar quantity. However, the PCT model, which
needs no internal representation of the disturbance, can operate without
this calculation, and so is simpler. Or to put it differently: it is always
better, when possible, to control the real-time perception than to control
the simulated one. When that is not possible, there are advantages to being
able to control a simulated perception.
This discussion represents, of course, only the beginnings of trying to
develop a hierarchical model. When as much work by as many people as have
been involved in other approaches has been done, we will know much more
about the feasibility of these suggestions.
Best,
Bill P.