[From Bill Powers (960118.0850 MST)]
Martin Taylor 960118 16:10 --
That's why the simulation is not anywhere close to behaving in the
way an analogue loop does. There are two independent trains of
behaviour chasing each other round the discrete simulation loop.
Do you really mean "independent?" If the even-numbered values were
independent of the odd-numbered values, you could arbitrarily change the
one without changing the other. Somehow I don't think this would work.
If you are going to simulate an analogue loop, you've got to have
filtering to ensure that the samples at any point are fast compared
to the changes in the waveform. Then your X(n-1) gets into the act
for X(n), but it doesn't do it by the effects on itself around the
loop.
Here comes the fire hose again.
The effect of a variable "on itself" is, of course, an effect over time.
In the algebraic treatment of a control system, the solutions reflect
the steady-state values to which the variables would come after
sufficient time has elapsed. The algebraic solutions are adequate if the
changes in reference signal and disturbance are slow enough that no
variable changes significantly during one trip of the signals and
physical effects around the loop. The definition of "significantly"
depends on the application. If you want an accuracy of 1% in your
algebraic prediction, an "insignificant" change is less than 1% of the
value of the variable. The algebraic solution remains valid within 1% as
long as the changes in disturbance and reference signal don't cause any
variable to change by 1% or more during one loop transit time.
This sort of quasi-static analysis is used all the time in the physical
sciences when accuracy to the limits of measurement is less important
than forming an understanding of what a system is doing, and how. It is
used, for example, in analyzing the principles of levers. If you press
down on one end of a lever with force F, the other end exerts a force
L2/L1*F on the load, and the reaction force equals the applied force. Of
course the output force doesn't actually reach that value at the same
time the input force is applied, nor does the reaction force from the
load equal the applied force right away: the initial force bends the
lever locally, and the bending wave travels at a finite speed to the
output end, and is then reflected back to the input end. The output
force and reaction force are delayed and both show a damped oscillation
around the final value. The instantaneous imbalances are made up by
inertial forces. However, the delay and oscillations are so small that
they have trivial effects on the results _at the level of detail at
which we are trying to understand the lever_.
Any real control loop is made up of multiple signals travelling in
redundant pathways (there are many muscle length detectors in the same
muscle, many tendon receptors in the same tendon). These signals are
generated by independent analog processes, so the impulses are not
generated synchronously. The total signal is much like an electric
current being generated by the movement of many electrons in an wire.
This signal is continuous with a temporal grain much finer than the
grain of any one signal. Furthermore, impulses keep appearing all the
time that the stimulus is present; there is no pause while a single
just-generated signal makes its way through intermediate circuits and
produces a twitch of a muscle fiber. While that signal is traveling, new
impulses are being generated in redundant pathways, and at high levels
of stimulation even in the same pathway. At a modest signal level of 100
impulses per second, even a single pathway with a loop delay of 0.16
seconds will be carrying 16 impulses at the same time, distributed along
the path. At a high signal level of 500 impulses per second, there will
be 80 impulses moving in that path -- in parallel with the
unsynchronized impulses moving in parallel redundant paths.
What this means is that the simplistic digital way of speaking of a
signal X[n] and X[n-1] is inapplicable to neural loops. Normal digital
logic is misleading. Consider a chain made up of three parts:
X ---> Y ---> Z --
If the transit time is 1 unit for each connection, the natural digital
representation would say that Z[t] = X[t-2]. If the loop were closed, we
would say that X[t] = X[t-3], and we would see a loop delay of 3 units.
But would it follow that X can change state only once every 3 time
units? No, it would not. There is, in fact, no limit on how many times X
can change state per unit time. Nothing was said here about a clock that
permits a change of state only once during each transit delay. If the
transit time is 1 second, X could be changing state 100 times per
second. Then Y and Z would also be changing state 100 times per second.
Any one value of a variable would depend on its value 3.00 time units
ago, but 1/100 of a second later, it would again depend on its (possibly
different) value 3.00 time units ago, with the time units being shifted
by 0.01 sec.
In a digital computer, the transit time between components is dictated
by the length of the connecting wires, and in most computers the wires
are kept short enough that at one nanosecond per 0.3 meters, that delay
is negligible. In a Cray computer (a few years ago) the wires that
crossed the main cylindrical chassis could have 4 changes of signal
state on them at the same time, but in integrated circuit CPU chips the
longest wires are only a fraction of a centimeter long, so the speed
limit is set by transition speeds inside the computing elements.
The nervous system is more like a computer in which the transistors are
spaced 100 feet apart or more, so any long signal line is likely to be
carrying far more than one impulse. Furthermore, the basic source of
signals is an analog conversion from a continuous physical variable into
a frequency-modulated signal, so the inter-impulse times can vary on a
continuous scale -- they can change by 1 millisec, or 1 microsec, or 1
picosec. And generally, a single physical stimulus like pressure on the
skin or the stretch of a tendon produces such signals continuously and
in many parallel pathways.
The result is that the behavior of the nervous system is far better
approximated as an analog process than as a digital process. Time-
delays are better represented as in a Laplace Transform -- as a
continuously-variable time shift of a continuous variable. Signals are
not on-off signals (unless constructed to be such, as I outlined
yesterday), but continuously variable frequencies, or in reciprocal
form, continuously variable inter-impulse times. And the effective
signal frequencies are not those in a single fiber, but the sum of all
the frequencies in parallel redundant fibers.
To this we can add that in every behavioral process, there is one place
where discrete impulses are summed over parallel pathways and converted
into a truly continuous analog variable: in the muscles and the limbs
they operate. The time-constants of muscles and the mass of the limbs
make sure that no single impulse has any behavioral significance. A limb
cannot simply change from one position instantaneously to another
position. It is moved by applying an accelerating force from muscles; it
picks up speed to some final velocity, perhaps coasts briefly at that
velocity, and is decelerated to a stop in a new position many tens of
thousands of neural impulses later. All trace of the discrete nature of
neural impulses is erased in this output process. Therefore every
control loop contains an analog component, the environmental link from
output back to input.
···
-----------------------------------------------------------------------
Martin Taylor 960118 17:10 --
What you really need is this:
o := o + slow*(gain*i - o)
"Slow" has to be a number less than 1. If all other gain factors in the
loop are 1, the minimum value for the slowing factor is 1/(1 + gain).
That will correct the error in one iteration.
Yes, but as we have several times discussed, this is true IN A
DISCRETE SIMULATION ONLY. It bears no relation to what happens in
the analogue loop being simulated. There is no equivalent
stabilization possibility in an analogue loop, nor will it occur in
a discrete simulation if you decouple the recompute sample time and
the signal delay around the loop. Bill P. has confirmed this in his
own simulations.
Even in an analog loop there is _some_ slowing factor -- introduced as
an R-C time constant -- that will stabilize a loop with a single
transport lag and an arbitrarily large loop gain. You are right that the
"minimum slowing factor" mentioned above applies to the problem of
eliminating the artifact of computational oscillations in a digital
computer.
-----------------------------------------------------------------------
Bill Leach 960117.20:06 U.S. Eastern Time Zone
What I was trying (not so clearly) to state is that in biological
control systems, control is usually in one direction only. That is,
the a perception can be increased or decreased by a single control
loop but not both.
It seems that _essentially_ most identified control loops have a
"complementary" loop that does effectively control the same
perception when that perception changes in the opposite direction
(beyond the control limits of the first system).
...
Did I make any more sense this time?
You always make sense about control theory, Bill. It was just a minor
problem with the wording.
-----------------------------------------------------------------------
Hans Blom, 960116 --
I agree generally with your remarks about prediction. The main point I
was trying to make is that predictions are used only at the time they
are meant to predict: that is, if I predict that a heavy ball is going
to land in my hand in 10 seconds, I don't start exerting the extra
upward force as soon as I finish the prediction. The prediction is meant
to apply _at the time the ball is predicted to drop_. Ideally, I want to
issue a command to increase the perceived force from my hand just a
small fraction of a second before the ball lands, so the force will
actually start increasing at the same time the ball contacts my hand.
Dealing with past, present, and future in control systems gets very
confusing. The best policy, I think, is to speak always in the present
and to consider what is presently happening at each point in the control
loop. A prediction creates a representation of something that has not
yet happened -- an event or the value of a variable. This representation
exists right now, in present time. It is based on _current_ and _past_
inputs only. But it will be applied only at the time for which the
prediction was made, so we have to consider how the application of the
prediction is held off until the predicted event is just about to occur.
There are cases, of course, where the time at which the preparation is
made doesn't matter much, but we have to consider the general case.
In modeling the environment, we can, as you say, run the model faster
than real time. This lets us predict the state of a variable at a given
future time, but it also lets us predict the earliest time when a given
state will occur. The latter, I think, is the more generally useful. We
are interested in certain preferred states of the environment: the stock
market reaching a bottom, a ball reaching our hands, a cake reaching the
state of being done. What we need to know is _when_ this will occur, if
ever. On the basis of this prediction and some means of measuring time,
we can set a reference time at which we should do something, like buy a
stock, brace for impact, or take the cake out of the oven. This is all
accomplished in the present, not the future. We set the oven timer, and
then go about our business until it buzzes.
Remember that my controller was able to remain in control when no
observations were forthcoming (under the condition that the world
did not change in the mean time)? That is the equivalent thing.
Speed the prediction up and what you do is predicting the future.
Well, it remained in control (in the computer simulation) for about 10
iterations, or 1/6 second. It didn't really do much better than my
simple control system with an integrating output function. The best
control was obtained when you were predicting the disturbance 1
iteration ahead, and naturally the quality of the prediction fell off
sharply with time. You could have done a little better by using more
derivatives of the disturbance, but as you know the gain from adding
derivatives is very quickly cancelled by the amplification of noise that
is introduced by taking successive derivatives. I think that the
usefulness of model-based control is very situation-dependent, and that
for the lowest levels of control like spinal reflexes, the usefulness is
neglible in comparison with simple feedback control.
Why would a controller want this possibility? I think that one
reason is to be able to keep controlling some modalities while the
attention (the employment of the perceptual apparatus, e.g. the
direction in which one looks) is focussed on something else. In
other words: to realize parallel processing cq. parallel control.
I wish we could agree not to call this open-loop predictive compensation
"control," because true control does not require anticipating
disturbances or having a completely regular environment. Also, attention
and control are two different subjects. When you are walking along a
street, you are consciously attending to the visual picture of the
environment, but not to every possible way that information could be
interpreted. For example, you are maintaining your balance, long-term,
with the aid of visual information, but what you are consciously
attending to may be only what you see in the shop windows.
What other use of the prediction capability? For planning.
But planning is not controlling. After you have made a plan, you then
have to carry it out, and you can't predict what you will have to do to
carry it out. Plans are usually cast in the form of descriptions of
perceptions that are to be brought about: pack my clothes, go to the
train station, buy a ticket, and so on. But the exact movements you will
make and the exact places you will go in realizing these plans will
depend on the environments you encounter on the way. If the ticket
counter is closed you will have to buy your ticket on the train, and so
forth. Plans describe _outcomes_, not _actions_.
By greatly speeding up, beyond real-time, the process of prediction
we can compute (model) what will happen in the future given a
prespecified future course of action. By testing a number of
possible courses of action, a search for optimum action is
possible. (This is a "forward" way of finding optimum behavior
which avoids reverse models). All this again under the condition
that the world doesn't change appreciably in the mean time.
You're right. There's no reason we can't make a fast-time prediction
based on our current inputs and actions, compare the predicted result
with what we want, change our current actions, and do the prediction
again, iteratively, until the predicted result is the result we want.
This is a well-known method for instrument-aided landing of airplanes.
The computer looks at the current readings for altitude, speed, angle of
attack, rate of turn, rate of descent, ILS radio information, and so
forth, and presents a picture of the runway with a big X where the plane
will land if nothing changes. The pilot moves the controls so as to move
this X to the right place on the picture of the runway and keep it
there. In the meantime, the computer is repeating this prediction
several times per second. This works because as the plane approaches the
runway, the prediction is made for shorter and shorter times into the
future, so the error keeps getting less and less. Just before touchdown,
the error is essentially zero.
Thnis ils an interesting variant on the usual concept of prediction,
where you get just one chance and then have to wait with your fingers
crossed to see if you did it right.
Wouldn't it be nice to know the effect of your actions before you
attempt something? That is what an internal world-model makes
possible.
Sometimes it would, and I think we do use internal world models in those
cases. But to worry about the effects of my actions in advance would
just get in the way for things like typing, writing my name, or driving
a car. At the lower levels, things happen too fast and change too fast
to bother with predictions. It's only at the higher, cognitive, levels
where variables change slowly that we have time to make predictions
before events overtake us.
I should mention that many things that look like predictive behavior
probably don't involve literal predictions. The example I have in mind
is steering a collision course toward a moving object. A lot of people
(including Norbert Wiener) treated this as a problem of figuring out
where the object will be some time in the future and then aiming at that
point. For a ballistic projectile this may be necessary. But for a
steerable projectile, it's much simpler to aim ahead of the moving
target by some amount, and then simply maintain a constant bearing
between the direction of travel and the direction of the target. If you
can steer, the lead angle isn't critical. The result will look exactly
as though you had picked a collision point and aimed at it, but in fact
you simply maintain a specific relationship to the target bearing and
the collision will result.
-----------------------------------------------------------------------
John Anderson (960118.0745) --
Bruce Abbott has shown how a PCT approach to the Aplysia problem would
be set up. I can't improve on it.
-----------------------------------------------------------------------
Best to all,
Bill P