[From Bill Powers (920522.1330)]
Chris Love (920522) --
I have passed the weighted percepts and references both through a
sigmoid (+- 0.5 range) and the error signal through another sigmoid >(+-
1.0). This gives me a dynamic range for the reference output of >about +-
0.5. Do you feel that it would be wiser to subtract the >weighted percept
sum from the weighted reference sum directly without >using sigmoids
directly on these summed inputs?
Did you say what you mean? It sounds as if the error signal and the percept
signal are inputs, with the reference signal as output. If so, you'd better
draw a diagram to show the organization you're using.
The sigmoid is an interesting idea -- it will put a realistic nonlinearity
into the perceptual signal. If the reference signal were derived from
previous recordings of the perceptual signal, it too would automatically
have the required sigmoid nonlinearity it in; when you generate it
directly, that isn't necessary. The loop gain will simply fall off for
large error signals, which may well help stability. The controlled variable
will be the inverse sigmoid function of the reference signal, when error is
zero. Go ahead and try the sigmoid; I'll be interest to see what happens.
You might also try one in the output function -- I predict that it will
have remarkably little effect.
So does FNI and FNO use some sort of nonlinearity, i.e., sigmoids?
No, it's a dynamic filter of a very simple kind. Suppose you have a
function that generates, basically, y = k*x, a simple amplifier. If you
want to put dynamic slowing into it without changing the steady-state
proportionality of output to input, you can do it this way:
y := y + (k*x - y)/S (a program step, not an equation)
The parenthesized part is the difference between the current value of y and
the target value which is k*y. A fraction 1/S of this difference is added
to the old value of y to produce the next value of y for the next
iteration. The result will be that on successive iterations, the value of y
will approach k*x asymptotically, approximating a curve of the form
1 - exp(-kt). This is like putting in a single-pole exponential filter,
which leaves the steady-state amplitude of the output alone, but attenuates
an alternating frequency by an amount proportional to frequency.
The optimum value of S, which makes the output reach a steady state in one
iteration, is 1 + G, where G is the loop gain of the closed-loop system in
which this step is the only filtering step. If you want control systems
with a loop gain greater than 1 (negative), you have to use this slowing
factor to get a stable system, assuming there are no other lags in the
loop. It's a quick and cheap way to stabilize models. See my 1979 article
in Psych Rev., "A quantitative analysis of purposive systems," for more
details.
Actually, you don't really need the true transport lag to get a reasonably
good model. We hardly ever use one in models of tracking behavior, because
they add so little to the predictivity that it isn't worth the bother. I
didn't use any in the Little Man. But you will need the slowing factor if
you don't have an element somewhere in the loop that limits the speed of
change of the variables. Otherwise you'll find that the system is unstable
for loop gains greater than 1, which wouldn't give you much control.
A slowing factor in the input function (in addition to one in the output
function) can add some realism to a model of tracking behavior, by creating
a slight overshoot response to a step disturbance. The "standard" human
transfer function used by engineering psychologists uses two lags, one with
a longer time constant than the other, as the best model. I didn't use this
in the Little Man, either.
The most important thing is to get a model running and play with the
parameters to see their effects. That's how I taught myself real control
theory. Books don't give much help to the intuition.
A learning algorithm we have used successfully (Tom Bourbon has used it,
and owes us a description of it) is based on the E. coli mode of steering.
Suppose you want the system to find its own best output gain (or
integration) factor. The criterion for "best" would be some measure of
changes in absolute error. Define a delta that can be randomly selected
from a range from -1 to 1. This delta, multiplied by a very small number,
is added to the current value of the output gain on each iteration. The
adaptation algorithm simply chooses a new delta at intervals that are
inversely proportional to the rate of change of average absolute error
signal (plus a constant to avoid infinite intervals at zero error). If the
rate of change of average error is positive (getting worse), a new delta is
chosen after a short interval. As the rate of change of average error gets
smaller and goes negative, new random deltas are chosen at longer and
longer intervals. The result is that changes in a bad direction are quickly
followed by a random change of direction, while changes in the right
direction are allowed to continue in that direction for a longer time
before a random change. The end point will be a series of random changes
near the optimum value of the parameter. Or you could decide that when
error is below some threshold, the deltas cease to be added. This is a
surprisingly efficient process, and it can overcome reasonable local-
minimum problems, given time.
I want to try this method for stabilizing the entire Little Man model. I
think the key to success will be to find, for each parameter that needs
adjustment, some measure of performance that is somewhat independent of the
measures used for adjusting the other parameters. This is like a concept of
a distributed reorganizing system that monitors many different intrinsic
variables and affects local parts of the system accordingly (and
appropriately).
By the way Bill, I really can see the usefulness of your "put up your
feet and teach the baby" routine that uses the grid system.
I'm getting old, Chris. Did I say that? Are you sure it wasn't someone
else? How about a little fuller description for my failing memory?