Littlebaby...

[From: Chris Love (920523.1900)]
[To: Bill Powers (920525.1000)]

Earlier (920522) I wrote to Bill saying,

This gives me a dynamic range for the reference output of about +-0.5.

  This caused some confusion since Bill replied with,

Did you say what you mean? It sounds as if the error signal and the percept

>signal are inputs, with the reference signal as output.

  Sorry Bill. The description was somewhat misleading now that I have
reread it. It is sometimes *tricky* to describe the output signals/reference
signals since they are they same except for the weighting factors
inbetween the various layers. That is to say, a output signal from a higher
elementary control module (ECM) as it propagates itself to lower modules
becomes weighted and then *becomes* a reference input to a lower ECM.

  In my previous description I mistakenly called an ECM output a reference
signal. This was wrong (in the context) so I think this is where the confusion
crept into the discussion.

  To be quite clear, percepts are inputs. Reference signals are inputs. The
difference between these two, called the error signal, is the output (aside
from filtering, amplification, transport lag,etc.).

I like your adaptation scheme Bill. I have worked through it and have
rewritten it in a pseudo algorithm form and have one suggestion to point
(5):

.........Suppose you want the system to find its own best output gain (or
integration) factor. The criterion for "best" would be some measure of
changes in absolute error.

..........(1) Define a delta that can be randomly selected from a range from -
1 to 1.
..........(2) Multiply this "delta" by a very small number and add it to the
..........current value of the output gain on each iteration.
..........(3) The adaptation algorithm simply chooses a new delta at intervals
..........that are inversely proportional to the rate of change of average
..........ABSOLUTE error signal (plus a constant to avoid infinite intervals at
..........zero error), i.e., If the rate of change of average error is positive
..........(getting worse), (Fig. 1 (a)) choose a new delta (after a short
..........interval).
..........(5) As the rate of change of average error gets smaller and goes
..........negative , *new* random deltas are chosen at longer and longer
..........intervals.

...........................^ Error
.....................*.....|
..........(a)....... *.....|
....................*......|
...................*.......|...............*
..................*........|.............*..*
.................*.........|.......... *.....*
................*..........|.........*........*
..............*............|.......*..........*
(Relative increase in (Relative decrease in Error is
  Error is increasing) increasing here)
--------------------------TIME---------------------->
Fig. 1 Error-time plots. (a) Error graph shows error increasing with time with
a larger incremental rate,i.e., rate of error is accelaerating. (b) Error
graph shows error is increasing at first then changing sign and decreasing
at a larger incremental rate, i.e., rate of error is decelerating.

  My suggestion is that if the 'rate of change' of error is decreasing over time
using a particular random number then this random value is good and you
may want to do one of two things:
  (i) Leave it (since it is working correctly).
  (ii) Enhance its value.

  First, if you choose to follow step (5), choosing the random deltas at longer
intervals makes sense, BUT the basic essence of my following suggestion
is - if it's working don't touch it! So here it is....

...changes in the right direction are allowed to continue in that direction for
   a *longer time* before a random change.

  Why do you *have* to choose a *new* delta if the error slope is negative,
i.e. the error is decreasing, especially if it is doing so more quickly at each
interval; the rate of change of the *decrease* in error is increasing??? (refer
to Fig. 1b) Unless you are worried about overshoot?

  I realize that step (3) does this somewhat using the inverse proportionally,
i.e, if the error slope is large (indicating a large changes) make the random
number selection less frequent and vice versa. But what I'm suggesting is
simply to try to *optimize* this characteristic.

This is a surprisingly efficient process..

  Ok. Maybe this is even faster. Is it important to be faster in adapting? I
suppose it depends on what your application is, right?

Anyways Bill - it's only a thought I had while reading your mail.

Last thing.
  You asked what I meant by,

"put up your feet and teach the baby" routine that uses the grid system.

Well I was referring to your,

7. A learning mode that can be used when map control is on, gradually

making the >kinesthetic space agree with the visual space. A random target
positioning mode is >available so you can turn learning on with a cleared
map, go away and read a >book for a couple of hours, and come back to
find the map all adapted (1331 cells).

  Well, have to go and implement some of these changes...
Talk to you guys later.
Chris.