# top-level

**URL:** <http://discourse.iapct.org/t/top-level/15268>\
**Category:** CSG1992\
**Created:** [August 18, 1992, 10:02pm UTC](http://discourse.iapct.org/t/top-level/15268 "1992-08-18T22:02:30Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jeffrey\_C\_Hunter](http://discourse.iapct.org/letter_avatar_proxy/v4/letter/j/e47c2d/32.png) [@Jeffrey\_C\_Hunter](http://discourse.iapct.org/u/Jeffrey_C_Hunter)\
**Post date:** [August 18, 1992, 10:02pm UTC](http://discourse.iapct.org/t/top-level/15268/1 "1992-08-18T22:02:30Z")

</div>

Here's a new topic, related to re-organisation.

&nbsp;&nbsp;I started wondering recently how a top-level Elementary  
Control System (ECS) can remain connected to reality.

&nbsp;&nbsp;To explain: let us take a high-level ECS in a Control Net.  
(A high-level node is many levels from the raw input and output.)  
Assume the net starts out untrained (or only partially trained)  
for its environment. Finally we assume that random re-organization  
is a major part of its training.  
&nbsp;&nbsp;Now the high-level ECS doesn't know what its inputs or  
its reference mean. All it must do is control that they match.  
It may be initially set up with input of (target-position -  
finger-position) and reference of (0). However after a few random  
re-organizations the input weight for "target-position" may have been  
set to zero, and the input weight from "elbow-angle" to a positive value.  
&nbsp;&nbsp;This leaves the ECS training to control "elbow-angle +  
finger-position" = 0. There is no way for the ECS (or for the random  
re-organisation) to know that this new function is nonsense.

&nbsp;&nbsp;In general it seems impossible to keep the input "relevant"  
to the reference without forcing it in some fashion (and thus adding  
another set of properties to the ECS).

&nbsp;&nbsp;One approach to "forcing it" is found in our Little Baby (a  
learning version of the Little Man). As in the Little Man the high-level  
references involve the distance of the finger from the target  
(as perceived in the right and left retinas).  
&nbsp;&nbsp;The Baby has one (or more) layers of ECSs attached to the outputs  
of its high-level ECSs. However the inputs are connected directly to  
the Baby's inputs.

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;R\* \<- top-level reference

> **···**
>
> > 
> 
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;--------  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> ECSs | \<- top-level ECS  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;--------  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;/ \  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> --------  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;direct -\>| | ECSs | \<-untrained ECSs  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;input | --------  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;to top | / \  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> / \  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;...................................  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\ / \  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Input Output \<- environment
> 
> &nbsp;&nbsp;The Little Baby is forced to learn to follow the target by  
> being provided with a fixed input function.  
> The Complex Environmental Variable (CEV) that the Baby is controlling  
> cannot be unlearned, however it can likewise never be learned.  
> This is a reasonable hack while we experiment  
> with re-organisation, but in the long run we can't always  
> hand-code/hard-code the inputs.
> 
> &nbsp;&nbsp;Bill seems to have also seen the problem since he has suggested  
> that the learning mechanism should not be completely blind. He wants  
> it to contain some simple CEVs (an oxymoron :?) which guide the  
> re-organisation. This may be necessary, but it also feels like a hack  
> to have a separate control hierarchy for learning.
> 
> &nbsp;&nbsp;I have a partial solution that does not add new variables or  
> structure to the existing hierarchy. Unfortunately the CEV in the  
> example is different than "finger on target".
> 
> &nbsp;&nbsp;Suppose we have Little Baby (Mark MCXLI) that can successfully  
> learn to control (i.e. we have solved some of the re-organisation  
> problem).  
> &nbsp;&nbsp;Now we wish to teach it to avoid a spot in its environment (say  
> the exact center of its cube).
> 
> &nbsp;&nbsp;We add an extra input (called Pain). We change the environment  
> so that Pain becomes large if the finger is close to the center of  
> the cube, but is very small elsewhere. (We now have a hot-spot.)
> 
> &nbsp;&nbsp;We also add a simple ECS that has Pain as input, zero as  
> reference, a large gain, and outputs to the arm muscles. This does  
> nothing while the finger is outside the hot-spot. If the Baby moves  
> the finger into the hot-spot this ECS will quickly yank it out, and will  
> then resume doing nothing. We have given Little Baby a pain reflex.
> 
> &nbsp;&nbsp;The Baby now avoids the hotspot very effectively, however  
> it will have trouble moving finger to target in some cases (assume  
> for the moment that we don't move the target into the hotspot).  
> If a trajectory goes through the hotspot the arm will jump. Some  
> target locations will even have the Baby caught in a cycle.
> 
> &nbsp;&nbsp;The rest of the Baby will presumably eventually re-organise  
> to avoid approaching the hot-spot. There are several strategies that  
> will succeed, and the one chosen depends on the learning mechanism.
> 
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;R\* \<- top-level reference  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\>  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;--------  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> ECSs | \<- multi-level CS (Control System)  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;--------  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;/ \  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;/ \ R\* \<- another top-level reference  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;/ \ |  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;-------- -------- -------  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> ECSs | | ECSs | | ECS | \<- pain reflex ECS  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;-------- -------- -------  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;/ \ / \ / \  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;.......................................  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> \ | \ Pain /  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> \ | \ /  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;+-----------+ \ / Environment  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\> \ \ /  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Inputs ------------Outputs
> 
> &nbsp;&nbsp;So why don't I consider this a cheat too? After all we have  
> hand-coded an ECS to perform a function. Well we haven't had to  
> add a separate learning hierarchy (as per Bill), or had to wire across  
> levels (as in the current Little Baby).
> 
> &nbsp;&nbsp;Below are the reasons I think we don't have to add any  
> new features to "force" the Baby to learn the task.
> 
> Simplicity:  
> &nbsp;&nbsp;The pain-reflex is easy to learn by simple means (such as  
> genetic algorithms or random search). We shouldn't need to hand-code  
> such control functions.
> 
> Effectiveness:  
> &nbsp;&nbsp;The pain-reflex is very effective at avoiding the hot-spot.  
> This is accomplished solely by setting the gain high on a simple task.
> 
> Stability:  
> &nbsp;&nbsp;The pain reflex is stable against random re-organisation.  
> Since it is "effective" it very seldom has a non-zero error.  
> (Persistant high local error should probably trigger re-organisation.)  
> Since it is "simple" it has very few weights. This makes it a small  
> target for a random mutation (compared to the rest of the net).  
> &nbsp;&nbsp;Lastly it is high gain. If there is a random change to an input  
> or output the Baby will thrash wildly. The strong accumulation of  
> local error should quickly cause a benign mutation.
> 
> &nbsp;&nbsp;Since the new top-level goal is quite stable the rest of the  
> Little Baby's brain is forced to re-learn.
> 
> &nbsp;&nbsp;Now for the proverb. I have realized that one of my original  
> assumptions was wrong. When I first learned PCT I assumed that  
> all the top-level ECSs (ones with fixed references) were also  
> high-level ECSs (far from the environment).  
> &nbsp;&nbsp;I now suspect that \*most\* of the top-level goals of an  
> organism are fairly close to the I/O level, and that most of the  
> high-level ECSs are just used to add efficiency to the satisfaction of  
> these low-level goals.
> 
> &nbsp;&nbsp;Top-level goals need not be high-level goals.
> 
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;... Jeff  
> --  
> De apibus semper dubitandum est - Winni Ille Pu