Attention, Reorganization, and Awareness

CSGnet Archive CSG2008
1

Dear CSG Group,

This is Bo from Shanghai. I made an adapted "Crowd" simulation in
which individuals are equipped with "reorganization system" using
e-coli reorganization. The simulation applet could be found at:
http://sites.google.com/site/devevo/
This site is still under heavy construction. I put it out for I think
this would be a reference to people discussing the relationship
between reorganization and cognition in other threads recently.

Intrinsic attentional modulation of reorganization I added to the
simulation makes it more unique. This modulation brings competitive
selection, under softmax rule, to the determination of reorganization
gains. It is restricted with "intrinsic" as its input is the
derivative and the integral of intrinsic errors (in the reorganization
system) instead of extrinsic signals from the environment. I
correlated this mechanism with biological attention for it resembles
the later one in two aspects: both of them introduce the competition,
and both of them are concerned with variation and accumulation of the
input.

In my adapted simulation, the larger an intrinsic error varies, the
larger those connections correlated with it would be reorganized. For
example, if the intrinsic error of proximity controller varies in
large amount frequently, the connection from controller's comparator
to actuators, velocity and direction difference in the Crowd
simulation, will be reorganized with a larger gain. Same ideas are
applied to the integral of intrinsic errors. Greater an intrinsic
error accumulated, greater its correlated connections reorganized.
Competition is brought in to differentiate reorganization gains for
different controllers, especially those conflicting ones.

The primary purpose of adding this mechanism is to solve the conflict
in the reorganization of conflicting control systems, like the tropism
system and the avoidance system in the Crowd simulation. While one
system is being reorganized, the attention competition would help to
suppress the reorganization of the other one. In other words, this
modulation enhances the reorganization by playing the role of a switch
between the global mode and the local mode. It dynamically biases the
search space.

Then, some guesses came out.

The attentional modulation in my simulation implies that it could be
the modification to behavior patterns that attention actually
modulates instead of the behaviors themselves. For example, a person
riding bike can focus on talking to someone else and be unaware of
his/her riding behavior. Control systems producing unaware behaviors,
riding in this case, can work perfectly, but it can only work with a
still pattern being reorganized tinily for "intrinsic attention
system" suppresses the reorganization of them.

On the other hand, does this phenomenon imply what we are aware of is
what we are reorganizing? I don't have an answer for that right now. I
thought attention is the key to our single stream awareness; now it
turns out that it is more likely to be an enhancement to the
reorganization. You may regard the whole control system of an organism
as an iceberg: those "underwater" systems are producing behaviors with
still patterns (reorganization of these patterns is so tiny that it
can be ignored); systems "above the surface" are producing behaviors
with patterns being reorganized enormously.

In the perspective of evolution, attention was brought into nervous
system as an enhancement to the "reorganization system" (regard it as
an individual system temporarily) at first, because organisms equipped
with attentional mechanism would have higher stability in functional
control systems and faster convergence to lower intrinsic error. These
benefits would result in better fitness, which will help the mechanism
to be passed on.

Conjecture above suggests that awareness might be a mere "by-product"
of this evolution process, if it did have something to do with the
attention system and the reorganization system. It is the enhancement
attention brought to reorganization that the evolution actually
"favors".

Best regards,

Bo Wang

2

[From Bill Powers (2008.09.24.1359 MDT)]

Dear CSG Group,

This is Bo from Shanghai. I made an adapted "Crowd" simulation in
which individuals are equipped with "reorganization system" using
e-coli reorganization. The simulation applet could be found at:
http://sites.google.com/site/devevo/
This site is still under heavy construction. I put it out for I think
this would be a reference to people discussing the relationship
between reorganization and cognition in other threads recently.

I got it to run, Bo, but I don't understand what I see on the screen. A group of objects moves toward the right side and disappears out of the frame. Could you make the viewing area larger, or reduce the scale of the figure, so the objects remain visible? I tried to follow them with the scroll bars, but I can't see all the objects that way.

It looks as if the objects are avoiding collisions from the beginning, and moving toward a goal, also. What is being reorganized here? Do you have any ways to measure how much improvement of control there is? I know you Chinese guys are more intelligent than we Americans (of all types), but be kind to us so we can learn, too!

I agree with you that reorganization follows attention. Attention, I think, goes to where the errors are greatest, which is a fact that can be observed by anybody. So this explains how reorganization is applied to control systems where the error is largest and not to systems that are working well.

I am delighted that you are working on these important simulations, and I hope you don't mind when I offer suggestions or ideas, and ask for more explanation. Perhaps I would understand better if you say more about what we should give attention to in your simulation.

I'm also very happy that your thoughts about awareness and reorganization are so close to mine. I have guessed the same things you have: when attention is somewhere else, a control system goes on with the same organization. But if some change is needed, attention needs to be focused on the problem, so reorganization will act on the control system.

If you can make reorganization work properly with the Crowd program, you will have done something very important.

Best,

Bill P.

···

At 01:53 AM 9/25/2008 +0800, Bo Wang wrote:

3

[From Bill Powers (2008.09.24.1359 MDT)]

Dear CSG Group,

This is Bo from Shanghai. I made an adapted "Crowd" simulation in
which individuals are equipped with "reorganization system" using
e-coli reorganization. The simulation applet could be found at:
http://sites.google.com/site/devevo/
This site is still under heavy construction. I put it out for I think
this would be a reference to people discussing the relationship
between reorganization and cognition in other threads recently.

I got it to run, Bo, but I don't understand what I see on the screen. A
group of objects moves toward the right side and disappears out of the
frame. Could you make the viewing area larger, or reduce the scale of the
figure, so the objects remain visible? I tried to follow them with the
scroll bars, but I can't see all the objects that way.

I updated the applet; you may download it from the same location. The
viewport is larger now, and one can change its size by editing the
applet tag in the "crowd.html". I'm sorry for the inconvenience.

It looks as if the objects are avoiding collisions from the beginning, and
moving toward a goal, also. What is being reorganized here? Do you have any
ways to measure how much improvement of control there is?

I wrote some instructions on my google site, where more people could
see it, but it seems that these instructions are obscure. Those text
boxes are editable. If you alter any weights, you will see the change
of their behavior pattern. For example, changing the 0.00 in the
top-right text box on the "Weights" panel to 0.05 will make those
objects move away from the goal at the beginning. With the simulation
goes on, you will see how the reorganization eliminate this bias and
form a new pattern of moving toward the goal. You may use a larger
bias; larger ones consume more time to be eliminated. If you are
interested in the behaviors of a certain object, click it or put its
number to the text box following the label "Focus on the individual
NO.". As a result, text boxes representing output connections of
tropism/avoidance control systems on the "Weights" panel will display
parameters of the selected object. The default selected object is
obejct number 0. (Clicking the active object to select it is only
available in the latest version)

The connection represented by the text box in the example above
propagates the error signal of destination proximity control system to
the actuator changing the travel direction. In your orignial program,
no such connection exist, so its default value is 0. I explained
meanings of these text boxes on the website.

I know you Chinese
guys are more intelligent than we Americans (of all types), but be kind to
us so we can learn, too!

I'm sorry for the obscure explaination. The statement above really
make me feel bad.

I agree with you that reorganization follows attention. Attention, I think,
goes to where the errors are greatest, which is a fact that can be observed
by anybody. So this explains how reorganization is applied to control
systems where the error is largest and not to systems that are working well.

Exactly. The modulation in this simulation introduces a competition to
determine which control system, or a number of control systems, will
be reorganizaed larger than others. This competition is based on the
intrinsic error; control systems with larger intrinsic errors win.

I am delighted that you are working on these important simulations, and I
hope you don't mind when I offer suggestions or ideas, and ask for more
explanation.

That is why I put this on the CSGnet. I am delighted receiving your feedback.

Perhaps I would understand better if you say more about what we
should give attention to in your simulation.

Change the weights as you like, and see how the reorganization form a
new pattern to accomplish the tropism/avoidance job. The effect of
attentional modulation will be more evident when multiple connections
are need to be reorganized.

The learning process might be restricted by reorganization gains
sometimes, please feel free to change them. These gains are editable
through the simulation. I set their default value with my intuition,
you can find a better set. In the future, genetic algorithm will be
applied to look for the best set of these gains.

I'll keep on updating the website and the applet. Any suggestions and
ideas will be appreciated

Best,

Bo

···

2008/9/25 Bill Powers <powers_w@frontier.net>:

At 01:53 AM 9/25/2008 +0800, Bo Wang wrote:

4

[From Bill Powers (2008.09l.25.0853 MDT)]

I updated the applet; you may download it from the same location. The
viewport is larger now, and one can change its size by editing the
applet tag in the "crowd.html". I'm sorry for the inconvenience.

OK, that works better. (Don't be sorry -- I'm very happy with what you're doing). I think we would learn more if you made this into just one single moving person and start all the weights at zero, so at first there is no motion. You could make the motions wrap around when the object gets to the edge of the screen if reorganization is too slow to prevent losing the object. By starting with no motion, you can let reorganization slowly start the object moving. The criterion for reorganization of the collision-avoidance system would be an increasing proximity to an obstacle, and for reorganization of the destination-seeking system a decreasing proximity to the destination.

As the demonstration works right now, most of the objects avoid obstacles right away and move toward the destination right away. We don't see any learning happening because most of the objects behave correctly as soon as the program starts. I think you can make them learn to avoid obstacles if reorganization is caused by getting too close to an obstacle, and they can learn to seek the destination if that reorganization depends on whether they are moving more toward or more away from the destination. We might learn a lot by finding out what we have to do to make these two kinds of learning happen.

> I know you Chinese

> guys are more intelligent than we Americans (of all types), but be kind to
> us so we can learn, too!

I'm sorry for the obscure explaination. The statement above really
make me feel bad.

Hey, I think you really are very intelligent. I didn't mean to make you feel bad. It was just a joke between friends, and not entirely a joke. I'm very impressed by what is happening in your part of the world.

In the future, genetic algorithm will be applied to look for the best set of these gains.

Good idea.

Best,

Bill P.

···

At 11:08 AM 9/25/2008 +0800, Bo Wang wrote:

5

[From Bill Powers (2008.09l.25.0853 MDT)]

I think we would learn more if you made this into just one single
moving person and start all the weights at zero, so at first there is no
motion.

Thanks to your suggestion, I updated the applet again. Here is a
screenshot of a simulation in which objects start all the weights at
zero. It exhibits the diversity of reorganized moving patterns.

The new applet, initiated with the same default weights with previous
versions, can accept zero as start weights. You can edit those text
boxes on the setup panel, change them to zero before clicking the
start button. In addition, please pay attention to my reorganization
gains setting in the screen shot. To short the waiting time, I use a
greater gain at g11. It is also this relatively large value that
result in the generation of those quick unstable movements.

Since the e-coli reorganization is a stochastic process, putting a
crowd of people on the field will give a better impression about the
average performance. If you are interested in a specific person, use
the "Focus on the individual No" checkbox.

You could make the motions wrap around when the object gets to the
edge of the screen if reorganization is too slow to prevent losing the
object.

Did you mean transforming the plain into a torus? I will try that.

As the demonstration works right now, most of the objects avoid obstacles
right away and move toward the destination right away. We don't see any
learning happening because most of the objects behave correctly as soon as
the program starts. I think you can make them learn to avoid obstacles if
reorganization is caused by getting too close to an obstacle, and they can
learn to seek the destination if that reorganization depends on whether they
are moving more toward or more away from the destination. We might learn a
lot by finding out what we have to do to make these two kinds of learning
happen.

Before clicking the start button, you can do anything to those text
boxes representing weights and gains. Default values in those text
boxes, especially those in the "Weights" frame, are magic numbers
resulting the correct behavior. Actually, these numbers are adapted
from your original Crowd simulation, in which objects behave
correctly.

For example, let alone the start button, change the initial weights

2.0 0.0
0.0 0.05
20.0 0.0
0.0 0.05

to

2.0 0.05
0.0 0.05
20.0 0.0
0.0 0.05

first. Then click the start button, you will find out their behaviors
are incorrect; they are not moving toward the destination at the
beginning. With the simulation goes on, this bias would be eliminated
by the reorganization, and objects moving toward the destination
correctly will emerge.

Selecting these correctly behaving objects, by clicking them or input
their number in the text box on the control panel, contents in the
"Weights" frame will display settings of these objects. You will find
that the 0.05 you inputed before the start are replaced by a tiny
number closed to zero. Zero is the value resulting the correct
behavior you observed when you did not do anything before the start.
If you have been focusing on this object, you can see how this value
varies, from 0.05 back to 0.0 (nearly).

To see how the learning of two systems happen simultaneously, you may
change the initial weights to

2.0 0.05
0.0 0.05
20.0 0.0
0.0 -0.05

before the simulation starts. As a result, objects cannot move toward
the destination and avoid the obstacle. They run into obstacles
instead. With the simulation goes on, you will see how some objects
regain these abilities. Please feel free to change the initial
settings, both weights and reorganization gains, before the simulation
start. During the simulation, they are still editable. If you are not
satisfied with the time consumed by the learning process, alter the
reorganization gains immediately. You will see the effect of your
alterations immediately. The weights of control systems, however, are
not that easy to be edited, for they are being changed by the
reorganization all the time.

Looking forward to hear more from you.

Best,

Bo

···

2008/9/25 Bill Powers <powers_w@frontier.net>:

6

[From Bill Powers (2008.09.26.1245 MDT)]

The new applet, initiated with the same default weights with previous
versions, can accept zero as start weights. You can edit those text
boxes on the setup panel, change them to zero before clicking the
start button.

I tried that, and all the weights just stay at zero. I would expect the weights to begin changing immediately, and the direction and speed of the movements of the object to start changing, too. Instead, the objects just stay in one place and the weights continue to be zero.

I think we should check to see if your reorganization algorithm is the same as mine. Can you send me a text version of the applet you are using? I don't know how to read the .jar file. I might be able to see if we are doing this in the same way.

In the E. coli algorithm, each weight has a small correction term added to it on every iteration. The correction term is determined by a random number between -1 and 1, multiplied by a gain factor, and also multiplied by the total Root-Mean-Square error in the intrinsic variable that is being used as the criterion for reorganization. Multiplying the correction term by the total RMS error before adding it to the weight makes the speed of changing the weight decrease as the error decreases.

If the RMS error increases, a "tumble" occurs, meaning that a new random number is chosen for each correction term.

In my demonstrations, the total RMS error is measured not on every iteration but after some number of iterations. If the total error is greater or equal to the previous total error, a tumble occurs and the total error is saved as the previous error and then is reset to zero.

Whether there is a tumble or not, each correction term is applied to the corresponding weight on every iteration, so the weights are always changing. This is like E. coli swimming in a straight line between tumbles. The difference from the real bacteria is that the speed of swimming depends on the RMS error, and slows down as the error becomes smaller. This is not the speed of motion through space in the Crowd program, but the speed of change of the weights.

For one control system there are only two weights that can be changed: the speed weight and the direction weight. This should work properly for the destination-seeking system, but it may not work for the collision-avoidance system. The reason is that the collision avoidance control system must turn in a direction that depends on whether the obstacle is to the right or to the left of the direction of travel. Perhaps the reorganization would set up the weights properly. I don't know.

One way to handle this would be to allow the error signal in every control system to affect both the direction and the speed of motion (through different weights). Thus for all moving objects there would be eight weights. For both the collision avoidance system and the destination-seeking system, the proximity error would affect both speed and direction, and the direction error would also affect both speed and direction. Reorganization would gradually adjust the weights in the proper way. Reorganization of the collision system would use the RMS proximity error for obstacles, and reorganization of the destination-seeking system would use the sum of RMS proximity and RMS direction error for the destination.

That is how I would begin, although I don't know whether these initial ideas will really work.

Best,

Bill P.

···

To: Bo Wang (2008.09.25) --

In addition, please pay attention to my reorganization
gains setting in the screen shot. To short the waiting time, I use a
greater gain at g11. It is also this relatively large value that
result in the generation of those quick unstable movements.

Since the e-coli reorganization is a stochastic process, putting a
crowd of people on the field will give a better impression about the
average performance. If you are interested in a specific person, use
the "Focus on the individual No" checkbox.

> You could make the motions wrap around when the object gets to the
> edge of the screen if reorganization is too slow to prevent losing the
> object.

Did you mean transforming the plain into a torus? I will try that.

>
> As the demonstration works right now, most of the objects avoid obstacles
> right away and move toward the destination right away. We don't see any
> learning happening because most of the objects behave correctly as soon as
> the program starts. I think you can make them learn to avoid obstacles if
> reorganization is caused by getting too close to an obstacle, and they can
> learn to seek the destination if that reorganization depends on whether they
> are moving more toward or more away from the destination. We might learn a
> lot by finding out what we have to do to make these two kinds of learning
> happen.

Before clicking the start button, you can do anything to those text
boxes representing weights and gains. Default values in those text
boxes, especially those in the "Weights" frame, are magic numbers
resulting the correct behavior. Actually, these numbers are adapted
from your original Crowd simulation, in which objects behave
correctly.

For example, let alone the start button, change the initial weights

2.0 0.0
0.0 0.05
20.0 0.0
0.0 0.05

to

2.0 0.05
0.0 0.05
20.0 0.0
0.0 0.05

first. Then click the start button, you will find out their behaviors
are incorrect; they are not moving toward the destination at the
beginning. With the simulation goes on, this bias would be eliminated
by the reorganization, and objects moving toward the destination
correctly will emerge.

Selecting these correctly behaving objects, by clicking them or input
their number in the text box on the control panel, contents in the
"Weights" frame will display settings of these objects. You will find
that the 0.05 you inputed before the start are replaced by a tiny
number closed to zero. Zero is the value resulting the correct
behavior you observed when you did not do anything before the start.
If you have been focusing on this object, you can see how this value
varies, from 0.05 back to 0.0 (nearly).

To see how the learning of two systems happen simultaneously, you may
change the initial weights to

2.0 0.05
0.0 0.05
20.0 0.0
0.0 -0.05

before the simulation starts. As a result, objects cannot move toward
the destination and avoid the obstacle. They run into obstacles
instead. With the simulation goes on, you will see how some objects
regain these abilities. Please feel free to change the initial
settings, both weights and reorganization gains, before the simulation
start. During the simulation, they are still editable. If you are not
satisfied with the time consumed by the learning process, alter the
reorganization gains immediately. You will see the effect of your
alterations immediately. The weights of control systems, however, are
not that easy to be edited, for they are being changed by the
reorganization all the time.

Looking forward to hear more from you.

Best,

Bo

No virus found in this incoming message.
Checked by AVG - http://www.avg.com
Version: 8.0.169 / Virus Database: 270.7.2/1689 - Release Date: 9/24/2008 6:51 PM

7

I see. "The new applet" I meant is the latest one I uploaded when I
sent the previous post. Check it out.(Please download the zip package
again.)

The bad thing that I cannot show the applet online is viewers have to
download the updated version again and again. The previous version,
might be the one you tried, has the problem exactly as you described.
I fixed that before sending my previous post.

This is a quick response, I will go on to read the rest part of your reply.

Best,

Bo

···

2008/9/26 Bill Powers <powers_w@frontier.net>:

[From Bill Powers (2008.09.26.1245 MDT)]

To: Bo Wang (2008.09.25) --

The new applet, initiated with the same default weights with previous
versions, can accept zero as start weights. You can edit those text
boxes on the setup panel, change them to zero before clicking the
start button.

I tried that, and all the weights just stay at zero. I would expect the
weights to begin changing immediately, and the direction and speed of the
movements of the object to start changing, too. Instead, the objects just
stay in one place and the weights continue to be zero.