Human Robot interactions at MIT`

From Bill Powers (2009.01.01.1755 MST)]

I was trying to explain the effect of punishment; I thought it causes
the control system fails in some way, hence the reorganization system
make the parameters diverge from the old point.

Yes, that's how I see it, too. The only effect of punishment that matters, however, is that it makes the intrinsic error large; it is the large intrinsic error that starts reorganization, not simply failure of a control system that, as a side-effect of its action, was formerly keeping intrinsic error small. The control system might be working as well as it can, but some very large disturbance overpowers it, or the environment changes so as to make control impossible. Example: a person has no power to resist a dozen policemen putting him in jail, and they withold food which he cannot get for himself. This is not the fault of the person's control systems; no person could resist that kind of force. But reorganization will begin because food is withheld, or because the person is hurt, or exhausts himself and hurts himself in futile efforts to escape. They say that prisons are schools where people learn to be better criminals. The jailers can't keep them from reorganizing. Some day we will use that fact to cure criminals -- to make it worthwhile for them to stop hurting and offending others, instead of letting them learn how to be more successful criminals.

So far as I know, the question about increasing level of
complexity is a central one in today's evolutionary robotics,
artificial life, and evolutionary biology. I prefer the idea that this
trend towards complexity is passive. Nothing is judging the E.Coli,
who cannot undertake complex task. From my perspective, it is the
paradigms, like the hierarchy of control systems, that emerged in the
hereditary representations provide the opportunity to achieve higher
level of complexity.

Yes. Exactly. The more complex the goals that control systems seek, the more kinds of disturbances can affect them. A clam pumps water in and out of its safe shell and eats whatever washes in. A sea anemone waves its tentacles in the water and has a better chance for food to drift into them -- but it has to exposes its body and tentacles to do that, and develops stingers that clams don't need. An animal that can swim has to chase its food, so it is exposed to a still wider range of dangers, which require it to learn more survival skills. Very few clams are killed by fly swatters or in airplane crashes.

I think evolution makes more sense if we see it as active reorganization by an organism rather than simply a chance filtering out of "unfitness" by environmental variations. Biologists talk freely about evolution as if it is purposive, even though they criticise everyone else for saying it has purposes. I say the urge to talk that way is right: evolution IS purposive, though it's not God's purposes that organisms are serving, but their own. Evolution is the result of a reorganizing system carried in the genome. Just watch; they'll discover it pretty soon. They've already seen repair enzymes, which have the ability to edit the genome that is needed to change organization. They'll find the rest of the control loops before too much longer -- those guys are getting awfully good at this game.

I'm not a geneticist, but I guess there is some kind of genes, like
hox or within the hox, in which the hierarchy of control systems is
encoded. Just like giving the extra pair of wings, some mutations on
this paradigms would easily alter the structure of neural control
systems. Though this alteration might lead to different levels
complexity at the same time, higher ones would be preserved and passed
on if it did bring more chances to survive.

I think complexity reduces chances to survive, and organisms must reorganize to defend against the new disturbances that complexity allows. The reorganizations create still more complexities that introduce new dangers.

Cockroaches have been around for something like 600,000,000 years almost unchanged because they found a niche that protected them from many kinds of disturbances and did not have to reorganize any further. They have reached a local minimum of vulnerability. The human race, which is its own worst enemy, has not found a safe niche yet, and may never do so.

I'm not a geneticist, either: what is a "hox"?

Best,

Bill P.

···

At 12:54 AM 1/1/2009 +0800, Wang Bo wrote:

You may find something interesting here:

Homeotic genes, also known as Hox genes, specify the
anterior-posterior axis and segment identity of metazoan organisms
during early embryonic development.

and here:

A homeobox is a DNA sequence found within genes that are involved in
the regulation of patterns of development (morphogenesis) in animals,
fungi and plants.

Best,

Bo

···

2009/1/2 Bill Powers <powers_w@frontier.net>:

From Bill Powers (2009.01.01.1755 MST)]

I'm not a geneticist, either: what is a "hox"?

[From Bill Powers (2009.01.02.1248 MST)]

You may find something interesting here:

Homeotic gene - Wikipedia
Homeotic genes, also known as Hox genes, specify the
anterior-posterior axis and segment identity of metazoan organisms
during early embryonic development.

and here:
Homeobox - Wikipedia
A homeobox is a DNA sequence found within genes that are involved in
the regulation of patterns of development (morphogenesis) in animals,
fungi and plants.

When I read that stuff I get a creeping sensation up the back of my neck. Some truly great discoveries are sure to develop out of this work. And the main thought as I read it is "How could they possibly have discovered all these facts?" The model behind all this work is, as far as I am concerned, completely unhelpful -- it doesn't suggest any functions to me, any organizing principles. I think I would be a complete failure in this field of research; the kind of mind needed to work with this ambiguous kind of material is something special, extraordinary, a step above intelligence. It's not often that I can't think of a single helpful suggestion that would contribute at least a little bit to someone else's work. This genetic work leaves me without anything useful to say. This is the most powerful kind of basic science, in which there are no guideposts, no helpful analogies, no rules except those that are made up by the researchers, hardly even a language with which to describe what is found. Yet all these things have been created by a kind of bootstrap process, in which order is magically created out of chaos.

I can only hope that over the next few years the basic organizing principles will start to become visible, so the likes of me will at least be able to understand what is going on. I can just see those principles, swimming like wavering shadows just beneath the surface. It's all going to start making sense before long, thanks to people who, like daring trapeze artists, can do science without a safety net.

Best,

Bill P.

···

At 10:20 AM 1/2/2009 +0800, Bo Wang wrote:

From Bill Powers (2009.01.01.1755 MST)]

They say that prisons are schools where
people learn to be better criminals. The jailers can't keep them from
reorganizing. Some day we will use that fact to cure criminals -- to make it
worthwhile for them to stop hurting and offending others, instead of letting
them learn how to be more successful criminals.

It sounds like speciation.

I think evolution makes more sense if we see it as active reorganization by
an organism rather than simply a chance filtering out of "unfitness" by
environmental variations. Biologists talk freely about evolution as if it is
purposive, even though they criticise everyone else for saying it has
purposes. I say the urge to talk that way is right: evolution IS purposive,
though it's not God's purposes that organisms are serving, but their own.
Evolution is the result of a reorganizing system carried in the genome. Just
watch; they'll discover it pretty soon. They've already seen repair enzymes,
which have the ability to edit the genome that is needed to change
organization. They'll find the rest of the control loops before too much
longer -- those guys are getting awfully good at this game.

When I first came to the e.coli. reorganization, I thought it is very
similar to the reproduction-mutation-selection loop in biological
evolution. Regarding the organism space as the e.coli. space, I find
counterparts for those essentials:

mutations are very similar to random tumblings, as both of them do not
have a purpose;
ecological niche is similar to intrinsic reference just like you conveyed;
the frequency of genotypes would be the counterpart of the output of
reorganization systems. When a genotype is appeared at an overwhelming
frequency in the population, it restricts the future evolution in
certain directions. Though it is not guiding the change to hereditary
representations, it makes certain alterations easier to be made than
the others, just like e.coli. find a direction, following which it
does not have to tumble frequently.

I think complexity reduces chances to survive, and organisms must reorganize
to defend against the new disturbances that complexity allows. The
reorganizations create still more complexities that introduce new dangers.

Neither is higher complexity a sufficient condition nor a necessary
condition to higher survivability.
It is just a factor, or just an indicator.

[From Bill Powers (2009.01.02.1248 MST)]

When I read that stuff I get a creeping sensation up the back of my neck.
Some truly great discoveries are sure to develop out of this work. And the
main thought as I read it is "How could they possibly have discovered all
these facts?" The model behind all this work is, as far as I am concerned,
completely unhelpful -- it doesn't suggest any functions to me, any
organizing principles. I think I would be a complete failure in this field
of research; the kind of mind needed to work with this ambiguous kind of
material is something special, extraordinary, a step above intelligence.
It's not often that I can't think of a single helpful suggestion that would
contribute at least a little bit to someone else's work. This genetic work
leaves me without anything useful to say. This is the most powerful kind of
basic science, in which there are no guideposts, no helpful analogies, no
rules except those that are made up by the researchers, hardly even a
language with which to describe what is found. Yet all these things have
been created by a kind of bootstrap process, in which order is magically
created out of chaos.

I can only hope that over the next few years the basic organizing principles
will start to become visible, so the likes of me will at least be able to
understand what is going on. I can just see those principles, swimming like
wavering shadows just beneath the surface. It's all going to start making
sense before long, thanks to people who, like daring trapeze artists, can do
science without a safety net.

I am very glad seeing your concerns. This is why the evolution is
charming: it seems that some simple principles behind pushed all these
forward.

In this field, I found two primary directions: exploring the root of
adaptivity and studying the "experiences" biological evolution
accumulated. The first one is about what the fundamental of this
open-ended process is; so far as I know, in biological systems, this
could lead to the thermodynamics and the dissipative systems theory.
The second direction is concerned about how evolved hereditary units
are expressing themselves, which immediately introduced the biological
complexity. Such studies are still in their infant phase; most works
are records of phenomena, and rare theories exist.

Best regards,

Bo

···

2009/1/2 Bill Powers <powers_w@frontier.net>:
2009/1/3 Bill Powers <powers_w@frontier.net>:

(From Dick Robertson, 2008.12.17.1945CDT]

Just catching up with reading Science News I came to an article entitled, “Body in Mind” in SCIN Oct. 25/08. ( Bruce Bower, author). It described an MIT team that is creating robots that learn, interact with humans and supposedly that "cognition may actually evolve as physical experiences and actions ignite mental life. Interestingly control theory or feedback never crops up in the article. It is also not found in a 200+ Dissertation on the theory behind the work. The demonstrations are impressive, if the article is out front that these robots are actually constructed so they learn the various behaviors describe and are not subtly pre-programmed for it. The Diss. did include code sections that presumably tell how they worked, but I couldn’t see any feedback systems a work, but then my knowledge is too rudimentary to be sure. Anyway, if anyone wants to have a look see: the article, or robotic.media.mit.edu

Best,

Dick R

···

[From Bruce Nevin 2008.12.18.1130 EDT]

Dick Robertson, 2008.12.18.921CDT

I was thinking of the report that when the robot saw the man
trying to open the box that no longer had the cookies, he
(the robot) "shifted his gaze from one box to the other
and[unlocks the box where the cookies actually were, instead
of the one the man asked him to open.] That shows learning,
and control as I see it. I was assuming that (somehow) the
control was all there, but the MIT kids had managed to blind
themselves to the way in which they had unknowingly built it
in. But, I don’t have enough ability to decipher their code
to find it.

If the robot was in fact controlling, unlocking the correct box would be evidence that it had acquired a reference value for the correct location of the cookies. I don’t have access to the journal and can’t find it or any reference to Bower or “body” at http://robotic.media.mit.edu . Was there evidence of control other than the shifting of gaze before opening the correct box? Was this a Theory of Mind setup, where the human going to the wrong box had been out of the room when the robot saw the cookies get moved? Doesn’t make sense–in that setup the subject (usually a child) is asked which box will be chosen by the person who was out of the room.

Control and learning are connected, obviously, but not the same thing. The confusion of learning and control is pervasive. As we know, linear-causation psychologies are explanatory systems for what amount to techniques for influencing reorganization (that is, for training organisms). Relying on these explanations, psychologists believe they are accounting for behavior. Since a change in what is controlled results in changes in behavior they can appear to be right, especially to those who want to control the behavior of others.

/B

[From Bruce Nevin 2008.12.18.1130
EDT]

If the robot was
in fact controlling, unlocking the correct box would be evidence that it
had acquired a reference value for the correct location of the cookies. I
don’t have access to the journal and can’t find it or any reference to
Bower or “body” at
http://robotic.media.mit.edu
.

I couldn’t find it, either, under doctoral theses or “SM”
theses.

Dick, how about narrowing it down?

Control and
learning are connected, obviously, but not the same thing. The
confusion of learning and control is pervasive. As we know,
linear-causation psychologies are explanatory systems for what amount to
techniques for influencing reorganization (that is, for training
organisms). Relying on these explanations, psychologists believe they are
accounting for behavior. Since a change in what is controlled results in
changes in behavior they can appear to be right, especially to those who
want to control the behavior of others.

I haven’t seen this observation before, Bruce. I think it merits an
article, akin to a review article. I’m going to copy this to Warren
Mansell, who will be managing editor of the second edition of Intro to
Modern Psychology, with the suggestion that you be asked to write,
yourself or with colleagues from the CSG of your choosing, an extended
treatment of this and other similar comparisons. The introduction to
modern psychology would not be complete without contrasting it with
old-fashioned psychology. I think you have put your finger on an
exceedingly important difference.

PCT is young enough that it has members who acquired an expertise in
old-fashioned psychology, including advanced degrees, before ever hearing
of PCT. They are the only ones living right now who have credentials in
both worlds, and may be among the last. Their views of the transition
will be of critical importance to our descendants and students of the
philosophy of science.

Best,

Bill P.

···

At 11:30 AM 12/18/2008 -0500, Bruce Nevin (bnevin) wrote:

The last time I posted this notion was during an extended thread about EAB involving mostly you and Bruce Abbott. My emphasis then was that such research has relevance and value for PCT, but only as pertains to reorganization, which is relatively little explored in PCT (your simulation of organization de novo being a notable contribution). The explanations of what they are doing are of course mostly unicorn polish.

/B
···

From: Control Systems Group Network (CSGnet) [mailto:CSGNET@LISTSERV.ILLINOIS.EDU] On Behalf Of Bill Powers
Sent: Thursday, December 18, 2008 12:13 PM
To:
CSGNET@LISTSERV.ILLINOIS.EDU
Subject: Re: Human Robot interactions at MIT`

At 11:30 AM 12/18/2008 -0500, Bruce Nevin (bnevin) wrote:

[From Bruce Nevin 2008.12.18.1130 EDT]
If the robot was in fact controlling, unlocking the correct box would be evidence that it had acquired a reference value for the correct location of the cookies. I don't have access to the journal and can't find it or any reference to Bower or "body" at [http://robotic.media.mit.edu](http://robotic.media.mit.edu/)
.

I couldn’t find it, either, under doctoral theses or “SM” theses.
Dick, how about narrowing it down?

Control and learning are connected, obviously, but not the same thing.  The confusion of learning and control is pervasive. As we know, linear-causation psychologies are explanatory systems for what amount to techniques for influencing reorganization (that is, for training organisms). Relying on these explanations, psychologists believe they are accounting for behavior. Since a change in what is controlled results in changes in behavior they can appear to be right, especially to those who want to control the behavior of others.

I haven’t seen this observation before, Bruce. I think it merits an article, akin to a review article. I’m going to copy this to Warren Mansell, who will be managing editor of the second edition of Intro to Modern Psychology, with the suggestion that you be asked to write, yourself or with colleagues from the CSG of your choosing, an extended treatment of this and other similar comparisons. The introduction to modern psychology would not be complete without contrasting it with old-fashioned psychology. I think you have put your finger on an exceedingly important difference.

PCT is young enough that it has members who acquired an expertise in old-fashioned psychology, including advanced degrees, before ever hearing of PCT. They are the only ones living right now who have credentials in both worlds, and may be among the last. Their views of the transition will be of critical importance to our descendants and students of the philosophy of science.

Best,

Bill P.

[From Bill Powers (2008.12.19.0819 MST)]

Dick Robertson.2008.12.18.2155CDT --

I found the dissertation, thanks.

Pretty hard to figure out that code; from the diagrams it looks as if the feedback all occurs outside the program when it runs. I don't see how the teddy bear finds the human's hand and moves its own to meet it. The text refers to the regularities in the human's hand position -- I suspect that it's the human who does all the necessary adjusting, while the robot just puts its hand in a preset position. It seems to be a logic-level program with some sequential control, very reminiscent of the AI stuff we used to see. The program decides what to do and then says "do it" -- but how that gets turned into just the right actions isn't clear. It seems mostly open-loop. I'll have to read it again.

Lots of window-dressing in this project -- the cute robot, the patty-cake game. It would be lovely to have an unlimited budget.

Best,

Bill P.

[From Bill Powers (2008.12.23.1227 MST)]

...somebody is closing the gap. Here is an article "Emotions: from
brain to robot" http://www.snl.salk.edu/~fellous/pubs/ArbibFellousTICS2004.pdf

The section on neurobiological roots of emotion is most interesting.
I found some points there, like the regulation of motivated
behaviors, are really close to the ideas in PCT, and am looking
forward to comments from CSG people.

I didn't see the places you refer to -- it would help if you would
quote some of them. It seems to me that Arbib doesn't have any very
clear idea of what emotion is -- or for that matter, "motivation." In
PCT what motivates behavior is error signals, differences between
what is being perceived and what a reference signal is specifying to
be perceived. I don't see any ideas like that in this article.

For reference, I'm attaching another version of my ideas on emotion.
I keep trying to find clearer ways of describing it. This doesn't
sound much like what Arbib et. al. are saying.

Best,

Bill P.

emotion20081223.doc (82.5 KB)

···

At 02:40 PM 12/23/2008 +0800, Wang Bo wrote:

[From Bill Powers (2008.12.24.1524 MST)]

I can add to your rolling iceberg a balloon overhead.
Awareness tends to dwell in the middle levels of
the hierarchy, and not so much in the lower or upper levels.

Like a tumbler?

I'll read those papers later. It's interesting that in all of these "genetic
algorithm" models that I have seen, the designers have to let the evolving
organism reproduce even though it hasn't yet learned how to do what is
necessary to survive. In your last quote above, we see that 50 generations
had to pass before the Quadropod "succeeded in moving towards high
concentration of simulated chemicals at last." If it didn't actually reach
the chemicals, how did it survive through 49 generations? It survived
because the kindly programmer saw it was changing in the right direction,
and allowed it to survive. There is no kindly programmer helping real
organisms that way.

You can see that the
genetic algorithm programmers have had to do this too -- they are serving as
the comparators in the reorganizing system, because they can see when the
changes are leading toward the goal or away from it. Their system tries to
work by "rewarding" behavior that leads toward a favorable result, which is
the idea behind reinforcement theory (though Skinner's version demands that
the favorable result actually occurs). In reorganization theory, the
organization of behavior is altered only when the result of behavior is
actually unfavorable.

I'm kind of against the use of fixed fitness function; just as you
said, the programmer will bring in the knowledge of final goals. In
"The Blind Watchmaker", Dr. Dawkins used an "evolutionary" program to
guess a line in Shakespeare's script. Though it did succeed, it seems
that the program know the result at the beginning, for its fitness
function is defined as how many different characters there are between
the generated sentence and Shakespeare's words. Accordingly, I prefer
evolving fitness functions, as they represent the ecological niche,
which is ever changing. I'm aware of that the fitness functions are
relatively static during a very short period in the history of
evolution.

Here is another stream in the evolutionary computation: people use a
simulated ecological system instead of a single population to evolve
the neural network, and the fitness function was replaced by a single
criteria "dead or alive". The following presentation "Polyworld: Using
Evolution to Design Artificial Intelligence" gave a remarkable
example.

The video is a bit lengthy, over an hour; you may watch it if you got time.

In all theories of learning, it turns out that something has to detect
whether the changes have more-favorable or more-unfavorable results. If
that's not done, survival becomes far too unlikely. The difficulties arise
when we ask how an organism, or the environment, can know that a change took
the organism closer to a favorable result, when that result didn't actually
occur. Reorganization theory offers a solution for that problem that doesn't
involve a helpful programmer.

In a space of adjustable parameters, from my perspective, the task of
learning is to diverge from the old point and converge to a new one;
the new point is commonly better according to some metrics.
Reorganization theory explains the divergence and convergence within
the same framework. For example, it gives punishments, which only make
the parameters diverge, a position, while the reinforcement learning
don't. Moreover, to improve the learning, we may need a method to bias
the space; I brought in attention months ago for this purpose.

Another problem is how to add dimensions to this space, say how to
increase the complexity of the system, and this is what the evolution
should do, at the phylogenetic scale.

P.S. Please tell us which is your family name and which is your personal
name.

It's "Wang, Bo". My surname is "Wang", and "Bo" is my given name.

Merry Christmas!

Bo

···

2008/12/25 Bill Powers <powers_w@frontier.net>:

[From Rick Marken (2008.12.17.2100)]

Dick Robertson (2008.12.17.1945CDT)–

Just catching up with reading Science News I came to an article entitled, “Body in Mind” in SCIN Oct. 25/08. ( Bruce Bower, author). It described an MIT team that is creating robots that learn, interact with humans and supposedly that "cognition may actually evolve as physical experiences and actions ignite mental life. Interestingly control theory or feedback never crops up in the article. It is also not found in a 200+ Dissertation on the theory behind the work.

Actually, control theory (the real thing, the kind that recognizes that it is a perceptual signal that is controlled) never crops up much in any discussion of robots or living systems – except here;-)

The demonstrations are impressive

Where are the demonstrations? I saw a cute video with a rather attractive robot. But impressive would be seeing it control. I saw so nifty looking output generation in the video but no obvious control (maintenance of a variable in a predetermined state, protected from disturbance). Also, where is the Dissertation with the code?

The Diss. did include code sections that presumably tell how they worked, but I couldn’t see any feedback systems a work,

Since these are mobile robots there is feedback in the fact that what is being perceived by the robot depends on what the robot does in the environment; and what the the robot does depends on what it perceives, which we know from the description of the robot (what it perceives – facial emotions, I think – causes what it does).

There is a lot of ignoring of feedback in the robotic literature; same as in psychology. One example I ran across was a robot called a Braitenberg vehicle (named after the fellow, also at MIT, who invented them). These were virtual robots. They were software objects operating in a software environment. But they were described as S-R devices because the output (movement of the right or left wheel, which is the Response) of the vehicle was caused by the magnitude of optical input to the right or left side of the vehicle’s sensor (the light being the Stimulus).

These vehicles could control for staying aligned with a lighted path; so it was concluded that an S-R device could control (steer). In fact, though, these S-R devices are actually control systems, with the feedback from output to sensory input, going through the vehicle’s simulated environment. The reference for the input is not explicitly specified so it was at the 0 point of the input range, which turns out to keep the optical input centered. The vehicle’s outputs keep it moving forward and moving left and right, as necessary, to keep the optical variable centered and, hence, the vehicle on the lighted path.

The point of all this is that robot engineers are often able to design workable systems based on what are basically S-R or open loop causal models (the models they borrow from psychologists) because robots are in the same situation as people; they exist in a closed loop. And if the sign of the feedback in the loop is negative and the loop gain and slowing are appropriate then behavior of the robot will be stable (the system will control something).

Best

Rick

···

Richard S. Marken PhD
rsmarken@gmail.com

[From Dick Robertson, 2008.12.18.921CDT]

[From Rick Marken (2008.12.17.2100)]

Dick Robertson (2008.12.17.1945CDT)–

Just catching up with reading Science News I came to an article entitled, “Body in Mind” in SCIN Oct. 25/08. ( Bruce Bower, author). It described an MIT team that is creating robots that learn, interact with humans and supposedly that "cognition may actually evolve as physical experiences and actions ignite mental life. Interestingly control theory or feedback never crops up in the article. It is also not found in a 200+ Dissertation on the theory behind the work.

Actually, control theory (the real thing, the kind that recognizes that it is a perceptual signal that is controlled) never crops up much in any discussion of robots or living systems – except here;-)

The demonstrations are impressive

Where are the demonstrations? I saw a cute video with a rather attractive robot. But impressive would be seeing it control. I saw so nifty looking output generation in the video but no obvious control (maintenance of a variable in a predetermined state, protected from disturbance). Also, where is the Dissertation with the code?

I was thinking of the report that when the robot saw the man trying to open the box that no longer had the cookies, he (the robot) "shifted his gaze from one box to the other and[unlocks the box where the cookies actually were, instead of the one the man asked him to open.] That shows learning, and control as I see it. I was assuming that (somehow) the control was all there, but the MIT kids had managed to blind themselves to the way in which they had unknowingly built it in. But, I don’t have enough ability to decipher their code to find it.

The dissertation is at robotic.media.mit.edu

The Diss. did include code sections that presumably tell how they worked, but I couldn’t see any feedback systems a work,

Since these are mobile robots there is feedback in the fact that what is being perceived by the robot depends on what the robot does in the environment; and what the the robot does depends on what it perceives, which we know from the description of the robot (what it perceives – facial emotions, I think – causes what it does).

There is a lot of ignoring of feedback in the robotic literature; same as in psychology. One example I ran across was a robot called a Braitenberg vehicle (named after the fellow, also at MIT, who invented them). These were virtual robots. They were software objects operating in a software environment. But they were described as S-R devices because the output (movement of the right or left wheel, which is the Response) of the vehicle was caused by the magnitude of optical input to the right or left side of the vehicle’s sensor (the light being the Stimulus).

These vehicles could control for staying aligned with a lighted path; so it was concluded that an S-R device could control (steer). In fact, though, these S-R devices are actually control systems, with the feedback from output to sensory input, going through the vehicle’s simulated environment. The reference for the input is not explicitly specified so it was at the 0 point of the input range, which turns out to keep the optical input centered. The vehicle’s outputs keep it moving forward and moving left and right, as necessary, to keep the optical variable centered and, hence, the vehicle on the lighted path.

The point of all this is that robot engineers are often able to design workable systems based on what are basically S-R or open loop causal models (the models they borrow from psychologists) because robots are in the same situation as people; they exist in a closed loop. And if the sign of the feedback in the loop is negative and the loop gain and slowing are appropriate then behavior of the robot will be stable (the system will control something).

Right, but I thought it might be interesting if you could show either that there was a hidden hierarchy of control systems involved, or that there were preprogrammed RS’s that made the robots look more indendent-acting than they really were.

Best,

Dick R.

···

Date: Wednesday, December 17, 2008 11:03 pm
Subject: Re: Human Robot interactions at MIT`

(From Dick Robertson.2008.12.18.2155CDT)

···
/B

I couldn’t find it, either, under doctoral theses or “SM” theses.
Dick, how about narrowing it down?

On robotic.media.mit.edu Home page click the teddy bear on left in second row, then click on the guy holding the lamp and then click on Hoffman’s Ph D. thesis. It looks like he is just talking about how the lamp controls but he doesn’t take that up until the last chapter of the thesis. In the earlier chapters he discusses his cognition/memory theory and gives the code that I think is used in most of the different robots.

Best,

Dick R.

[From Dick Robertson (2008.12.19.0950CDT)]

[From Bill Powers (2008.12.19.0819 MST)]

Dick Robertson.2008.12.18.2155CDT –

I found the dissertation, thanks.

Pretty hard to figure out that code; from the diagrams it looks
as if the feedback all occurs outside the program when it runs. I
don’t see how the teddy bear finds the human’s hand and moves its own to
meet it. The text refers to the regularities in the human’s hand
position
– I suspect that it’s the human who does all the necessary
adjusting, while the robot just puts its hand in a preset
position.

So, as usual the teddy bear gets credit for “learning” to do what the programmers programmed him to do?

It seems to be a logic-level program with some sequential
control,
very reminiscent of the AI stuff we used to see. The program
decides
what to do and then says “do it” – but how that gets turned
into
just the right actions isn’t clear. It seems mostly open-loop.
I’ll
have to read it again.

Lots of window-dressing in this project – the cute robot, the
patty-cake game. It would be lovely to have an unlimited budget.

Good stuff to attract science writers, eh?

Best,

Dick R

[From Bill Powers (2008.12.23.1227 MST)]

I didn't see the places you refer to -- it would help if you would quote
some of them. It seems to me that Arbib doesn't have any very clear idea of
what emotion is -- or for that matter, "motivation." In PCT what motivates
behavior is error signals, differences between what is being perceived and
what a reference signal is specifying to be perceived. I don't see any ideas
like that in this article.

It starts at page 555.
"An animal comes with a set of basic 'drives' that provide the 'motor'
(motivation) for behavior. Most of the neural circuitry underlying
these drives involves specific nuclei of the hypothalamus. Swanson
[24] introduced the notion of the 'behavioral control column' (Figure
1), comprising interconnected sets of nuclei in the hypothalamus
underlying specific survival behaviors: spontaneous locomotion,
exploration, ingestive, defensive and reproductive behaviors. The
hypothalamus sends this information to higher centers such as the
amygdala and the orbitofrontal cortex."

I'm aware of that error signals motivate behaviors in PCT. In my
opinion, the stuff he was trying to refer to is more likely to be the
reference, and Figure 2(a) looks like the diagram of a control system
in PCT, if signals coming from the hypothalamus are reference of the
mentioned system. I don't know much about structure of the brain; the
hypothalamus seems to be just a third order structure instead of some
ultimate level sending out the primitive reference, according to B:CP.

For reference, I'm attaching another version of my ideas on emotion. I keep
trying to find clearer ways of describing it. This doesn't sound much like
what Arbib et. al. are saying.

The ideas are wonderful! From my perspective, interactions in the
hierarchy of control system might caused the emotion, a side-effect as
you recognize it, to be perceived as a stimuli in the S-R system. For
example, behaviors of control system A bring up some changes, not only
in the controlled perceptions of system A but also in other parts of
the whole system, and perceptions of these "other parts" are
controlled by another system B, whose reference value is static at
that time. In this process, the signals go down a branch of system A,
and go up through another path in the system B, then go down again.
People losing sight of the whole process could easily link the
emotion, brought up in the first "valley" of the go up and down
process, with the behaviors produced in system B.

Besides, it is exciting to see the idea "Awareness is mobile". I
recognize the awareness as a "rolling ice ball floating in the water";
the part beneath the water surface is what we are unaware of. It's
nothing like Freud's iceberg metaphor as his "iceberg" never roll. If
there is something hard to be awared of, it is not included in this
ball. I am not quite sure about what kind of "forces" make the ball
roll; it could be something like attention.

ps:
Here is another interesting example about the controller in robots. In
a summary of evolutionary robotics:
http://www.mae.cornell.edu/ccsl/papers/Biomimetics05_Lipson.pdf
Professor Lipson described an experiment in the section "Evolving
Controllers" on page 4. In Figure 1(b), there are two "mysterious"
nodes B1 and B2 without any explanation in the related text, which is
filled up with the words like "mapping inputs to outputs". In the
reference "Evolved Sensor Fusion and Dissociation in an Embodied
Agent" about the original work
http://www.cs.uvm.edu/~jbongard/papers/BongardWGW02.pdf
there is a single sentence mentioning these two:
"There is an additional bias neuron at the input and hidden layers
that outputs a constant signal of 1."
In this experiment, 200 candidate controllers were evolved for 50
generations, and the Quadrapod succeeded in moving towards high
concentration of simulated chemicals at last. I don't know how output
weights of B1 and B2 evolve in the simulation, but I guess their
outputs are recognized as reference signals and negative feedback
emerged.

···

2008/12/24 Bill Powers <powers_w@frontier.net>:

[From Bill Powers (2008.12.24.1524 MST)]

In PCT what motivates
behavior is error signals, differences between what is being perceived
and what a reference signal is specifying to be perceived. I don’t see
any ideas like that in this article.

It starts at page 555.

"An animal comes with a set of basic ‘drives’ that provide the
‘motor’

(motivation) for behavior.

According to PCT, this is incorrect. The motivation for an animal’s
behavior is the same as the motivation for a human being’s behavior: a
difference between what is being perceived and a magnitude or state that
a reference signals specifies as the desired or intended perception.
There are no “drives” and there is no such thing as
“motivation.” Those words go with a model of behavioral
organization based on some bad guesses, a model which PCT replaces
entirely.

Most of the neural circuitry
underlying

these drives involves specific nuclei of the hypothalamus. Swanson

[24] introduced the notion of the ‘behavioral control column’
(Figure

1), comprising interconnected sets of nuclei in the hypothalamus

underlying specific survival behaviors: spontaneous locomotion,

exploration, ingestive, defensive and reproductive behaviors. The

hypothalamus sends this information to higher centers such as the

amygdala and the orbitofrontal cortex."

If PCT is correct, this paragraph is a remarkable collection of
confusions. The hypothalamus sends information to higher centers, yes:
but that is the set of perceptual signals (some of them) being controlled
by those higher centers, which compare the perceptions against
their reference signals and, on the basis of the error, adjust the
reference signals in the hypothalamus by a return path. This is the
somatic branch of the hierarchy to which I refer; I would guess that the
upgoing signals carry the “feeling” perceptions of which my
model speaks, though of course those can come from elsewhere, too. The
downgoing signals set reference signals for the patterns of physiological
states that are adjusted to fit with the actions governed by the
behavioral branch.

I’m aware of that error signals
motivate behaviors in PCT. In my

opinion, the stuff he was trying to refer to is more likely to be
the

reference, and Figure 2(a) looks like the diagram of a control
system

in PCT, if signals coming from the hypothalamus are reference of the

mentioned system. I don’t know much about structure of the brain;
the

hypothalamus seems to be just a third order structure instead of
some

ultimate level sending out the primitive reference, according to
B:CP.

I agree, I put it at the third order of control, too: the controlled
variables are configurations of somatic sensations.

For reference, I’m
attaching another version of my ideas on emotion. I keep

trying to find clearer ways of describing it. This doesn’t sound
much like

what Arbib et. al. are saying.

The ideas are wonderful! From my perspective, interactions in the

hierarchy of control system might caused the emotion, a side-effect
as

you recognize it, to be perceived as a stimuli in the S-R
system.

What SR system? There are no SR systems in organisms. You have to get
used to that idea if you’re going to use PCT. Individual components of
control systems convert input signals to output signals, but these
components make up a set of control systems which are not SR systems.
They are not SR systems because they control, and they are not SR systems
because we do not conceive of inputs and outputs of components are
consisting of brief events (except at one proposed level, the fifth). In
lower-order systems from intensities to relationships, neural signals are
continuously variable, not discrete, and at all levels there are
indications of an underlying continuum even when discrete variables are
used.

For example, behaviors of
control system A bring up some changes, not only

in the controlled perceptions of system A but also in other parts
of

the whole system, and perceptions of these “other parts”
are

controlled by another system B, whose reference value is static at

that time. In this process, the signals go down a branch of system
A,

and go up through another path in the system B, then go down again.

People losing sight of the whole process could easily link the

emotion, brought up in the first “valley” of the go up and
down

process, with the behaviors produced in system B.

Yes, but wait until you see some of the demonstrations in the new book,
which show multiple control systems can share a common environment which
they all sense and affect, yes continue to control independent aspects of
that environment. The phenomenon you describe is just the tip of the
iceberg.

Besides, it is exciting to see
the idea “Awareness is mobile”. I

recognize the awareness as a “rolling ice ball floating in the
water”;

the part beneath the water surface is what we are unaware of. It’s

nothing like Freud’s iceberg metaphor as his “iceberg” never
roll. If

there is something hard to be awared of, it is not included in this

ball. I am not quite sure about what kind of “forces” make the
ball

roll; it could be something like attention.

Great minds think in the same channels, right? I can add to your rolling
iceberg a floating balloon. Awareness tends to dwell in the middle levels
of the hierarchy, and not so much in the lower or upper levels. If
I asked you to tell me right now what goals are behind the fact that
you’re reading this post at this instant, the first thing you’d have to
do would be to stop reading. Then you would have to look inward, and
goals would come to your awareness, and you would tell me. But you
couldn’t tell me the answer until you had moved your awareness away from
the main thing you were consciously doing. The higher goals and control
systems were obviously operational all the time, but you weren’t
conscious of them.

ps:

Here is another interesting example about the controller in robots.
In

a summary of evolutionary robotics:


http://www.mae.cornell.edu/ccsl/papers/Biomimetics05_Lipson.pdf

Professor Lipson described an experiment in the section
"Evolving

Controllers" on page 4. In Figure 1(b), there are two
“mysterious”

nodes B1 and B2 without any explanation in the related text, which
is

filled up with the words like “mapping inputs to outputs”. In
the

reference "Evolved Sensor Fusion and Dissociation in an
Embodied

Agent" about the original work


http://www.cs.uvm.edu/~jbongard/papers/BongardWGW02.pdf

there is a single sentence mentioning these two:

"There is an additional bias neuron at the input and hidden
layers

that outputs a constant signal of 1."

In this experiment, 200 candidate controllers were evolved for 50

generations, and the Quadrapod succeeded in moving towards high

concentration of simulated chemicals at last. I don’t know how
output

weights of B1 and B2 evolve in the simulation, but I guess their

outputs are recognized as reference signals and negative feedback

emerged.

I’ll read those papers later. It’s interesting that in all of these
“genetic algorithm” models that I have seen, the designers have
to let the evolving organism reproduce even though it hasn’t yet learned
how to do what is necessary to survive. In your last quote above, we see
that 50 generations had to pass before the Quadropod “succeeded in
moving towards high concentration of simulated chemicals at last.”
If it didn’t actually reach the chemicals, how did it survive through 49
generations? Because the kindly programmer saw it was changing in the
right direction, and allowed it to survive. There is no kindly programmer
helping real organisms that way.
In PCT we use a different basic principle, that of reorganization.
Reorganization depends on sensing some critical variable and detecting
whether it is changing toward or away from some specific reference level.
Changes in system’s properties – the coefficients in the equations that
define it – are always going on at different rates; those different
rates are like E. coli swimming in a straight line in some hyperspace of
coefficients. If the error signal is decreasing, those changes continue.
If the error signal ever increases, however, this reorganizing system
“tumbles” by altering the rates of change of all the
coefficients, at random. The random tumbles continue until the error is
once again decreasing. The system now swims in a new direction through
the coefficient hyperspace. To make the system converge to a final result
properly, the speed of swimming is determined by the total squared error
in all the critical variables.
You knew all that, but I thought I’d just say it again. You can see that
the genetic algorithm programmers have had to do this too – they are
serving as the comparators in the reorganizing system, because they can
see when the changes are leading toward the goal or away from it. Their
system tries to work by “rewarding” behavior that leads toward
a favorable result, which is the idea behind reinforcement theory
(though Skinner’s version demands that the favorable result actually
occurs). In reorganization theory, the organization of behavior is
altered only when the result of behavior is unfavorable.
In all theories of learning, it turns out that something has to sense
whether the changes have more-favorable or more-unfavorable results. The
difficulties arise when we ask how an organism can know that a change
took it closer to a favorable result, when that result didn’t actually
occur. Reorganization theory offers a solution for that problem that
doesn’t involve a helpful programmer.
Best,
Bill P.
P.S. Please tell us which is your family name and which is your personal
name. It’s very confusing because Chinese writers, being friendly and
polite people, sometimes switch the two names around and put the personal
name first so we Westerners (who live to the east of you) will know which
is the “first name”. But of course, some Chinese writers
don’t do that, so we never know whether to switch the names back
or not.

In the English-speaking American navy, I would write my name on forms or
speak it to superior officers as “Powers, William, 7275198” The
comma, or a little drop in tone and a pause when speaking, indicate that
the last name is being given first (to please clerks who like to list
people alphabetically by their family names). So ALL YOU GUYS OVER THERE,
how about simply writing your names the way you like to see them, and
putting a comma after the family name, as in

“Wang, Bo” – or is it “Bo, Wang”? If we don’t see
the comma we’'ll assume you’ve put the family name last the way we do. If
we do see it, we’ll leave it the way it is. If each of you oriental types
tells two others (who haven’t been told by someone else) all of you will
be doing this in less than one day and the world will be a better
place.

WTP, also known as P, WT.

···

At 08:14 PM 12/24/2008 +0800, Wang Bo wrote:

[From Bill Powers (2008.12.24.1524 MST)]

In PCT what motivates
behavior is error signals, differences between what is being perceived
and what a reference signal is specifying to be perceived. I don’t see
any ideas like that in this article.

It starts at page 555.

"An animal comes with a set of basic ‘drives’ that provide the
‘motor’

(motivation) for behavior.

According to PCT, this is incorrect. The cause of an animal’s behavior is
the same as the cause of a human being’s behavior: a difference between
what is being perceived and a magnitude or state that a reference signal
specifies as the desired or intended perception. There are no
“drives” and there is no such thing as “motivation.”
Those words go with a model of behavioral organization based on some bad
guesses, a model which PCT replaces entirely.

Most of the neural circuitry
underlying

these drives involves specific nuclei of the hypothalamus. Swanson

[24] introduced the notion of the ‘behavioral control column’
(Figure

1), comprising interconnected sets of nuclei in the hypothalamus

underlying specific survival behaviors: spontaneous locomotion,

exploration, ingestive, defensive and reproductive behaviors. The

hypothalamus sends this information to higher centers such as the

amygdala and the orbitofrontal cortex."

If PCT is correct, this paragraph is a remarkable collection of
confusions. The hypothalamus sends information to higher centers, yes:
but that is the set of perceptual signals (some of them) being controlled
by those higher centers, which compare the perceptions against
their reference signals and, on the basis of the error, adjust the
reference signals in the hypothalamus by a return path. This is the
somatic branch of the hierarchy to which I refer; I would guess that the
upgoing signals carry the “feeling” perceptions of which my
model speaks, though of course those can come from elsewhere, too. The
downgoing signals set reference signals for the patterns of physiological
states that are adjusted to fit with the actions governed by the
behavioral branch.

I’m aware of that error signals
motivate behaviors in PCT. In my

opinion, the stuff he was trying to refer to is more likely to be
the

reference, and Figure 2(a) looks like the diagram of a control
system

in PCT, if signals coming from the hypothalamus are reference of the

mentioned system. I don’t know much about structure of the brain;
the

hypothalamus seems to be just a third order structure instead of
some

ultimate level sending out the primitive reference, according to
B:CP.

I agree, I put it at the third order of control, too: the controlled
variables are configurations of somatic – biochemical –
sensations.

For reference, I’m
attaching another version of my ideas on emotion. I keep

trying to find clearer ways of describing it. This doesn’t sound
much like

what Arbib et. al. are saying.

The ideas are wonderful! From my perspective, interactions in the

hierarchy of control system might caused the emotion, a side-effect
as

you recognize it, to be perceived as a stimuli in the S-R
system.

What SR system? There are no SR systems in organisms. You have to get
used to that idea if you’re going to use PCT. Individual components of
control systems convert input signals to output signals, but these
components make up a set of control systems which are not SR systems.
They are not SR systems because they control, and they are not SR systems
because we do not conceive of inputs and outputs of components as
consisting of brief events (except at one proposed level, the fifth). In
lower-order systems, from intensities to relationships, neural signals
are continuously variable, not discrete, and at all levels there are
indications of an underlying continuum even when discrete variables are
used. I have mentioned that I think the “event” level is out of
place, but I don’t know where to put it.

For example, behaviors of
control system A bring up some changes, not only

in the controlled perceptions of system A but also in other parts
of

the whole system, and perceptions of these “other parts”
are

controlled by another system B, whose reference value is static at

that time. In this process, the signals go down a branch of system
A,

and go up through another path in the system B, then go down again.

People losing sight of the whole process could easily link the

emotion, brought up in the first “valley” of the go up and
down

process, with the behaviors produced in system B.

Yes, but wait until you see some of the demonstrations in the new book,
which show multiple control systems sharing a common environment which
they all sense and affect, yet continue to control independent aspects of
that environment. The phenomenon you describe is just the tip of the
iceberg.

Besides, it is exciting to see
the idea “Awareness is mobile”. I

recognize the awareness as a “rolling ice ball floating in the
water”;

the part beneath the water surface is what we are unaware of. It’s

nothing like Freud’s iceberg metaphor as his “iceberg” never
roll. If

there is something hard to be awared of, it is not included in this

ball. I am not quite sure about what kind of “forces” make the
ball

roll; it could be something like attention.

Great minds think in the same channels, right? I can add to your rolling
iceberg a balloon overhead. Awareness tends to dwell in the middle levels
of the hierarchy, and not so much in the lower or upper levels. If
I asked you to tell me right now what goals are behind the fact that
you’re reading this post at this instant, the first thing you’d have to
do would be to stop reading. Then you would have to look inward, and
goals would come to your awareness, and you would tell me. But you
couldn’t tell me the answer until you had moved your awareness away from
the main thing you were consciously doing. The higher goals and control
systems were obviously operational all the time, but you weren’t
conscious of them.

ps:

Here is another interesting example about the controller in robots.
In

a summary of evolutionary robotics:


http://www.mae.cornell.edu/ccsl/papers/Biomimetics05_Lipson.pdf

Professor Lipson described an experiment in the section
"Evolving

Controllers" on page 4. In Figure 1(b), there are two
“mysterious”

nodes B1 and B2 without any explanation in the related text, which
is

filled up with the words like “mapping inputs to outputs”. In
the

reference "Evolved Sensor Fusion and Dissociation in an
Embodied

Agent" about the original work


http://www.cs.uvm.edu/~jbongard/papers/BongardWGW02.pdf

there is a single sentence mentioning these two:

"There is an additional bias neuron at the input and hidden
layers

that outputs a constant signal of 1."

In this experiment, 200 candidate controllers were evolved for 50

generations, and the Quadrapod succeeded in moving towards high

concentration of simulated chemicals at last. I don’t know how
output

weights of B1 and B2 evolve in the simulation, but I guess their

outputs are recognized as reference signals and negative feedback

emerged.

I’ll read those papers later. It’s interesting that in all of these
“genetic algorithm” models that I have seen, the designers have
to let the evolving organism reproduce even though it hasn’t yet learned
how to do what is necessary to survive. In your last quote above, we see
that 50 generations had to pass before the Quadropod “succeeded in
moving towards high concentration of simulated chemicals at last.”
If it didn’t actually reach the chemicals, how did it survive through 49
generations? It survived because the kindly programmer saw it was
changing in the right direction, and allowed it to survive. There is no
kindly programmer helping real organisms that way.
In PCT we use a different basic principle, that of reorganization.
Reorganization depends on sensing some critical variable and detecting
whether it is changing toward or away from some specific reference level.
Changes in system’s properties – the coefficients in the equations that
define it – are always going on at different rates; those different
rates are like E. coli swimming in a straight line in some hyperspace of
coefficients. If the error signal is decreasing, those changes continue.
If the error signal ever increases, however, this reorganizing system
“tumbles” by altering the rates of change of all the
coefficients, at random. The random tumbles continue until the error is
once again decreasing. The system now swims in a new direction through
the coefficient hyperspace. To make the system converge to a final result
properly, the speed of swimming is determined by the total squared error
in all the critical variables. As the error decreases, the changes become
slower. They may stop at some minimum but nonzero error level.
You knew all that, but I thought I’d just say it again. You can see that
the genetic algorithm programmers have had to do this too – they are
serving as the comparators in the reorganizing system, because they can
see when the changes are leading toward the goal or away from it. Their
system tries to work by “rewarding” behavior that leads toward
a favorable result, which is the idea behind reinforcement theory
(though Skinner’s version demands that the favorable result actually
occurs). In reorganization theory, the organization of behavior is
altered only when the result of behavior is actually
unfavorable.
In all theories of learning, it turns out that something has to detect
whether the changes have more-favorable or more-unfavorable results. If
that’s not done, survival becomes far too unlikely. The difficulties
arise when we ask how an organism, or the environment, can know that a
change took the organism closer to a favorable result, when that result
didn’t actually occur. Reorganization theory offers a solution for that
problem that doesn’t involve a helpful programmer.
Best,
Bill P.
P.S. Please tell us which is your family name and which is your personal
name. It’s very confusing because Chinese writers, being friendly and
polite people, sometimes switch the two names around and put the personal
name first so we Westerners (who live to the east of you) will know which
is the “first name”. But of course, some Chinese writers
don’t do that, so we never know whether to switch the names back
or not.

In the English-speaking American navy, I would write my name on forms or
speak it to superior officers as “Powers, William, 7275198.”
The comma, or a little drop in tone and a pause when speaking, indicate
that the last name is being given first (to please clerks who like to
list people alphabetically by their family names). So ALL YOU GUYS OVER
THERE, how about simply writing your names the way you like to see them,
and putting a comma after the family name, as in

“Wang, Bo” – or is it “Bo, Wang”? If you want to
write it our way, just leave out the comma. You can send the Nobel Peace
Prize to my current address.

···

At 08:14 PM 12/24/2008 +0800, Wang Bo wrote:

From Bill Powers (2008.12.27.1837 MST)]

In "The Blind Watchmaker", Dr. Dawkins used an "evolutionary" program to
guess a line in Shakespeare's script. Though it did succeed, it seems
that the program know the result at the beginning, for its fitness
function is defined as how many different characters there are between
the generated sentence and Shakespeare's words.

That's a good example. The real problem is to find a method that will work without a teacher or any external help.

Accordingly, I prefer
evolving fitness functions, as they represent the ecological niche,
which is ever changing.

Some things don't change so fast: the need for food, water, oxygen, protection from extreme cold or extreme heat. The "intrinsic reference levels" of reorganization theory, in PCT, are meant to be the kinds of goals that evolution can create, and which can quide further reorganization.

I'm aware of [don't need the "of" here] that the fitness functions are relatively static during a very short period in the history of
evolution.

Here is another stream in the evolutionary computation: people use a
simulated ecological system instead of a single population to evolve
the neural network, and the fitness function was replaced by a single
criteria "dead or alive". The following presentation "Polyworld: Using
Evolution to Design Artificial Intelligence" gave a remarkable
example.
http://www.youtube.com/watch?v=_m97_kL4ox0
The video is a bit lengthy, over an hour; you may watch it if you got time.

Later, maybe. The nervous young man makes me jittery just watching him.

In a space of adjustable parameters, from my perspective, the task of
learning is to diverge from the old point and converge to a new one;
the new point is commonly better according to some metrics.

Yes, that's the basic principle of reorganization theory.

Reorganization theory explains the divergence and convergence within
the same framework. For example, it gives punishments, which only make
the parameters diverge, a position, while the reinforcement learning
don't. Moreover, to improve the learning, we may need a method to bias
the space; I brought in attention months ago for this purpose.

  There's no "punishment" in reorganization theory. What happens is that a control system fails in some way so that intrinsic error increases. That is the signal to try some new variation in the parameters, and the simplest and most powerful kind of variation is random -- the E. coli principle. Between reorganizations there are various kinds of systematic change going on -- the parameters are changing, each at its own speed and in its own direction. This slowly changes the intrinsic error, initially decreasing it (because increases lead immediately to a "tumble", changing the direction in which the parameters are changing). When the changes bring the system as close as possible to a state of minimum intrinsic error, the next changes cause the error to start increasing and another change of direction takes place. Again we get one tumble after another until the error begins decreasing again and the parameters continue to change in the new directions. As the error gets smaller the speed of change of the parameters decreases.

This arrangement guarantees convergence of the parameters toward the values that will produce the least possible intrinsic error. It's not efficient, but it's much more efficient than simply changing the parameters at random, in big jumps.

In reinforcement theory, we have a positive feedback loop. A behavior that increases reinforcement causes changes that make the behavior happen more often or more strongly. That causes more reinforcement, which increases the behavior even more, and so on until a limit is reached.

Unfortunately, reinforcement theory requires that in order to produce a desired effect (the reinforcers), the system must learn a particular behavior, and in the real world, repeating a behavior does not repeat its effects, because of disturbances and small random variations in the system's own parameters. So reinforcement theory, as described by behaviorists, cannot work in the real world.

Another problem is how to add dimensions to this space, say how to
increase the complexity of the system, and this is what the evolution
should do, at the phylogenetic scale.

I agree about that. I have had 500 control systems, acting on a common environment of 500 variables, all reorganizing at the same time until each control system can independently make its own perceptual signal (a weighted sum of all 500 environmental variables) match any arbitrary reference value independently of any of the other systems. So achieving sufficient complexity is not a problem. Perhaps getting evolution to demand it is the problem. But the world is complex enough to provide enough problems to solve, I think.

Best,

Bill P.

···

At 09:21 PM 12/25/2008 +0800, Wang Bo wrote:

From Bill Powers (2008.12.27.1837 MST)]

Accordingly, I prefer
evolving fitness functions, as they represent the ecological niche,
which is ever changing.

Some things don't change so fast: the need for food, water, oxygen,
protection from extreme cold or extreme heat. The "intrinsic reference
levels" of reorganization theory, in PCT, are meant to be the kinds of goals
that evolution can create, and which can quide further reorganization.

Exactly. The evolution has been "trying to find" appropriate positions
for every organisms, and intrinsic references, from my perspective,
are the record of these positions (niches).

I'm aware of [don't need the "of" here] that the fitness functions are
relatively static during a very short period in the history of
evolution.

Thanks for pointing out the problem with my grammar.

There's no "punishment" in reorganization theory. What happens is that a
control system fails in some way so that intrinsic error increases. That is
the signal to try some new variation in the parameters, and the simplest and
most powerful kind of variation is random -- the E. coli principle. Between
reorganizations there are various kinds of systematic change going on -- the
parameters are changing, each at its own speed and in its own direction.
This slowly changes the intrinsic error, initially decreasing it (because
increases lead immediately to a "tumble", changing the direction in which
the parameters are changing). When the changes bring the system as close as
possible to a state of minimum intrinsic error, the next changes cause the
error to start increasing and another change of direction takes place. Again
we get one tumble after another until the error begins decreasing again and
the parameters continue to change in the new directions. As the error gets
smaller the speed of change of the parameters decreases.

I was trying to explain the effect of punishment; I thought it causes
the control system fails in some way, hence the reorganization system
make the parameters diverge from the old point. I really appreciate
the consistency in your theory. Regarded as a behavior in intrinsic
environment, learning could be explained by control theory just like
other behaviors in extrinsic environment.

I agree about that. I have had 500 control systems, acting on a common
environment of 500 variables, all reorganizing at the same time until each
control system can independently make its own perceptual signal (a weighted
sum of all 500 environmental variables) match any arbitrary reference value
independently of any of the other systems. So achieving sufficient
complexity is not a problem. Perhaps getting evolution to demand it is the
problem. But the world is complex enough to provide enough problems to
solve, I think.

Magnificent! So far as I know, the question about increasing level of
complexity is a central one in today's evolutionary robotics,
artificial life, and evolutionary biology. I prefer the idea that this
trend towards complexity is passive. Nothing is judging the E.Coli,
who cannot undertake complex task. From my perspective, it is the
paradigms, like the hierarchy of control systems, that emerged in the
hereditary representations provide the opportunity to achieve higher
level of complexity. Mutations on these paradigms can change the
structure of organisms relatively easier. Hox genes, encoding the
morphology, are widely spotted in a vast number of animals, from
Drosophila to Human Beings. In experiments, some mutations would give
the fruit fly an extra pair of wings.

I'm not a geneticist, but I guess there is some kind of genes, like
hox or within the hox, in which the hierarchy of control systems is
encoded. Just like giving the extra pair of wings, some mutations on
this paradigms would easily alter the structure of neural control
systems. Though this alteration might lead to different levels
complexity at the same time, higher ones would be preserved and passed
on if it did bring more chances to survive.

Happy New Year To All!

Bo

···

2008/12/28 Bill Powers <powers_w@frontier.net>: