···
-------------
From Tom Bourbon [940215.0910]
This post addresses a few points raised in an exchange between
Bill Powers and Martin Taylor. My comments are directed toward
Martin.
It began with:
[From Bill Powers (940210.1100 MST)]
Subject: Misc remarks on backlog
It continued with:
<Martin Taylor 940211 16:20>
Bill Powers (940210.1100)
Subject: Re: Making it go
In his post, Bill described the following (beautiful!) incident:
"When my son Denny, who is now a practicing mechanical engineer,
was 7 or 8 years old, he announced that he was going to design a
spaceship. After much work, he showed me some drawings. Here is
the nose which has the radios in it. Just behind is the control
room with seats and a wheel to steer with. Behind that is a place
to sleep and a galley to prepare food. And in the very back is
the thing that makes it go."
Bill then expressed concern that invocations of "things that make
it go" have become more frequent in discussions on this net --
discussions in which people talk of "better" or "improved" PCT
models. One idea that *sometimes* takes on the appearance of a
"thing that makes it go" is an assumed model of the world that
allegedly "makes the PCT model go." I am not questioning the
possibility that model-based control might be useful or that it
might happen; my comments are directed toward some specific points
in recent posts to this net.
Bill P.
"What does the model actually do in relation to perception, such
that control in a real environment occurs?"
Martin T.
"In Hans's arrangement, which seems quite natural to me, the
output of the world model IS the perception that is used in
control. But the value of that perception is always updated
using the output of the normal PIF using incoming sensory data.
Any discrepancy between the current perceptual signal from the
PIF and the output of the world model is used to update the world
model, so that subsequent world model outputs will be more likely
to match the actual sensed signal, apart from the effects of
disturbing variables on the CEV."
Tom B., now:
The model you refer to, Martin (after Hans), is shown below. It
appeared in the post <Martin Taylor 940207 11:45>. This is the
model you said I also proposed, a claim that I rejected (Tom Bourbon
[940210.1214]) in a post titled "Not me!"
Clearly, the model in this diagram does not match the model
you described in the more recent passage quoted above. In the
recent post, you said that in the model Hans and you favor,
"the output of the world model IS the perception that is used in
control." However, in your diagram of Hans's model, the
signal from the *perceptual input function* is the signal that is
used in control -- the vertically routed path from the PIF enters
the same comparator as the reference signal for the outer loop,
the loop you call the standard ECS. In the diagram, the
other path followed by the regular perceptual signal is into a
comparator it shares with the output of the world model.
Unlike the output of the PIF, in your diagram the output of
the world model *does not* enter the same comparator as the
reference signal for the outer loop, therefore it *cannot be*, ".
. . the perception that is used in control." Instead, the world
model occupies an interior loop insulated in every way from the
*functioning* of the outer ECS. Hans and you refer to the
inner loop as an "imagination loop," with its output as THE
object of control. Perhaps it is, but not as diagrammed here; in
the version yo posted, I think the inner loop might be
more accurately characterized as an "idle thought loop," or
perhaps a "fantasy loop," (no insult intended, Martin) running
harmlessly and ineffectually in the background while the
unembellished outer loop produces control. Was there a mistake
in the diagram, Martin? Or am I misreading you?
Your diagram of Hans's model follows:
<Martin Taylor 940207 11:45> [a post titled "Mostly IT and
PCT."]
Here's a diagram.
>
V
>
----------O--------------------
> >
^ V
perceptual output
input -->--O------->---- function
function | | |
> > correction |
> ^ signal |
> > > >
^ | V |
> > world |
> imagined <-----model-------|
> perception function |
> >
> >
==========^===============================V=============== >
> >
> >
CEV<---world transfer function-<-
> >
^ ^
> > "ratio" disturbance
disturbance | (equivalent changes in the WTF) >
The outer loop is a standard ECS. But there is an imagination
connection that I have labelled "world model function." The
output of this function is a signal that would match the
perception if there were no external disturbances, and if the
function correctly matched the "world transfer function"and the
PIF. This function is updated according to the discrepancy
between the imagined perception and the actual perception.
Tom B., now.
On re-examination, I see that the diagram *does* match your
*original* description, quoted in the passage just below
the figure. In that passage, you seem to describe a
*pure* imagination mode -- one that runs completely independently
from the actual controlling loop. Is that what you intended, or
do I misread you? Your description here does not seem to match
the one from your more recent post that I quoted and discussed a
few lines back -- the post in which you said the output of the
world model IS the perceptual signal that is controlled by a PCT
system.
For it to match your more recent description, in which the output of
the world model is the controlled perceptual signal for the outer
loop, I believe your diagram would necessarily assume the
following form (I'll leave out the "real world" portion):
p* = reference signal for p(m)
>
\|/ e
,----->C--------------------,
> >
> >
> p(m) world |
split p(m) ,---+---------- model <---------|
> function |
\|/ /|\ |
C-----------------' |
/|\ correction |
> signal |
> p(r) |
> \|/
perceptual output
input function
function |
/|\ |
> \|/
========================================================== >
"REAL WORLD"
where p(m) = perceptual signal (output signal) from world model;
p(r) = "real" perceptual signal out of PIF; C = comparator;
e = error signal.
This diagram seems to include the relationships required to match
your more recent *description* of a self-adapting control system
The world model function receives two signals; the error signal from the
outer loop and the perceptual signal from the outer perceptual input
function. Also, p(m), the output of the world model, goes to the
comparator for p*, the reference signal of the outer loop; but that
output also goes to a comparator for p(r), the "real" perceptual signal,
with the difference between the two perceptual signals serving as a
"correction signal" to the world model function. Is that the
relationship you described, but did not diagram, Martin? If it is, I have
a few questions.
1. Do you assume that this organization applies at all levels of the
perceptual hierarchy, or to only some levels? If not to all levels, then
to which ones -- higher, lower, some of each?
2. Do you know of any neuroanatomical evidence that resembles this
organization? Is such an organization found in the nervous system of any
species?
3. What happens inside the "world model function?" What happens to the
error signal that comes in from the outer loop, and to the perceptual
signal that comes in from the perceptual input function of the outer
loop? What is the source of p(m)?
4. Will you "talk us around the loop(s)?"
On a different topic, in your more recent post you ostensibly quoted Bill
Powers, but you modified his original wording, giving me the impression
that you believe his discussion of "the thing that makes it go" does not
apply to Hans's model, or to your interpretation of Hans's model. Compare
the following two passages:
[From Bill Powers (940210.1100 MST)]
"If you look at our working models of behavior, you will find that
there is no "thing that makes it go" in the model. The model
works by itself; it is complete, if not necessarily correct. When
you start the program, the model goes to work and produces
outputs that match real behavior, with no external help and with
no arm-waving. There is no controller in a control system: the
whole control system IS the controller."
From <Martin Taylor 940211 16:20>
If you look at [a self tuning world model], you will find that
there is no "thing that makes it go" in the model. The model
works by itself; it is complete, if not necessarily correct.
When you start the program, the model goes to work and produces
outputs that match [the behavior of the world], with no
external help and with no arm-waving.
========================================
Tom B., now:
Martin, you are claiming the following equivalences between Bill's
remarks and yours:
our working models of behavior = a self tuning world model
real behavior = the behavior of the world
Perhaps you are right and the phrases can be interchanged; but I
am not yet convinced. Your claim will be corroborated when you
present data showing better performance by a self-tuning world model
than by the simplest PCT model. Which version of the self-tuning model
that might be, I cannot discern in your recent posts where it appears
to me that your descriptions and diagrams of self-tuning models do not
match. In any case, I feel certain the self-tuning model you employ
will not be the same as the one I described in my earlier exchanges with
Hans, the one I mentioned in, "Not me!"
Until later,
P.S. On the subject of diagrams and my version of an adaptive PCT model,
in "Not me!" I said I would not post a diagram of my model, unless
someone requested it. Someone did. In addition, given Martin's mistaken
impression that my model used a model of the world, it might be worth
reproducing the figure here. (I know, Gary Cziko, I always use too many
figures. Them's the breaks.)
===================================
<[Bill Leach 940212.11:49 EST(EDT)]
Tom Bourbon [940210.1214]
I'll not include my diagram, unless you, or another reader, thinks that
would help.
It might well help Tom.
Here it is. I was modeling an interactive tracking task, in which Person A
kept a disturbed cursor aligned with a target. Actions by Person B also
affected the cursor controlled by A. B attempted to "make" A's actions
follow a particular time sequence (/\/\/\/\). Remember, Person B affects
only A's cursor. When B experiences an error signal (A's handle position
does not match the one B desires) B must move handle B, to affect
cursor A -- not "too little," or "too much," but "enough." That means
the gain on B's output function must be "good enough." To make B an
adaptive controller, I gave it an adaptive loop that monitored the error
signal in B's primary loop and when the time integral of that signal went
"out of bounds," the adaptor loop gave out a random change (dk) in the
magnitude of the gain factor (k) in the output function of B's primary
controlling loop -- the loop that affects A's cursor.
No "world model," but effective adaptive control. Not me, Martin!
If you need more information, Bill L., I'll send it to you directly --
unless someone asks to see it on the net.
=================================
From Tom Bourbon [930902.0900]
. . .
As modified for my tracking tasks, the "adaptor loop" has a perceptual
function that computes the present change in the error signal in the
primary control loop in model B. In the adaptor loop, the computational
steps, which approximate the time integral of error, (1) calculate the
present CHANGE in the error signal, (2) add that change to the previous
total of changes, (3) determine if the new total > a criterion value, (4)
if the new total > the criterion, re-set the total to zero and produce a
random change in "something." In their chemotaxis paper, the change was
to a randomly selected new direction of travel for the simulated E. coli
bacterium; here, I made it a randomly-selected signed magnitude of change
in the integration factor, k, in the Output function of the primary
control loop in model B. Said another way, in B, when error in the loop
controlling A relative to /\/\/\/\ integrated to a criterion value, a
change, of random magnitude and sign, was added to the integration factor
in the Output function.
The . . . diagram, including the adaptor loop in model B:
PCT MODEL OF PERSON B PCT MODEL OF PERSON A
r = A-[/\/\]=0 r = [c-t=0]
> >
> r=[if q > Q, |
> > dk = rand dk] |
> > >
> \|/ |
> >---|---|---| |
> > I | C | O | |
> >---|---|---| |
\|/ /|\ | \|/
>-----| | dk |-----|
present ,->| C |--*' | present ,->| C |--,
value of | |-----| | | value | |-----| |
A-/\/\ = p e = p-r | of c-t = p e = p-r
> \|/ | | \|/
>-----| |-----| | |-----| |-----|
> I | | O |<----' | I | | O |
>-----| |-----| |-----| |-----|
/|\ | /|\ /|\ |
> > > > >
> > t | |
> > /|\ | |
> > T | |
> > > >
> '----> B ---------------> c <-- A <-'
> /|\ |
> D |
> >
'-------------------------------------------'
In the adaptor loop, q = present sum (integral) of the error signal, e;
dk = random change added to k (in O). B has become an adaptive
controller. In the runs I did, now a couple of years ago, B converged
within a few seconds on a value for k that kept A-/\/\/\ near zero, and A
kept c-t=0. I have not run the model since that time.
Now some of the differences between this particular adaptive PCT model,
and the model described by Hans seen even clearer to me. For one thing,
the equivalent of what Hans called the "second controller" in the
adaptive controller has no model for A and there is no expectation of
what A will do; rather, the "second loop" has a reference signal for the
value of the time integral of sensed error being < a criterion value, and
an output function that adds a random change to the integration factor of
the primary control loop. There is no "optimal" value of anything -- any
k that keeps the integral of sensed error below the criterion value will
"stick." For this loop, anything that is "in the ballpark" is good
enough. This implementation of the E. coli procedure in the PCT model
struck me then, as it does now in recollection, as a powerful
demonstration of random variation and selective retention as a possible
mechanism for learning (and as some on the net have speculated, for
evolutionary, or phylogenetic, change).
The second loop "knows" only the magnitude of the integral of sensed
error, but in this implementation, it did not, and need not, know that to
which the error pertains -- it doesn't need to know where the error comes
from or why it is there, and it does not know "what it is doing" when it
outputs a randomly selected value. The second loop is not modeled as
"intentionally" modifying the gain if another system, or as controlling
the actions of yet another system.