Modeling Reorganization

[Abbott_Bruce_Indiana]

[From Bruce Abbott (941228.1255 EST)]

Rick Marken (941227.1115) --

Rick, I've already replied to Bill on the issues you raise concerning my
"evolution" model, so I won't repeat myself here. But I would like to raise
another issue, which is why it is that you were not able to grasp the
essential nature of the model by reading the program's code. It's really not
difficult--all the necessary information about what the program is doing is
contained in the (very short) main loop, and the variables have been given
names that make it clear what they represent.

This troubles me, because the program is supposed to be the arbiter of any
disputes as to what the model actually is--it's supposed to short-circuit
these endless verbal tangles. You should have been able to examine the code
and tell me whether the model provides an analog of the reorganization system-
-as I said it did--even if you disagreed with my description of the model as
selection by consequences. I can understand getting lost in the ECOLI4a code-
-its logic is by no means easy to comprehend--but the evolution program's
logic is simple.

What happened instead is that you started out by agreeing that the model was
an example of selection BY consequences and then arguing with me that it could
NOT be an analog of the reorganization system, because, you said, selection BY
consequences cannot result in control, and reorganization is a control system.
If even the explicit presentation of program code cannot settle the question
of what a model actually represents, why bother with it at all? Why not just
default to verbal arguments and let it go at that?

I agree with your description (and diagram) of the evolution model but--and I
have to emphasize this again--I think it's just a matter of what you wish to
emphasize about the operation of the model. For a given reference, what
happens next depends on the outcome of the previous round of control. In the
comparator function, the reference AND the perceptual input jointly determine
the error. Thus, for a given reference, the joint effect of output and
environment on the perceptual signal determine what happens next; for a given
perceptual input, the value of the reference does so. To understand why the
model behaves as it does, you must know both.

At least I now have your agreement that it the model does encode a form of
reorganization system. With slight modification it could be converted to a
model in which a single organism reorganizes its own system parameters when
the lower-level system fails to achieve control over its perceptual variable.
The next question is, would such a system usually achieve control before the
organism expired (i.e., is the system efficient enough to be practical?).

One other point--a question, really. You have stated that the ECOLI4a model
does not learn because it only changes the parameters of an existing control
system, rather than creating a new one. By your definition, the kind of
reorganization proposed in the evolution model would not qualify as learning
either, for the same reason--it just "finds" workable parameters for an
existing control system. Do you agree? If so, how would reorganization have
to differ for it to qualify as a form of learning?

Regards,

Bruce

P.S. I'd appreciate a copy of the "blind men" paper, which you offered to
send on request. I want to read about the blind man who says "An elephant is
like a set of perceptual control systems...". (;->

[Richard_Marken]

[From Rick Marken (941228.2230)]

Bruce Abbott (941228.1255 EST)--

Rick...why it is that you were not able to grasp the essential nature
of the model by reading the program's code.

I confess that I didn't read the code that carefully because I can't run it.
But I'm not sure I would have noticed the control structure based on just
the code anyway; your diagram helped me see that there was a variable that
functioned as a perceptual variable; and that there was an implicit
reference specification for this variable. This was hard to see in the
code because the comparison between reference and perceptual variable was
never explicitly made; the perceptual variable was treated as the error
variable so the reference signal and comparison process was implicit.

This troubles me, because the program is supposed to be the arbiter of any
disputes as to what the model actually is--it's supposed to short-circuit
these endless verbal tangles. You should have been able to examine the code
and tell me whether the model provides an analog of the reorganization system-
-as I said it did--even if you disagreed with my description of the model as
selection by consequences. I can understand getting lost in the ECOLI4a code-
-its logic is by no means easy to comprehend--but the evolution program's
logic is simple.

The code helps enormously and I think we can get to agreement on it (as I
think we basically have on your "evolutionary" selection model) far
better than we could if we had just words. But there will be difficulties.
I mentioned one; the way you coded the model makes the reference AND the
comparison process implicit. In our control models we try to be very
explicit about how we assign program code to control loop variables and
functions. But once we figure there things out the code helps a lot and
a WORKING model helps even more.

What happened instead is that you started out by agreeing that the model was
an example of selection BY consequences and then arguing with me that it could
NOT be an analog of the reorganization system, because, you said, selection BY
consequences cannot result in control, and reorganization is a control system.

Sorry, I was confused. I took your word that your program was an evolutionary
model, implying no reference for any particular consequence. If this were
true, it would have satisfied my (and S. J. Gould's) notion of a "selection
by consequences" process. Such a selection by consequences model would,
indeed, NOT have been a reorganization model because it would not
have been a control system. But it turns out that it IS a control system
model; it is NOT a selection by consequences model, as you claimed and I
assumed was true. In fact, the program senses the state of each system,
implcitly compares the sensed state (perception) to a reference for
surviving and acts (revises system parameters) if the sensed state of a
system is "dead". If you consider a system "non-functional" rather than
"dead" then your model is a perfectly reasonable reorganization system.
It is definitely NOT a model of evolution, by the way, unless you think
there is some entity in the environment (the equivalent of your "program")
that is controlling for organisms that "survive".

If even the explicit presentation of program code cannot settle the question
of what a model actually represents, why bother with it at all? Why not just
default to verbal arguments and let it go at that?

Because we do get to agreement, eventually, with the model.

I agree with your description (and diagram) of the evolution model but--and I
have to emphasize this again--I think it's just a matter of what you wish to
emphasize about the operation of the model. For a given reference, what
happens next depends on the outcome of the previous round of control. In the
comparator function, the reference AND the perceptual input jointly determine
the error. Thus, for a given reference, the joint effect of output and
environment on the perceptual signal determine what happens next; for a given
perceptual input, the value of the reference does so. To understand why the
model behaves as it does, you must know both.

See my earlier post today to see why this is not true. I see that this
will be a VERY tough nut to crack. My advice is to keep chanting "the
percpetual signal is CONTROLLED; the perceptual signal is CONTROLLED"
over and over again while you prepare the apparatus for doing some
real PCT research;-)

At least I now have your agreement that it the model does encode a form of
reorganization system.

Yes. Because it is a control system; and because it is a control system
there is no "selection by consequences".

One other point--a question, really. You have stated that the ECOLI4a model
does not learn because it only changes the parameters of an existing control
system, rather than creating a new one.

This was not my main beef with the ECOLI4a model. I am happy to call the
parameter changes "learning" if you like; it's just that the parameter
changes are the result of environmental changes; the result of this
"learning" is an organism that can no longer control in an environment
in which it could still be controlling if "learning" had not occurred.
The "learning" issue is obscuring what ECOLI4a shows very nicely;
"selection by consequences" deprives a control system of the ability to
control.

By your definition, the kind of
reorganization proposed in the evolution model would not qualify as learning
either, for the same reason--it just "finds" workable parameters for an
existing control system. Do you agree?

No. The parameters of the control systems in your "evolution" model are
not "selected" in the same way as the parameters in the ECOLI4a model.
In the latter model, the values of the parameters were directly determined
by consequences (rewards and punishments); in the former, the values
of the parameters are determined by a random number generator; it is a
consequence of the behavior that results from these parameters (survival
or death) that determines whether the parameters are changed or not; but
there is no direct effect of consequences (the gradients experienced by the
bugs themselves or the survival or non-survival of the bugs themselves)
that determines the values of these parameters. The bugs in your
reorganization program are doing what I call "real learning".

P.S. I'd appreciate a copy of the "blind men" paper, which you offered to
send on request. I want to read about the blind man who says "An elephant is
like a set of perceptual control systems...". (;->

Tell me where to send it and I'll try to find something that feels like
a mail box;-)

Best

Rick