[From Bill Powers (941211.0830 MST)]
Bruce Abbott (941210.1330 EST) --
... we now seem to be saying that the relationships captured by the
empirical law of effect are in fact nicely explained by the
reorganization concept. Fine, I agree with you. Who's on first? What
have we been arguing about? Why?
Well, Lou, it's the same old problem: getting ideas from one head to
another through the medium of language, which ain't telepathy. We're at
the point now where you say "now we're getting somewhere" and I'm about
to say "Yes, now we understand the essential difference between the Law
of Effect and reorganization theory."
What I've been saying all along (I thought) was that the consequences
of behavior (and by this I always understood them in terms of ultimate
effect on controlled perceptions) provided the selection criteria that
determine which behaviors are retained in the output function and which
are not.
But the law of effect says more than that. It says that a behavior is
selected and retained on the basis of _previous_ effects of the behavior
on the consequence. So you end up with a logical process that goes "The
last time I got what I wanted I was doing x, so if I want it again, I
should do x again." This is surely an algorithm for choosing among
behaviors x, y, z, ... on the basis of past experience, but is it not
reorganization: it is logic. A system that can't reason in syllogisms
can't do this kind of learning.
To "select" behaviors means to select _among_ behaviors. This means
knowing that there is a variety of behaviors that could be produced, and
remembering associations between each one and the consequences it has
produced in the past. It means comparing the outcomes of behavior in
imagination with the outcome that is currently desired, and finding the
behavior which, if carried out, would produce the desired consequence.
If you were going to build a device capable of learning in this way, you
would have to build into it the ability to remember past experience and
apply at least elementary logic to what is remembered. That, in fact, is
what Ecoli4 does: It can perceive the current dNut, remember the past
value of dNut, compute the change of dNut that took place across the
previous tumble, and reason that if the change was positive and the
previous value was positive, the probability of a tumble for positive
dNut should be increased, and so forth through the other three
combinations. All of this requires perception, memory, the ability to
carry out elementary logical reasoning, and the ability to alter a
property of the system -- a probability -- in a way consistent with the
conclusion.
This algorithm will work in those circumstances where past experience is
a guide to present behavior. It's a rather advanced form of learning,
and I could argue that it's not really learning at all, since when the
facts are known, the outcome is a foregone conclusion and can be
generated by a fixed process.
In contrast, reorganization makes no use of past experience at all. It
does not remember what happened the last time a behavior was used, and
it does not require that a known association exist between specific
behaviors and specific outcomes. It is driven strictly by the difference
between the actual outcome of a behavior and the desired outcome. If
there is a match, no change takes place. If there is an error, a change
takes place after a time inversely proportional to the error. There is
no memory, no logic, and no choosing among alternative behaviors.
I don't think you have recognized the implications of your Ecoli4 model.
As a good modeler (and you are definitely a good modeler), you were able
to break down the Law of Effect into the operations that the E. coli
system, if it could learn, would have to be able to perform in order to
make this law work. You got all the essential elements into it: memory,
comparisons, reasoning, and changes in the probability of a tumble. But
I think you were looking at this part of the program as if it were
separate from the rest -- as if it were part of an abstract description
of the phenomenon. In fact you deduced, correctly I think, what
_mechanisms_ would be required inside any system that behaves according
to the Law of Effect. By simulating those mechanisms in the model, you
produced a model that worked according to the Law of Effect.
There is more to a model than the overall effect it produces. We agree
that what our model should do is to end up generating those behaviors
that will create the desired consequence. But the fact that two
different models succeed in creating the same effect doesn't mean that
the models are equivalent. The non-equivalence is brought out by
changing the conditions under which the model must work. If one model
continues to produce the desired result and the other doesn't, we have a
basis for choosing between the models.
That was the point in my offering a ridiculously simple model, which
said that if dNut was positive, decrease PTS+, and if dNut was negative,
increase PTS-. That is not a Law of Effect model (or a reorganization
model), but it produces the same result that your Ecoli4 model produces,
and moreover continues to produce it even when the circumstances are
changed so that past experience no longer can be used as a guide.
I think of reorganization as the process by which we can learn new means
of control when past experience is of no use, and when the organism in
question couldn't understand hints from past experience even if they
existed. Before logic or any other kinds of symbolic computations can be
used, something must create the ability to do symbolic computations.
Whatever that something is, it clearly can't make use of symbolic
computations in its operation. In fact, the reorganizing process can't
make use of _any_ of the levels of perception and control, because it
has to work before they are brought into existence. This means that it
can't use any orderly method that we would recognize: ergo, it must be
what we call a "random" process. To suppose otherwise would be to open
the door to infinite regress or the all-knowing homunculus.
ยทยทยท
------------------------------------
There's another Who's on First problem. In a previous post you indicated
that you didn't see any difference between selecting a behavior and
altering a parameter of the system. The difference is in how you think
of behavior in relation to its consequences.
One way is this
B1 ---> C1
B1 ---> C2
...
Bn ... Cn
Here we have strictly an association. A particular behavioral event goes
with a particular consequence-event. If you produce behavioral event B2,
you will get consequence-event C2. If you want a different consequence,
you have to produce a different behavior.
But another way is this:
B = k(C* - C)
C = f(B) + D
where C* and D are both functions of time.
Now B and C don't represent any particular events; they are variables
which can take on a range of values. However, these values remain in
certain relationships to one another, described by the system of two
equations.
Reorganization doesn't specify any particular values of B and C.
Instead, it works to alter the parameter k and the form of the function
f. For any particular value of k and form of f, variations in D and C*
will result in corresponding changes in B and C. The outcome of
reorganization is not any particular state of B and C, but a particular
relationship between these variables and the independent variables C*
and D.
When k and f are within a broad range of possibilities, we will have a
control system. The result of the process of control will be,
approximately, that
C = C* and
deltaB = -deltaD
Now we have behavior B varying independently of the consequence because
of variations in D, and the consequence C being held near a reference
state C* despite the variations in D. It's clear now that there will be
an association of particular values of B with particular values of C
_only when D is constant_.
A reorganizing system attached to this system would sense the error
signal, which is the difference C* - C. It would randomly alter k and
the coefficients that determine the form of f. These random variations
would become smaller and less frequent as the average absolute error
signal approaches zero. In the end, k and f would cease to change, and
the error would be as small as possible for ALL values of D and C*. The
main feature of the E. coli style of reorganization is that there is in
fact a strong tendency to converge toward a state of zero error.
But notice that this does not imply any fixed behaviors or fixed
consequences. What has become organized is a control system, not an
association between specific behaviors and specific consequences. The
criterion for cessation of reorganization is not that any particular
behavior or consequence occur, but that the error signal be as small as
possible for ALL values of C* and D.
Because of variations in D, any consequence within the range of C might
be produced by any behavior within the range of B. So past experience
with B-C pairs is useless. Yet a control system can bring about and
maintain any particular value of C even though doing so requires
producing widely varying values of B.
The Law of Effect solution works only when C* and D are constants.
Experiments done under that paradigm are almost always set up to make
sure that at least D is a constant and preferably zero, and that the
desired consequence C* is one with a predictable value.
So now we have What on second and Otherguy on third.
-------------------------------------
What I've been saying all along (I thought) was that the consequences
of behavior (and by this I always understood them in terms of ultimate
effect on controlled perceptions) provided the selection criteria that
determine which behaviors are retained in the output function and which
are not.
But this isn't right, is it? Even without disturbances, the consequence
of a behavior is simply whatever it is, and the resulting perception is
whatever it is. There is nothing in the consequence or the perception of
it that carries a label saying "desired" or "undesired." The criterion
is determined and represented by the reference signal, which doesn't
depend on the consequence. If the consequence of moving the point of aim
to the left is a dart planted one foot to the left of the bullseye, the
perception simply reports "one foot to the left of the bullseye." It
doesn't say whether that is good or bad. If you're trying to hit a spot
one foot to the left of the bullseye, which you may well be trying to do
in a pub contest, that is good. The value of a given consequence isn't
contained either in the consequence or in a perception of it, but in the
reference signal that defines the state of the perception that is
wanted. The desired value is set by C*, not by C.
------------------------------------------------------------------------
Rick Marken and Martin Taylor --
After a husband has been rejected by his wife enough times, he concludes
quite correctly that his wife is a terrible person who continuously
rejects him. If the wife then undergoes therapy and becomes accepting
and loving, the husband still perceives her as a rejecting person, and
makes no further overtures. Why? What's the use, with such a rejecting
wife? In fact, the husband may well start trying to provoke rejection
just to prove how rejecting his wife is. Result: the husband bolsters
his sense of justification, and the marriage is doomed.
End of allegory.
Rick, I suggest that you take your experimental papers, both published
and unpublished, and any others that seem appropriate, and work them up
into a review of basic experiments that demonstrate fundamental points
of PCT. The cumulative effect should be quite powerful.
Martin is offering us a fine opportunity. Why not use it?
------------------------------------------------------------------------
Lars-Christian Smith (941210.1713) --
Thanks for the reference to Harry Klopf's paper. I gave Klopf a very
hard time some years ago when he kept trying to model control systems
without control systems, and haven't been in communication since then.
We have his paper on order; if he has started using PCT I will owe him
an olive branch.
-----------------------------------------------------------------------
Dick Robertson] (941210.2055 CST) --
Lovely post on reorganization in the clinical world. The suggestions you
make about modeling it are tantalizing -- how I wish we could do such
things! But describing what needs to be done and doing it are worlds
apart right now, at least for me.
Your observations about the phenomena are invaluable, because they
suggest where to look to get more evidence. Just because we modelers
can't handle the complexity, there's no reason to quit making the
observations. PCT can be applied as a plausible organizing principle
even where we can't literally prove that it's the right explanation.
Sometimes just seeing how the theory MIGHT work provides enough
incentive for a researcher to decide to find out if it DOES work that
way.
I think we'll get there sooner rather than later, but right now it
should be obvious from conversations on the net that the main
preoccupation is working out conflicts among different views of
behavior. It is SOOO hard to work out disagreements at the system
concept level; just when you think agreement is at hand, something comes
up to show yet another way of disagreeing. People come into PCT with
huge networks of ideas that have all been shuffled around until they
seem to fit together, and picking them apart is very time-consuming.
When you add to this a touch of humility, by realizing that we, too, are
in a big network of ideas and might have to do some picking apart of our
own, it all gets even slower. But this is what we have to do, if we
aren't to start all over with every new person who comes into PCT.
-----------------------------------------------------------------------
Best to all,
Bill P.