[From Bill Powers (941123.0845 MST)]
Boy, this is frustrating. I got a snippet of a reply to my last post
from a citation of a 1630 Bruce Abbott post by Tom Bourbon written at
1730 PM last night. I still haven't received Bruce's post at 0845 this
morning. From header information it seems that my mail is being delayed
about 12 to 14 hours in the Berkeley, CA server before being send on to
Boulder, CO. The delay does not seem to be in our own server. This makes
for some weird overlapping conversations.
Anyway --
OK, Bruce, you want to express your system's logic in the original form.
Remember that I have never said that ECOLI4 doesn't work. It does work,
and for exactly the reasons you lay out. If the organism is swimming up
the gradient, the probability is that the next tumble will leave it
swimming less favorably up the gradient or even down the gradient. Your
program is designed to take advantage of this fact, and it does so. No
problem there.
You may remember that some time back I suggested that the program might
work because of a peculiarity or assymetry in the E. coli situation. And
I have persistently expressed doubts that your program is "the essence
of the law of effect and the kernel of reinforcement theory," or however
you put it.
If your program is what you say, then we should be able to express it in
general terms and see if those terms apply to other situations that are
also said to be examples of the Law of Effect. I will try to do that,
subject to corrections from you. We start with the snippet from Tom's
post:
[ If things have gotten better, then this is reinforcement. Increase
the probability of a tumble given the condition present prior to the
tumble (i.e., positive or negative NutSave).]If DeltaNutRate > 0 then { Nutrient rate increased by tumble: reinforce
}
begin { tumbling
}
If NutSave > 0 then { If S+ present during tumble then
}
begin { increase probability of tumble given S+
}
pTumbleGivenSplus := pTumbleGivenSplus + LearnRate;
if pTumbleGivenSplus > pMax then pTumbleGivenSplus := pMax;
end
else { S- present when last tumbled then
}
begin { increase probability of tumble given S-
}
pTumbleGivenSminus := pTumbleGivenSminus + LearnRate;
if pTumbleGivenSminus > pMax then pTumbleGivenSminus :=
pMax;
end
end
else
Let's unpack a little:
S = state of discriminating stimulus (terminology?)
S(old) state of S before previous action
S(new) state of S after previous action
So "DeltaNutRate > 0" means
Previous action made S(new) more favorable than S(old)
Identifying variables,
NutSave = S(old).
dNut = S(new).
DeltaNutRate = S(new) - S(old)
The first part of your logic says, therefore:
If the previous action made S(new) better than S(old), then
If S(old) was favorable, decrease the probability of the action, else
If S(old) was unfavorable, iuncrease the probability of the action.
The other part of the logic is the converse, when the action makes
S(new) less favorable than S(old), so whatever we say about the first
part will extend to the second.
Your claim is that the rule above expresses the essence of the Law of
Effect, which is presumably the grandfather law and applies to all
situations. Your program, you say, is an instance of the general law.
Let's apply the above rule to another situation. A rat is in a maze,
following the smell of cheese. It is trying to keep the smell growing
stronger. The action it can take is to keep going up one corridor, or
turn back (and go up a different corridor).
S = rate of change of smell of cheese
action = move ahead or turn back
increase probability of action = go ahead more
decrease probability of action = turn back
The rule above says,
If the previous direction of progress made the cheese smell grow faster
than its previous rate, then
if the previous rate of change was positive, turn back, else
if the previous rate of change was negative, go ahead more.
Note that we have changed the geometry; now the only choices of movement
are ahead and back, so it is no longer true that a positive rate of
change of S predicts a less positive rate of change. In fact, in this
case a positive rate of change of S predicts an equal or more positive
rate of change of S if the same action is repeated or increased.
So I have three questions:
1. Have I generalized properly from your program?
2. Have I applied the generalization correctly to the new situation?
3. Is the prediction in the new situation in agreement with the Law of
Effect?
···
---------------------------------------------------------------------
Best,
Bill P.