JAVA demos

[Bruce_Abbott]

[From Bruce Abbott (970320.1425 EST)]

Rick Marken (970320.0730 PST)]

Bruce Abbott (970319.1920 EST)

I've taken a look at Rick's JAVA demos and have a couple of
comments to make.

Hmmm. Why am I thinking this is not going to be unrestrained praise;-)

Because you've already read the rest of the post, that's why.

I will be happy to purge the word "handle" from my
write-ups wherever it occurs. Could you please tell me where you found it?

In the write-up accompanying one of the demos. Don't recall which one.

First, in reinforcement theory, consequences don't "select
responses"

Gee. I thought Skinner's "Selection by consequences" article in _Science_
(the one that suggested the title for my demo -- and paper) was about how
consequences _select_ responses? I guess Skinner didn't understand
reinforcement theory, eh? Sure wish you had explained it to him, Bruce.

That's "by" as in "by means of," not as in "by someone," as if a consequence
were a thing that could purposefully select. Skinner didn't need the
explanation, you do.

Second, how movement away from the target can be considered
reinforcement in the demo is beyond my understanding.

Well, all my teachers in graduate school (know-nothings like David Premack)
told me that a reinforcing consequence is one that increases the probability
of the response that produces it. In the demo, if dot movement away from the
target is the consequence of a key press, the probability of another press
is higher than if the consequence of a key press had been dot movement toward
the target.

The reason the liklihood of a press increases is that having the dot move
away from the target is aversive, given that you are attempting to make it
move toward the target. A response that terminates an aversive stimulus
will be reinforced (negative reinforcement). Responses that occur when the
dot is moving away tend to replace the aversive stimulus with one that is
not so aversive (moving away less rapidly), or is attractive (moving toward
the target). Such changes are reinforcing, according to the theory.

Guess you must have skipped class when Premack discussed about that part.

Rick and Bill and I went over this rather thoroughly a couple of
years ago, and I was able to prove via computer simulation that
a _correct_ reinforcement-based model converged on the target
just as the control-system model did.

As I recall, we all agreed that your "_correct_ reinforcement-based
model" was just an arcane version of a control-system model.

No, what we all agreed on was that the reinforcement process embodied in
that model ended up producing a negative feedback relationship, so that it
functioned as a control system. And that being the case, it showed that
reinforcement theory, as I applied it to this situation, could indeed
produce the right sort of behavior, contrary to your claim that it could not.

To put it bluntly, Rick's description of the reinforcement model
and his "demonstration" of its behavior in this situation is
incorrect

But other than that, Mr. Skinner, how'd you like the demo;-)

Other than that, it was great! And I'm eagerly awaiting your next demo, in
which you prove that Darwinian evolution is wrong by showing how all those
"fossils" are natural rock formations, except for the more recent ones of
still-extant species, which of course are the bones of animals drowned in
the Great Flood.

Now before anyone thinks that I am defending reinforcement theory,

Well, if you were "prosecuting" (rather than defending) reinforcement
theory then this was the worst exhibition of prosecutorial skills
I've seen since the OJ trial;-)

You are sadly mistaken, my old friend. If you want to be taken seriously by
the scientific community, you should demonstrate that you've done your
homework and can criticize opposing theories on proper grounds as opposed to
oversimplifications of them. As I am interested in having PCT taken
seriously, I see your demo as an embarrassment that will only make that task
more difficult.

I don't think that PCT is so weak that it can demonstrate its
advantages over reinforcement theory only by misrepresenting
the latter.

Well, we certainly agree on this!

O.K., then DO something about it.

Rick, how about giving that demo the pitch? Pretty please?

Thank you Bruce! I'm honored. This is the best response to my demos yet.
I consider a demo a success when I see a conventional psychologist
walking away from it muttering things like "straw man" or "he doesn't
understand the theory" or "so what".

Too bad for you -- and PCT. I'm no "conventional psychologist" in your
sense of the term, but by throwing me into that category you give yourself
permission to ignore my advice, and indeed, use it as it were some kind of
_proof_ that you've scored a "hit." Come on, Rick, think about what you're
saying. It doesn't make any sense.

But I am kind of disappointed that you don't want me to pitch any of the
other demos. How about the "S-R vs Control" demo. Surely you take exception
to my claim that the input in this task is _not_ the cause of the outputs.
I mean we're talking about the foundations of scientific psychology here.
And how about the "Behavioral Illusion" demo. Surely you take exception to my
claim that this demo shows the futility of trying to understand behavior
using conventional IV-DV methods. I mean, this is about textbook contracts,
Bruce. Big _bucks_ are at stake;-)

That's right Rick, I have an ulterior motive. Why, I should be ashamed of
myself! All this for a few lousy bucks. Yep, ya got me. I've been
terribly frightened that PCT is going to come along and ruin it all for me
-- the girls, the penthouse apartment, the flashy cars -- so I've tried
everything in my power to discredit it, even to the point of spending a full
year pretending to do research on perceptual control systems. (You didn't
think the data I sent you every week were real, did you?) I'm just surprised
that it's taken you so long to see through me. Now if you'll excuse me,
I've got to get back to making money, and planning my next attack.

Regards,

Bruce

[Richard_Marken]

[From Rick Marken (970320.1350)]

Me:

Gee. I thought Skinner's "Selection by consequences" article in _Science_
...was about how consequences _select_ responses?

Bruce Abbott (970320.1425 EST) --

That's "by" as in "by means of,"

That's what I thought. So?

Reinforcement theory says that responses are selected by ("by means of")
their consequences. As Bill Powers (970320.0445 MST) noted, this concept of
reinforcement not only differs from the PCT notion of the relationship
between behavior and its consequences, it is also a _wrong_ concept of that
relationship -- dead wrong.

If you want to be taken seriously by the scientific community, you should
demonstrate that you've done your homework and can criticize opposing
theories on proper grounds as opposed to oversimplifications of them.

Well, then, do you think Bill Powers' (970320.0445 MST) proposed demo of the
failure of reinforcement theory will past the muster of this "scientific
community" of yours?

I find it rather interesting that you are always quick to see my portrayal
of psychological theories as an "oversimplification" or "straw man". (You
make me feel like I was pretty lucky to make through graduate school at
all.) But given the depth of your understanding of these theories, I find it
strange that you don't _help me_ develop demos that test these theories
properly. I feel like you're working against me rather than with me.

I don't think that the E. coli demo is necessarily the best way to show the
failure of reinforcment theory. But I think it does show rather clearly
what is wrong with the most basic version of reinforcent theory -- the one
that Skinner described. The E. coli demo puts the subject in a very simple
situation where "key press" responses produce random consequences. Regardless
of the relative "reinforcingness" of the consequences, it seems to me that
the prediction of basic reinforcement theory is that random consequences will
lead to random behavior (as shown by the behavior of the reinforcement model
in the demo). But the subject in this demo does not respond randomly; indeed,
the subject produces a consistent result -- dot on target -- as does the
control model. I can't see how a reinforcement theorist could have
_legitimately_ predicted this result in advance based on the tenets of
reinforcment theory.

I can see how a reinforcment theorist would be able to predict these
results AFTER he's seen them. It's easy for any theorist to explain results
post hoc. So it's no surprise that it's was possible for you to invent a
version of what you called reinforcement theory (which was actually a
control model) that behaved in the same way as the subject. But why did you
develop this post hoc bail out of reinforcement theory? Why the accusations
that I had oversimplified reinforcement theory? If there was something wrong
with the logic of my test WHY NOT EXPLAIN THE CORRECT WAY TO TEST THE
THEORY!! Why not help me design a demo so that it will discriminate control
and reinforcment theories appropriately instead of showing that you can
bail reinforcement theory out of ANY predicament.

As I am interested in having PCT taken seriously, I see your demo as an
embarrassment that will only make that task more difficult.

OK. Again, why not suggest a non-embarassing demo? Would Bill's suggested
demo (adding a disturbance to mouse movements) work for you? Is there any
way to show that consequeces don't select responses but that responses are
used as the means to select consequences?

I'm no "conventional psychologist" in your sense of the term

Well, you sure do a very convincing imitation;-) You keep saying "I'm for
PCT, I'm for PCT" but your higher level, conventional psychologist control
systems keep pushing back against PCT every time it disturbs your perception
of the value of conventional psycholology. For example, these control
system's can't seem to accept the fact that reinforcement theory is dead
wrong; they can only accept the possibility that PCT is a "better theory".
That's why you can't believe that my demo could show that selection
by consequences doesn't happen; it has to happen (according to some of those
high level control systems of yours); and PCT has to be a better model of
how it happens. That's why the lady is a conventional psychologist;-)

That's right Rick, I have an ulterior motive...I've been terribly
frightened that PCT is going to come along and ruin it all for me
-- the girls, the penthouse apartment, the flashy cars -- so I've tried
everything in my power to discredit it,

No. That's not your ulterior motive. Your ulterior motive is to have PCT be
a theory that provides a better explanation of the "facts" of behavior (as
conventionally defined) than other theories (like reinforcement theory). You
defintely do not want PCT to be seen as a theory that reveals flaws in the
fundamental assumptions on which the behavioral sciences have been built. I
know that this has nothing to do with an interest on your part in money
or fame. I think it has to do with references you have for system level
perceptions of the way science should work and the way living systems do
work.

even to the point of spending a full year pretending to do research on
perceptual control systems.

I look forward to reading the report of that research some day. Perhaps you
could present it at the next CSG meeting?

Best

Rick

[Bill_Powers1]

[From Bill Powers (970320.1340 MST)]

Bruce Abbott (970320.1425 EST)--

Rick sez:

I thought Skinner's "Selection by consequences" article in _Science_
(the one that suggested the title for my demo -- and paper) was about how
consequences _select_ responses?

And you sez:

That's "by" as in "by means of," not as in "by someone," as if a
consequence were a thing that could purposefully select.

Let's get this straight. A causes B (that is, B is a consequence of A).
Therefore A is selected by means of B? The cause is selected by means of its
effect? It seems to me that this is the sort of thing we could find in that
nice list of logical and rhetorical errors that Chuck Tucker posted.

Even saying that the _current_ behavior is selected by means of the
_previous_ consequence doesn't fix the problem. There's still a missing
agent or process. What IS the means? This is like saying that the turning on
of a light is the means by which the _next_ occasion of flipping the switch
is selected. We observe that flipping the switch more frequently causes the
light to turn on more frequently. Where is there any suggestion that the
increase in frequency of the light's turning on is having any influence on
the increase in frequency of flipping the switch? If the frequency of
switching is being slowly increased by an open-loop life-testing apparatus,
it would be ridiculous to say that the on-off frequency of the light is
having some effect on the frequency of switch-flipping. Yet what would you
observe that's any different?

The burden of proof is on you: you have to show that there is a means by
which the past turning on of the light affects the next flip of the switch.
Otherwise your claim that some kind of selection process is going on is
nothing more than an unsupported premise offered out of the blue.

All we actually observe is that as the frequency of a particular behavior
increases, the frequency of its consequences also increases. As you hammer
faster, the nail goes in faster. As you press the lever faster, food is
delivered faster. As the engine turns faster, the car goes faster. Nothing
in these observations explains why the causal event increases in frequency
or speed. There is nothing in the observations to indicate that the
consequences are having any influence at all on their own causes. Any such
influence is entirely a figment of the observer's imagination.

All Skinner was saying was that the environment must be causing behavior,
even when we can't see the means by which it does this. So he simply
imagined a reverse effect, without ever offering the slightest evidence that
it exists. This isn't "Post hoc, ergo propter hoc;" it's the other one that
was described with it in the list Chuck posted. It's asserting a causal
relationship where none is actually observed.

Best,

Bill P.

[Bruce_Abbott]

[From Bruce Abbott (970320.2120 EST)]

Bill Powers (970320.1340 MST) --

Let's get this straight. A causes B (that is, B is a consequence of A).
Therefore A is selected by means of B? The cause is selected by means of its
effect? It seems to me that this is the sort of thing we could find in that
nice list of logical and rhetorical errors that Chuck Tucker posted.

Think in terms of Darwin's theory of evolution. If a gene-specified feature
of an animal aids in the reproductive ability of an animal operating within
a given environment, then those genes tend to become more prevalent within
that enviroment, in the population. A (the gene) causes B (improved ability
to survive, attract a mate, etc.) and B (the said improvemnts) cause, not A
(the gene), but C (increased prevalence of the gene). Back to
reinforcement. A (performing a certain operant) leads to B (receiving the
reinforcer). B (receiving the reinforcer) leads to C (increased frquency of
the the operant). Same logic. Thus, by asserting that reinforcement theory
"is the sort of thing we could find in that nice list of logical ...
errors," you are logically asserting the same thing about Darwin's theory of
evolution. Do you really want to do that?

Even saying that the _current_ behavior is selected by means of the
_previous_ consequence doesn't fix the problem. There's still a missing
agent or process.

You offer this as an improvement, but this latter interpretation is what
I've been offering all along. Now that you are on the right track, you are
correct that the theory provides no mechanism to explain how a given operant
becomes more likely in a given situation, when it has been followed by the
type of consequence termed a reinforcer. This is a descriptive theory, not
a mechanistic one. It only states that certain relationships hold, it does
not offer any mechanisms through which they do. Darwin's theory was missing
a mechanism, too; it only states the observable relationships. Darwin could
not explain how characteristics could vary, where new ones came from, or how
they were transmitted to the offspring. That doesn't seem to have detracted
from its enormous importance to biology.

The burden of proof is on you: you have to show that there is a means by
which the past turning on of the light affects the next flip of the switch.
Otherwise your claim that some kind of selection process is going on is
nothing more than an unsupported premise offered out of the blue.

The burdon of proof is not on me at all, but on those who wish to develop
reinforcement theory into a mechanistic view (like Peter Killeen). I'd
rather work on showing how perceptual control theory explains the same
phenomena that reinforcement theory does, using only purely physical
structures and signals. And how it can explain experimental outcomes that
reinforcement theory can explain only by making a lot of ad hoc assumptions,
if it can be made to do so at all.

However, the claim that some kind of selection process is going on is simply
a generalization from observation. When a given act is followed by certain
consequences, those acts tend to increase in frequency within that same
context. That's a fact, Jack. When organisms have certain characteristics
in a given environment, they tend to thrive, have more offspring, and those
characteristics become more common in the evolving population. Same kind of
fact. Only recently has biology succeeded in identifying some of the major
mechanisms through which these observed relationships of Darwinnian
evolution emerge. Recently there has been a proposal that promises to fill
in some of the mechanical gaps in reinforcement theory. It's called PCT.
Under that view, reinforcers are simply environmental consequences of
behavior that happen to oppose the effects of disturbances to a controlled
variable. However, the role played by these contintent effects in the
process of reorganization has yet to be explored.

Regards,

Bruce

[Bill_Powers1]

[From Bill Powers (970322.1733 PST)]

Bruce Abbott (970320.2120 EST)--

Let's get this straight. A causes B (that is, B is a consequence of A).
Therefore A is selected by means of B? The cause is selected by means of
its effect? It seems to me that this is the sort of thing we could find
in that nice list of logical and rhetorical errors that Chuck Tucker >>posted.

Think in terms of Darwin's theory of evolution. If a gene-specified
feature of an animal aids in the reproductive ability of an animal
operating within a given environment, then those genes tend to become more
prevalent within that enviroment, in the population. A (the gene) causes
B (improved ability to survive, attract a mate, etc.) and B (the said
improvemnts) cause, not A (the gene), but C (increased prevalence of the
gene). Back to reinforcement. A (performing a certain operant) leads to B
(receiving the reinforcer). B (receiving the reinforcer) leads to C
(increased frquency of the the operant). Same logic. Thus, by asserting
that reinforcement theory "is the sort of thing we could find in that nice
list of logical ... errors," you are logically asserting the same thing
about Darwin's theory of evolution. Do you really want to do that?

See my previous post about shifting the meaning of a term, here compounded
by asserting that "prevalence" (in numbers) and "frequency" (of occurrance
of a particular event) are equivalent, in the middle of an argument. A true
parallel would be to say that when some organisms produce a certain operant,
the result is an increased prevalence of organisms that produce that
operant. Whether that is true or not, it is the correct parallel, while
yours is not.

And what possible difference could it make whether I am asserting a logical
criticism of Darwin's theory of evolution? Or, as implied, that many people
would vote in favor of that theory? This is not what is wrong with your
argument, but it is a gratuitious appeal to irrelevancies that represents
yet another fallacious mode of reasoning: the appeal to authority, and the
appeal to numbers, not to mention the implied threat (Do I want to be known
as the sort of person who would dispute the great Darwin?).

Even saying that the _current_ behavior is selected by means of the
_previous_ consequence doesn't fix the problem. There's still a missing
agent or process.

You offer this as an improvement, but this latter interpretation is what
I've been offering all along. Now that you are on the right track, you
are correct that the theory provides no mechanism to explain how a given
operant becomes more likely in a given situation, when it has been
followed by the type of consequence termed a reinforcer.

That isn't the main problem: the main problem is that the way the operant is
defined, there is no evidence that anything but the consequence becomes more
likely, and that is not supported except by showing that its frequency of
occurrance increases (there is no evidence that any "likelihood" has increased).

And look how you shift the meanings of the words around. You say "a given
operant becomes more likely ... when it has been followed by ... a
reinforcer." Here you use the term "given operant" as if it refers to a
given action, which of course could be followed by a reinforcer. But an
operant can't be defined in terms of any given _action_: it is a given
_consequence_. In fact, here it is specifically the consequence that is
reinforcing. You are saying that when a given reinforcer follows from any of
the actions that could produce it, and is followed by that reinforcer, it
becomes more likely. This boils down to saying that if occurrance of the
reinforcer becomes more likely, it becomes more likely.

It seems to me that the whole fallacy of reinforcement is wrapped up in
Skinner's concept of the operant. It is a fallacy to say that if A and B
produce the same change in C, they must have some relationship to each
other, something (other than the common effect) that puts them into the same
class. By this reasoning, blowing on your coffee must have something to do
with adding cold water to the coffee, because they both make it cooler.
Making extra payments on your mortgage must have something in common with
going bankrupt, because they both reduce your debt. If person A fires a
black worker, and person B, a known bigot, also fires a black worker, then
person A must be a bigot, too.

When you can define a "class" of behaviors ONLY in terms of their having the
same consequence, you haven't defined them at all. You can't say in advance
whether any two behaviors would belong to the operant; you can only wait to
see their effects, and assert their association after the fact if they turn
out to have the same effect. As I pointed out in a previous post, no
behavior has to become more likely in order to make the reinforcer occur
more often; each instance of the reinforcer, under the definition of the
operant, could be caused by a different behavior, with no one action
becoming more likely. In the tracking example, we could easily arrange a
long series of trials in which the same "reinforcing" consequence occurred
each time, with both the direction and magnitude of the mouse movements
being randomly distributed. We don't need E. coli to show an example in
which the net reinforcing effect on the "operant" is zero.

Best,

Bill P.