VI-VI Simulation

[From Bruce Abbott (941017.0900 EST)]

Rick Marken (941016.1830)

Another, less important, problem with your way of describing things is
that a schedule of reinforcement is not actually a disturbance; it defines a
functional relationship, i =g(o), between output and input. By changing g(o),
the schedule, you get changes in o. I find it hard to think of changes in a
function as generating changes in its argument, but this is actually
what is happening when you change the schedule in an operant
conditioning experiment.

Yes, the disturbance in this example is brought on by food deprivation, and
the schedule provides the linkage between output (keypecks) and the controlled
variable, thus closing the loop.

Actually, I spent this afternoon learning a bit about VI-VI schedules
using simulation.

Hey, now we're getting somewhere! Care to impart the details of your
simulation? I think it would be helpful for me to see how you went about it.

One thing I learned is that matching does not occur
if the organism simply responds at a high rate, shifting from one key to
another at random intervals. So Bill Powers' intuition about matching
on VI schedules is incorrect; matching is not a trivial result of
responding at high rates to two keys, each with a different VI schedule.

In my response yesterday to Bill's post, I mentioned that I thought something
was wrong, but needed to take some time to go over his reasoning. I'm still
not sure I'm reading him correctly, but his calculations appear to assume that
the schedule associated with one key halts while the bird responds on the
other key. Otherwise, I don't know how the bird spends 12.5 minutes ON EACH
SCHEDULE to earn 100 reinforcements. It would actually require only 12.5
minutes TOTAL.

Right now, I think your intuition about how this
works is based on the idea that perception determines how the
organism responds. Remember, in a control loop, perception is the
dependent variable.

Well, perhaps I'm confused here. What I had in mind was the input function
that converts X (perhaps objective rate of reinforcement) to the perceptual
signal, and thus determines TO WHAT the system responds.

I found your discussion of the results of your simulation informative; I'll
make just a couple of comments as I have to rush off to class now. Response
effort is definitely a factor that should be included in the model (there have
been several attempts to do this within a non-PCT framework, none particularly
successful). And I'm glad to hear that you've found something interesting
going on here.

I'm looking forward to receiving the details about your VI-VI simulation. And
thanks for the help.

Regards,

Bruce

Tom Bourbon [941017.1359]

Breaking in on:

[From Bruce Abbott (941017.0900 EST)]

Rick Marken (941016.1830)

. . .
Rick:

Right now, I think your intuition about how this
works is based on the idea that perception determines how the
organism responds. Remember, in a control loop, perception is the
dependent variable.

Bruce:

Well, perhaps I'm confused here. What I had in mind was the input function
that converts X (perhaps objective rate of reinforcement) to the perceptual
signal, and thus determines TO WHAT the system responds.

Bruce, I think Rick has left town for a few days, so you probably won't see
a reply from him until later in the week. When he returns, he can correct
me if need be, but I believe Rick would reply that the system does not
"respond to" any perceptual signal, whether of X or any other variable.
Instead, if there is a reference signal (x*) that specifies a particular
magnitude of the perceptual signal (x), and if x is an analog of
environmental variable X, then the system's actions will be driven, at
least in part, by k(x-x*), where k is a constant of integration.

On that assumption, and consistent with what Rick was saying, the system
would act to control the level of x, keeping it at the level x*.

I have inverted parts of your reply to Rick. You opened the reply with
the following:

Rick:

Another, less important, problem with your way of describing things is
that a schedule of reinforcement is not actually a disturbance; it defines a
functional relationship, i =g(o), between output and input. By changing g(o),
the schedule, you get changes in o. I find it hard to think of changes in a
function as generating changes in its argument, but this is actually
what is happening when you change the schedule in an operant
conditioning experiment.

Bruce:

Yes, the disturbance in this example is brought on by food deprivation, and
the schedule provides the linkage between output (keypecks) and the controlled
variable, thus closing the loop.

Deprivation is probably a disturbance to perceptions of variables specified
by intrinsic reference signals, perhaps a reference (n*) for a perception (n)
of level of nutrient (N). Error (n - n*) <> 0) at this level sets the
magnitudes of reference signals for other systems, traditionally known as
"behavioral systems," or "response generators," or simply "behaviors." If
my assumptions are correct, then other environmental events can act as
disturbances to the perceptual control systems that produce what an observer
sees as the animal's actions or behaviors -- pecking, bobbing, ingesting,
swallowing, and so on. Through all of that, I suspect Rick and you are
right when you say that the different schedules are not what we _typically_
think of as disturbances, not unless you broaden the definition of
"disturbance" to include deliberate manipulations by the experimenter of the
functional relationship, i = g(o).

This is an interesting thread. On a different subject, but one I believe is
related to EAB, perhaps you would be interested in some of my studies in
which I use continuous interactions between two controllers to illustrate
how one (A) can control the actions of the other (B), but only if while A
sees B doing what A wants to see, B also controls what B intends to control.
The simplest examples of those interactions lead easily to demonstrations
of conflict and countercontrol.

Later,

Tom

[Bruce Abbott (941018.1300 EST)]

Tom Bourbon [941017.1359] --

. . .the system does not
"respond to" any perceptual signal, whether of X or any other variable.
Instead, if there is a reference signal (x*) that specifies a particular
magnitude of the perceptual signal (x), and if x is an analog of
environmental variable X, then the system's actions will be driven, at
least in part, by k(x-x*), where k is a constant of integration.

Tom, I understand this full well, although it may not appear that way from my
statements. I was assuming a constant reference. If x* is constant then k(x-
x*) varies with perceptual signal x and the system will "respond to" ("be
driven, at least in part by") changes in x.

Deprivation is probably a disturbance to perceptions of variables specified
by intrinsic reference signals, perhaps a reference (n*) for a perception (n)
of level of nutrient (N). Error (n - n*) <> 0) at this level sets the
magnitudes of reference signals for other systems, traditionally known as
"behavioral systems," or "response generators," or simply "behaviors." If
my assumptions are correct, then other environmental events can act as
disturbances to the perceptual control systems that produce what an observer
sees as the animal's actions or behaviors -- pecking, bobbing, ingesting,
swallowing, and so on.

I'm not sure I follow you here, as you seem to be stating the obvious.
Deprivation-induced disturbance to high-level perceptual variables sets
reference levels for "behavioral systems" that produce, through setting
references for even lower-level systems, organized sequences of behavior
(either innately given, built through experience, or more likely both) whose
output has in the past countered the disturbance. Sounds like HPCT to
me...what's new?

. . . On a different subject, but one I believe is
related to EAB, perhaps you would be interested in some of my studies in
which I use continuous interactions between two controllers to illustrate
how one (A) can control the actions of the other (B), but only if while A
sees B doing what A wants to see, B also controls what B intends to control.
The simplest examples of those interactions lead easily to demonstrations
of conflict and countercontrol.

How do you see this as being related to EAB? (I'd be interested even if it
weren't). Thank's for the offer, but let's wait on taking up these studies
for now--I'm getting overwhelmed at the moment and need to see some of these
other threads through first, e.g., schedule simulation, Rescorla-Wagner.

Bruce

Tom Bourbon [941019.1000]

[Bruce Abbott (941018.1300 EST)]

Tom Bourbon [941017.1359] --

. . .the system does not
"respond to" any perceptual signal, whether of X or any other variable.
Instead, if there is a reference signal (x*) that specifies a particular
magnitude of the perceptual signal (x), and if x is an analog of
environmental variable X, then the system's actions will be driven, at
least in part, by k(x-x*), where k is a constant of integration.

Tom, I understand this full well,

Good. That will simplify your collaboration with Bill Powers. I was merely
a little uncertain about what you meant when you said the system "responds
to a perceptual signal." We generally try to avoid that wording, since it
_clearly_ implies that "perception controls behavior (PCB)." Making the
case for PCT is hard enough even when we try to show that PCB is false and
the actual relationship is "behavior controls perception (BCP)".

although it may not appear that way from my
statements. I was assuming a constant reference. If x* is constant then k(x-
x*) varies with perceptual signal x and the system will "respond to" ("be
driven, at least in part by") changes in x.

The idea that behavior is "driven, at least in part by changes in the
perceptual signal, x" seems a little too close (for my comfort) to the idea
that perception controls behavior, or to similar ideas, for example, that
there is information in perception about environmental disturbances that
more or less "drive" behavior. For the present, I won't say any more about
that particular issue -- we recently spent _months_ debating it and dropped
the thread more out of exhaustion than because of any sense that we had
resolved it.

I just this moment became aware of another source of my uncertainty about
what you mean when you say that at least in part perception controls
behavior. It has to do with something you said in your post to Bill, part
of which follows.

ยทยทยท

================================

[From Bruce Abbott (941018.1030 EST)]

Bill Powers (941017.1030 MDT)

Bruce Abbott (941016.1530 EST)

Bruce:

I've actually WRITTEN programs that execute VI (and other) schedules
of reinforcement and collect the data, so if you like we can use one of
those as the basis for the environmental side of the simulation.

Bill:

Wonderful. I'd like to see the programs. I can read the source code if
you email it to me (and Rick) or put it on the net. [TB -- I would also

like to see it, and I can. You included it in your reply to Bill.]

Bruce:
Bill, I haven't fished out my old programs yet. Meanwhile, here is my version
of the simulation, complete with a VERY dumb pigeon. Both the pigeon and
schedule are modeled as constant-probability response generators.

You model the pigeon as a probabilistic response generator, rather than as a
control system. I had guessed you might use that kind of model, after
I read some of your earlier posts. I am not being critical. I am simply
trying to understand why I was "probing" you on what you thought about the
fact that, in a PCT model, behavior is driven by k(error), not by perception.
I did not have a clear idea about how you think behavior is generated. I
still don't. My problem, not yours.

Tom:

Deprivation is probably a disturbance to perceptions of variables specified
by intrinsic reference signals, perhaps a reference (n*) for a perception (n)
of level of nutrient (N). Error (n - n*) <> 0) at this level sets the
magnitudes of reference signals for other systems, traditionally known as
"behavioral systems," or "response generators," or simply "behaviors." If
my assumptions are correct, then other environmental events can act as
disturbances to the perceptual control systems that produce what an observer
sees as the animal's actions or behaviors -- pecking, bobbing, ingesting,
swallowing, and so on.

Bruce:

I'm not sure I follow you here, as you seem to be stating the obvious.

I agree, but given the fact that you model the organism as a probabilistic
response generator, not as a control system, I simply didn't know I was
stating the obvious to you. It will help for me to see how your
collaboration with Bill unfolds.

Deprivation-induced disturbance to high-level perceptual variables sets
reference levels for "behavioral systems" that produce, through setting
references for even lower-level systems, organized sequences of behavior
(either innately given, built through experience, or more likely both) whose
output has in the past countered the disturbance. Sounds like HPCT to
me...what's new?

What's "new" is that up to now PCT has not been about setting references for
behavior. In my post I was trying to say that error relative to intrinsic
references sets the reference perceptions for lower-level control systems,
which produce behavior that is _uncontrolled_. My allusions to "behavioral
systems" and "behaviors" were _intended_ as _dismissive_ of those
traditional ideas. I blew it.

Tom:

. . . On a different subject, but one I believe is
related to EAB, perhaps you would be interested in some of my studies in
which I use continuous interactions between two controllers to illustrate
how one (A) can control the actions of the other (B), but only if while A
sees B doing what A wants to see, B also controls what B intends to control.
The simplest examples of those interactions lead easily to demonstrations
of conflict and countercontrol.

How do you see this as being related to EAB? (I'd be interested even if it
weren't).

Hmm. You will need to educate me on EAB. My reading of the EAB literature
comes in spurts and my knowledge of the field must include rather large
holes. I thought at least in part EAB was about determining those features
of the environment (context) that control (or set the occasion for)
behavior. And I thought, from reading people like Steven Hayes and BF
Skinner, that the strongest "proof of understanding" in EAB was a
demonstration that the behavior analyist had identified variables through
which he or she could control the behavior of another. And I thought
counter control was often described as a common feature of interactions
between a controller and a controllee -- a nearly inevitable feature, if
the controller and controllee are two humans.

My tasks and models show how easily PCT explains those phenomena, which I
perhaps erroneously thought were part of EAB. Can you show me where I went
astray in my interpretation of EAB?

Thank's for the offer, but let's wait on taking up these studies
for now--I'm getting overwhelmed at the moment and need to see some of these
other threads through first, e.g., schedule simulation, Rescorla-Wagner.

Fine, but can you give me a few clues about where I went wrong in my
understanding of EAB? I planned to submit a manuscript to JEAB, but there
is no need to do that if I have seriously misconstrued EAB. (For those not
familiar with the notation: EAB = experimental analysis of behavior [a
contemporary instantiation of radical behaviorism -- not a pejorative
label]; JEAB = _Journal of the EAB_.)

Later,

Tom

[FROM: [Dennis Delprato (941019)]

Tom B. (941019.1000) [to earlier post by Bruce Abbott]:

You model the pigeon as a probabilistic response generator, rather than as a
control system. I had guessed you might use that kind of model, after
I read some of your earlier posts. I am not being critical. I am simply
trying to understand why I was "probing" you ....

Tom, this may bring up a profound point upon which you could make
a rather straightforward case for the PCT model as unique.
Of course, EAB models would treat the organism as a probabilistic response
generator. The operant is said to be "emitted" on a probabilistic
basis as determined primarily by reinforcement history. It
is thought to be _controlled by_ history of environmental
reinforcement, not _controlling_ input. And,
the "dependent variable" is said to be reflecting probability
of response.

You, or someone, could teach us a lot by preparing a paper
that clearly shows how models differ based on the two fundamentally
different sets of assumptions (1: behavior as controlled by
history of reinforcement and as generated by a probabilistic process
somehow established by this history & 2: behavior as controlling input).
Comparisons of the models' ability to predict actual data would help
as well. This may seem very elementary, but work directly addressing
the two incompatible sets of assumptions could be fundamental for
changing outlooks from _behavior as controlled_ to _behavior as
controlling_.

I'd be interested in what Bruce Abbott has to say about this.

For starters at least,
could you point out what you saw in Bruce's model that
led you to detect the assumption he was making?

Dennis Delprato
psy_delprato@emuvax.emich.edu