Controlling Rats

[From Rick Marken (2002.02.23.0830)]

Fred Nickols (2002.02.23.0925)--

My comments merely reflect my notion that a behaviorist would interpret and
label what was going on as reinforcing or reinforcement at work (with
reinforcement defined as strengthening a response or increasing its
occurrence). "Rewarding" will do just as well. But, now that you ask, I
wonder, do we act to control our perceptions because we find it rewarding?

Yes, of course. We control perceptions that we have been rewarded for controlling
in the past. Control of perception is a result of our history of reinforcement.
This has been tested and shown to be statistically significant at the .01 level.
:wink:

Best regards

Rick

···

--
Richard S. Marken
MindReadings.com
marken@mindreadings.com
310 474-0313

[From Rick Marken (2002.02.23.1600)]

Bill Powers (2002.02.23.1107 MST)--

Now, now. A little smiley doesn't make it all right to be sarcastic, does it?

I certainly hope not. Sarcasm, like sex, is best when it's definitely not "all
right".

Best regards

Rick

···

---
Richard S. Marken
MindReadings.com
marken@mindreadings.com
310 474-0313

[From Bruce Abbott (2002.02.17.2250 EST)]

While doing some library research I came across an old experiment by Larry
Stein and Oakley S. Ray (1959). Stein and Ray implanted electrodes into an
area of the rat's brain that will produce self-stimulation, as if the
experience were, at some current levels at least, pleasurable to the rat.

In the experimental arrangement, the rat could press either of two
levers. Pressing either lever delivered a brief train of electrical pulses
to the electrode. However, each after each press of one lever, a stepper
motor increased the current intensity by a slight amount, whereas after
each press of the other lever, the stepper decreased the intensity by a
similar amount. By switching levers at appropriate times, the rat could
keep the intensity of stimulation between certain bounds, and this it did.

Five-minute periods during which lever-pressing could produce stimulation
were alternated with 5-minute periods during which the current was switched
off. At the start of each new period of stimulation the initial current
level was reset to its lowest value and the rat had to work the intensity
back up to the preferred level by repeatedly pressing the lever yielding
intensity increases. Despite these repeated disturbances, the rat always
quickly reestablished the same narrow range of stimulation intensity that
it had produced on the previous occasions.

The experiment showed very nicely that the rats in the study developed a
preferred level of electrode stimulation (i.e., established a reference
level of intensity), actively defended that intensity against disturbance,
and maintained excellent control over that intensity by appropriately
varying the lever on which they directed their outputs.

It seems to me that the rats' behaviors could be modeled very nicely by an
ordinary two-way control system modified to include a threshold error
level. The output mechanism would switch output (lever-presses) to the
"down" lever when positive error exceeded threshold, and to "up" lever when
the negative error exceeded threshold.

···

-------------------------------------------------------
Stein, Larry, & Oakley, S. Ray (1959). Self-regulation of brain-stimulating
current intensity in the rat. Science, 130, 570-572.

[From Bill Powers (2002.02.18.0446 MST)]

Bruce Abbott (2002.02.17.2250 EST)--

While doing some library research I came across an old experiment by Larry
Stein and Oakley S. Ray (1959). Stein and Ray implanted electrodes into an
area of the rat's brain that will produce self-stimulation, as if the
experience were, at some current levels at least, pleasurable to the rat.

In the experimental arrangement, the rat could press either of two
levers. Pressing either lever delivered a brief train of electrical pulses
to the electrode. However, each after each press of one lever, a stepper
motor increased the current intensity by a slight amount, whereas after
each press of the other lever, the stepper decreased the intensity by a
similar amount. By switching levers at appropriate times, the rat could
keep the intensity of stimulation between certain bounds, and this it did.

I think I saw a report on this experiment, somewhat later. Didn't you use
something similar to give rats control of their cage illumination? Of
course we have plenty of data on rats who control the availability of food
and water as well as the frequency of receiving electric shocks (to the
extent permitted by the schedule). I recall, vaguely, an experiment where a
rat controlled something by varying the place where it pressed along a long
thin bar -- a rare change from the rate-of-pressing measure of behavior.

Of course we can interpret the results of most (maybe all) op-cond
experiments as showing how the animal controls variables important to it in
the presence of various disturbances and given the availability of
different feedback functions (schedules). I seem to recall cases of analog
control -- where an animal, for example, learns to steer a little vehicle,
or to pull on strings to move things, and so forth. There have also been
experiments in which animals had to vary an output force to exert control.
What else? I'll bet that quite a collection could be found. That would make
a dandy book, expecially if the second half of the book reported on new
experiments specifically designed to test analog control. Do you maybe have
something like this in mind?

Best,

Bill P.

···

Five-minute periods during which lever-pressing could produce stimulation
were alternated with 5-minute periods during which the current was switched
off. At the start of each new period of stimulation the initial current
level was reset to its lowest value and the rat had to work the intensity
back up to the preferred level by repeatedly pressing the lever yielding
intensity increases. Despite these repeated disturbances, the rat always
quickly reestablished the same narrow range of stimulation intensity that
it had produced on the previous occasions.

The experiment showed very nicely that the rats in the study developed a
preferred level of electrode stimulation (i.e., established a reference
level of intensity), actively defended that intensity against disturbance,
and maintained excellent control over that intensity by appropriately
varying the lever on which they directed their outputs.

It seems to me that the rats' behaviors could be modeled very nicely by an
ordinary two-way control system modified to include a threshold error
level. The output mechanism would switch output (lever-presses) to the
"down" lever when positive error exceeded threshold, and to "up" lever when
the negative error exceeded threshold.
-------------------------------------------------------
Stein, Larry, & Oakley, S. Ray (1959). Self-regulation of brain-stimulating
current intensity in the rat. Science, 130, 570-572.

[From Bruce Abbott (2002.02.18.1255 EST)]

Bill Powers (2002.02.18.0446 MST) --

Bruce Abbott (2002.02.17.2250 EST)

While doing some library research I came across an old experiment by Larry
Stein and Oakley S. Ray (1959). Stein and Ray implanted electrodes into an
area of the rat's brain that will produce self-stimulation, as if the
experience were, at some current levels at least, pleasurable to the rat.

Of course we can interpret the results of most (maybe all) op-cond
experiments as showing how the animal controls variables important to it in
the presence of various disturbances and given the availability of
different feedback functions (schedules). I seem to recall cases of analog
control -- where an animal, for example, learns to steer a little vehicle,
or to pull on strings to move things, and so forth. There have also been
experiments in which animals had to vary an output force to exert control.
What else? I'll bet that quite a collection could be found. That would make
a dandy book, expecially if the second half of the book reported on new
experiments specifically designed to test analog control. Do you maybe have
something like this in mind?

Yes, but the problem is to find clear examples where it was actually
possible for the animal to bring a variable near some reference value and
keep it there by varying its outputs as necessary. The Stein and Ray
brain-stimulation study is a nice example despite the use of an unusual CV,
although it might have been nicer if it had smoothly and irregularly
disturbed (varied) the stimulation intensity.

When an experiment specifically allows the subject to control some variable
the study is usually billed as examining "regulation" or "self-regulation,"
or "behavioral regulation." as if using actions to assert control over some
variable were a special case. It is not generally recognized that, far from
constituting a special case, in virtually all cases behavior is organized
around the control of the organism's inputs.

I expect to be returning to the lab in the relatively near future to begin
gathering further quantitative data on escape and avoidance. These have the
advantage (over control of food, water, ect.) of not being subject to
numerous automatic physiological controls that tend to get in the way of a
clean analysis of control mediated by the animal's behavior.

By the way, anyone who's interested can see Stein and Ray's (1959) Figure 1,
showing behavioral control of stimulation current, at the following website:

http://users.ipfw.edu/abbott/pct/index.html

This is my PCT page; follow the link there to the Stein and Ray data. I'll
be adding more examples as I come across them. If anyone has other
candidate examples, please provide the references and I'll take a look at them.

One that just occurred to me was a study demonstrating that goslings follow
the mother or fellow goslings at a distance that keeps the angle subtended
by the bird's top constant. Thus they follow other goslings at a closer
distance than they follow their mother. But I don't know where I learned of
this . . .

Best wishes,

Bruce A.

[From Fred Nickols (2002.02.19.0740)] --

[From Bruce Abbott (2002.02.17.2250 EST)]

While doing some library research I came across an old experiment by Larry
Stein and Oakley S. Ray (1959).

<snip>

The experiment showed very nicely that the rats in the study developed a
preferred level of electrode stimulation (i.e., established a reference
level of intensity), actively defended that intensity against disturbance,
and maintained excellent control over that intensity by appropriately
varying the lever on which they directed their outputs.

<snip>

Stein, Larry, & Oakley, S. Ray (1959). Self-regulation of brain-stimulating
current intensity in the rat. Science, 130, 570-572.

Nice find. How did they interpret the results? In what kind of conceptual
scheme did they frame them?

Now that I'm no longer with ETS, I don't have access to that kind of
material through the ETS library. How does an ordinary citizen lay hands
on that kind of material? Can I get it through the local public library?

P.S. My relocation to Ohio is complete. We're all moved in, settled in,
and starting to settle down. I treated myself to a new home office so I'm
enjoying that, although there is no high-speed internet access in this area
so I'm stuck with dial-up for now. I finished the paper about "A Better
Model" (a PCT view of human performance) about which I solicited comments
this past October and it is now up on my web site in the Training & Human
Performance section.

Regards,

Fred Nickols
740.397.2363
nickols@att.net
"Assistance at A Distance"
http://home.att.net/~nickols/articles.htm

[From Bruce Abbott (2002.02.19.1715 EST)]

Good to hear from you!

Fred Nickols (2002.02.19.0740) --

Bruce Abbott (2002.02.17.2250 EST)

Stein, Larry, & Oakley, S. Ray (1959). Self-regulation of brain-stimulating
current intensity in the rat. Science, 130, 570-572.

Nice find. How did they interpret the results? In what kind of conceptual
scheme did they frame them?

"Increasing the intensity of the stimulating current train has at least two
important effects: (a) the probability or frequency of activation of
neurons in the stimulating field is increased; and (b) the effective
stimulating field is enlarged to include new elements. On the assumption
that the animals will work to maximize the firing of positively reinforcing
cell groups, it may be expected that, within physiological limits, the
greater the current the better. That intermediate intensities are favored
may be due, in part, to the spreading of stimulation with large currents to
inhibitory or negatively reinforcing regions."

As I noted in the previous post, there is no indication that these
researchers recognized that this situation, in which the animals
behaviorally regulated the level of stimulating current, might hint at a
more general principle -- that most behavior is in fact organized around
the control of the organism's inputs. They saw their procedure as little
more than a method for assessing the rat's preferred level of self-stimulation.

Now that I'm no longer with ETS, I don't have access to that kind of
material through the ETS library. How does an ordinary citizen lay hands
on that kind of material? Can I get it through the local public library?

This article and one I will post shortly both appeared in Science, which is
subscribed to by many public libraries. The particular copies I have were
reprints that appeared in "The Experimental Analysis of Behavior" (edited
by Thom Verhave), part of the old Century Psychology Series.

The better place to look for research like this would be at a university
library, of course. There are also search facilities available on the web
that for a small fee will send you copies of any articles you identify in a
search.

P.S. My relocation to Ohio is complete. We're all moved in, settled in,
and starting to settle down.

I've forgotten -- where in Ohio did you say you've moved to? I'm just over
the border in Indiana.

Best wishes,

Bruce A.

[From Fred Nickols (2002.02.21.0637)] --

Bruce Abbott (2002.02.19.1715 EST)]

Good to hear from you!

Thanks.

>Fred Nickols (2002.02.19.0740) --
>
>>Bruce Abbott (2002.02.17.2250 EST)
>>
>>Stein, Larry, & Oakley, S. Ray (1959). Self-regulation of brain-stimulating
>>current intensity in the rat. Science, 130, 570-572.
>
>Nice find. How did they interpret the results? In what kind of conceptual
>scheme did they frame them?

"Increasing the intensity of the stimulating current train has at least two
important effects: (a) the probability or frequency of activation of
neurons in the stimulating field is increased; and (b) the effective
stimulating field is enlarged to include new elements. On the assumption
that the animals will work to maximize the firing of positively reinforcing
cell groups, it may be expected that, within physiological limits, the
greater the current the better. That intermediate intensities are favored
may be due, in part, to the spreading of stimulation with large currents to
inhibitory or negatively reinforcing regions."

Hmm. I would assume that maintaining a reference specification, especially
in the face of disturbances, would indeed be reinforcing, no matter the
nature of the reinforcer or the reference specification. In this case, the
action seems to be occurring at a very low level in the hierarchy.

As I noted in the previous post, there is no indication that these
researchers recognized that this situation, in which the animals
behaviorally regulated the level of stimulating current, might hint at a
more general principle -- that most behavior is in fact organized around
the control of the organism's inputs. They saw their procedure as little
more than a method for assessing the rat's preferred level of
self-stimulation.

A technical question about your use of "inputs." If the rats were indeed
controlling for some internal state, is that an "input" to the "organism"
or would it be more technically correct to say that "most behavior is in
fact organized around the control of perceptions." I've always been
confused by perceptions as inputs. To me, perceptions are internal
states. Can you clarify this for me? Thanks.

>Now that I'm no longer with ETS, I don't have access to that kind of
>material through the ETS library. How does an ordinary citizen lay hands
>on that kind of material? Can I get it through the local public library?

This article and one I will post shortly both appeared in Science, which is
subscribed to by many public libraries. The particular copies I have were
reprints that appeared in "The Experimental Analysis of Behavior" (edited
by Thom Verhave), part of the old Century Psychology Series.

The better place to look for research like this would be at a university
library, of course. There are also search facilities available on the web
that for a small fee will send you copies of any articles you identify in a
search.

Phil Runkel was kind enough to remind me that interlibrary can get anything
so I'll get on it soon.

>P.S. My relocation to Ohio is complete. We're all moved in, settled in,
>and starting to settle down.

I've forgotten -- where in Ohio did you say you've moved to? I'm just over
the border in Indiana.

I'm just outside Mount Vernon, north and east of Columbus. It's a 45
minute drive to Columbus. If memory services, it's about an hour from
there to Dayton and that's right next to Indiana.

Regards,

Fred Nickols
740.397.2363
nickols@att.net
"Assistance at A Distance"
http://home.att.net/~nickols/articles.htm

[From Bruce Abbott (2002.02.21.0955 EST)]

Fred Nickols (2002.02.21.0637) --

Bruce Abbott (2002.02.19.1715 EST)

"Increasing the intensity of the stimulating current train has at least two
important effects: (a) the probability or frequency of activation of
neurons in the stimulating field is increased; and (b) the effective
stimulating field is enlarged to include new elements. On the assumption
that the animals will work to maximize the firing of positively reinforcing
cell groups, it may be expected that, within physiological limits, the
greater the current the better. That intermediate intensities are favored
may be due, in part, to the spreading of stimulation with large currents to
inhibitory or negatively reinforcing regions."

Hmm. I would assume that maintaining a reference specification, especially
in the face of disturbances, would indeed be reinforcing, no matter the
nature of the reinforcer or the reference specification. In this case, the
action seems to be occurring at a very low level in the hierarchy.

I find it interesting that the rats do have a reference value for this
unnatural stimulation and that it is well above zero. But I'm puzzled by
your last sentence above. How do you define a "very low level"?

As I noted in the previous post, there is no indication that these
researchers recognized that this situation, in which the animals
behaviorally regulated the level of stimulating current, might hint at a
more general principle -- that most behavior is in fact organized around
the control of the organism's inputs. They saw their procedure as little
more than a method for assessing the rat's preferred level of
self-stimulation.

A technical question about your use of "inputs." If the rats were indeed
controlling for some internal state, is that an "input" to the "organism"
or would it be more technically correct to say that "most behavior is in
fact organized around the control of perceptions." I've always been
confused by perceptions as inputs. To me, perceptions are internal
states. Can you clarify this for me? Thanks.

When I stated that organisms control their inputs, I was thinking of inputs
as perceptual signals, although that wasn't clear from the context. This
statement is meant to emphasize that control systems control their inputs
(perceptions) and not their outputs (actons). Confusion about this
sometimes arises because of the way engineers tend to diagram and label a
control system such as a voltage regulator. They tend to label the
reference as the "input" (because that's where the human being inputs the
reference setting) and the input as the output (because the regulated
voltage is considered the "output" or product of the device -- it's
"product" is regulated voltage). In comes the specification of the voltage
desired, out comes the desired voltage. But this is a description from the
user's point of view, not from the point of view of the device
itself. From the latter viewpoint, the voltage being "output" is monitored
at the device's input, compared to reference, and actions taken internally
as necessary to keep the monitored voltage near its reference value.

Although I'm sure you already understand what follows, I thought that it
wouldn't hurt, and might do some good, to review how terms like "input" and
"perception" (or "perceptual signal") are often used in this context.

An "input" might be defined as a variable entering a system or some part of
a system. By this definition, in the basic control system the perception
and reference are both inputs to the comparator, the error signal is the
sole input to the output function, the output action is the input to the
environment, and so on.

However, when speaking of the control system as a whole, we usually define
the "input" (at the lowest level, at least) as an environmental variable
that is directly transduced to the perceptual signal by sensory
receptors. (This environmental variable usually has the advantage for
researchers of being easily observed and measured.) Consider the ordinary
home thermostat. We usually say that its input is the temperature of the
air surrounding the thermostatic element, but it would be in more accurate
to say that what is being controlled is the position of the electrical
contacts. For a person adjusting the intensity of a light we might say
that light-intensity is the input (and the controlled variable) but
recognize that the variable entering the comparator of this control system
is not the light-intensity itself of a correlate of it. In turn, the
person's actions affect the intensity of the light fairly directly, as a
means of varying the perception of its intensity.

At higher levels in the hierarchy, the input may simply be the output of a
system at the next-lower level. In that case the input and perceptual
signal are one and the same. The distinction becomes important, however,
when the perceptual signal is some non-identical function of another
variable or is synthesized from some combination of inputs.

I hope that was clear.

>P.S. My relocation to Ohio is complete. We're all moved in, settled in,
>and starting to settle down.

I've forgotten -- where in Ohio did you say you've moved to? I'm just over
the border in Indiana.

I'm just outside Mount Vernon, north and east of Columbus. It's a 45
minute drive to Columbus. If memory services, it's about an hour from
there to Dayton and that's right next to Indiana.

Ah, I know just where that is! A century and a half ago, ancestors of mine
lived just north of Fredericktown, which is a mere stone's throw from Mount
Vernon.

Bruce A.

[From Rick Marken (2002.02.21.0820)]

Fred Nickols (2002.02.21.0637) --

Hmm. I would assume that maintaining a reference specification, especially
in the face of disturbances, would indeed be reinforcing, no matter the
nature of the reinforcer or the reference specification. In this case, the
action seems to be occurring at a very low level in the hierarchy.

Bruce Abbott (2002.02.21.0955 EST)

I find it interesting that the rats do have a reference value for this
unnatural stimulation and that it is well above zero. But I'm puzzled by
your last sentence above. How do you define a "very low level"?

I'm puzzled by Fred's first sentence: "maintaining a reference
specification...would indeed be reinforcing". What is "reinforcing"? Rewarding?
Strengthening? Isn't the maintenance of a perception in a specified reference
state just called "controlling"?

Best

Rick

···

---
Richard S. Marken, Ph.D.
The RAND Corporation
PO Box 2138
1700 Main Street
Santa Monica, CA 90407-2138
Tel: 310-393-0411 x7971
Fax: 310-451-7018
E-mail: rmarken@rand.org

[From Bill Powers (2002.02.21.9852 MST)]

Bruce Abbott (2002.02.19.1715 EST)--

Stein, Larry, & Oakley, S. Ray (1959). Self-regulation of brain-stimulating
current intensity in the rat. Science, 130, 570-572.

...

"Increasing the intensity of the stimulating current train has at least two
important effects: (a) the probability or frequency of activation of
neurons in the stimulating field is increased; and (b) the effective
stimulating field is enlarged to include new elements. On the assumption
that the animals will work to maximize the firing of positively reinforcing
cell groups, it may be expected that, within physiological limits, the
greater the current the better. That intermediate intensities are favored
may be due, in part, to the spreading of stimulation with large currents to
inhibitory or negatively reinforcing regions."

.. They saw their procedure as little
more than a method for assessing the rat's preferred level of

self-stimulation.

Right, and notice how they were forced to handle the response to an
_excess_ of stimulation. If stimulating that region was reinforcing, then
the greater the stimulation, the greater the reinforcement, and the animals
should have ended up at the maximum possible stimulation. Thus it becomes
necessary to introduce some imagined facts: " That intermediate intensities
are favored
may be due, in part, to the spreading of stimulation with large currents to
inhibitory or negatively reinforcing regions."

This tells me that the authors were smart enough to see that we _should_
have seen a positive feedback effect under the assumption that the current
is positively reinforcing, and that failure to see it has to be accounted
for. The only way to account for it without giving up the initial
interpretation of positive reinforcement is to introduce some suppressing
or inhibiting effect that comes into play at some level of stimulation and
grows rapidly for higher rates of stimulation. This added assumption is a
patch added to repair a failure of the basic model.

Bruce Abbott (2002.02.21.0955 EST)--

I find it interesting that the rats do have a reference value for this
unnatural stimulation and that it is well above zero.

Actually we have no idea what the rats are controlling, precisely for the
reason you suggest: the stimulation is "unnatural." If a heat receptor is
electrically stimulated, we feel heat, not electricity. If the retina is
elecrically or mechanically stimulated, we see light rather than
electricity or pressure. See Wilder Penfield's experiments with the
stimulation of awake subjects' brains. When Penfield electrically
stimulated the brain of one subject, the subject got a strong impression
that the doctor was rushing toward the subject (while staying in the same
place!). Another subject, simulated in the temporal cortex, got a brief,
repeatable, vivid image of a scene -- a picnic, if I remember right. Many
experiences of many kinds were perceived during electrical stimulation --
but there was no experience of electricity.

PCT proposal:
Whatever the rat was controlling, it was equivalent to a perceptual signal
for which it already had a reference level set at an intermediate value,
but was not currently satisfying by current actions. Pressing the lever
could increase that signal, and so the rat suddenly was able to correct the
error by maintaining some intermediate level of bar-pressing.

Note other experiments (Olds?) purporting to identify a "pleasure center."
Rats would press a bar even in preference to eating to obtain stimulation
in this area. Of course this needn't have anything to do with pleasure.
Suppose the experimenter happened to find the center that is normally
stimulated by the sensation of a pleasantly full stomach. Then the rat
could make itself feel full of food immediately by pressing a lever, and it
would then have no reason to eat anything. Or the stimulation could arouse
some very high-priority perceptual signal such as sexual sensations. Given
an immediate means of controlling this signal, the rat would switch to
controlling it even at the expense of giving up, temporarily, other control
processes.

Note also that if this stimulation bypassed natural receptors that normally
habituate and cease to respond after some time, the artificial stimulation
might be maintained much longer than the natural version could normally be
maintained. The endless orgasm, for example.

Best,

Bill P.

[From Bill Curry (2002.02.21.1220 EST)]

Bill Powers (2002.02.21.9852 MST)

Suppose the experimenter happened to find the center that is normally
stimulated by the sensation of a pleasantly full stomach. Then the rat
could make itself feel full of food immediately by pressing a lever, and it
would then have no reason to eat anything. Or the stimulation could arouse
some very high-priority perceptual signal such as sexual sensations. Given
an immediate means of controlling this signal, the rat would switch to
controlling it even at the expense of giving up, temporarily, other control
processes.

The reference to a "very high priority perceptual signal" piqued my interest. I
find it curious that the sexual sensation control system can be happily perking
along (presumably at zero error with zero sensation), and upon stimulation, shift
to a reference for high stimulation and shed other important control
processes. How does HPCT explain the concept of priority in this context? When
the rat switches [or is switched] to controlling a variable creating sexual
sensations, what mechanism preempts the other control processes?

My best,

bill

···

--
William J. Curry Capticom, Inc.
bill@powerseed.com 603.756.9933

[From Bruce Abbott (2202.02.21.1730 EST)]

Rick Marken (2002.02.21.0820) --

>Fred Nickols (2002.02.21.0637)

>Hmm. I would assume that maintaining a reference specification, especially
>in the face of disturbances, would indeed be reinforcing, no matter the
>nature of the reinforcer or the reference specification. In this case, the
>action seems to be occurring at a very low level in the hierarchy.

I'm puzzled by Fred's first sentence: "maintaining a reference
specification...would indeed be reinforcing". What is "reinforcing"?
Rewarding?
Strengthening? Isn't the maintenance of a perception in a specified reference
state just called "controlling"?

The whole concept of pleasure/displeasure is missing from PCT. Once a
control system gets organized, it just controls what it controls; it seems
unnecessary to add another variable, correlated with the controlled
variable, to represent its experienced pleasantness or
unpleasantness. Indeed, many variables we control all the time seem to
have no subjective quality of pleasantness/unpleasantness associated with them.

Which leads one to wonder why some of them do.

Bill Powers (2002.02.21.9852 MST) --

Note also that if this stimulation bypassed natural receptors that normally
habituate and cease to respond after some time, the artificial stimulation
might be maintained much longer than the natural version could normally be
maintained. The endless orgasm, for example.

Hey, sign me up!

Bruce A.

[From Bill Powers (2002.02.21.1641 MST)]

Bill Curry (2002.02.21.1220 EST)--

I find it curious that the sexual sensation control system can be happily

perking

along (presumably at zero error with zero sensation), and upon

stimulation, shift

to a reference for high stimulation and shed other important control
processes.

Me, too. But it does seem to work that way, doesn't it? I once proposed a
"universal error curve" in an attempt to deal with this general problem.
The general problem is that when errors get _very_ large, people don't (I'm
guessing) just continue producing more and more output. Instead, they
"give up." What is a _very large_ error? Well, consider a dog chasing a
car. If the car is going 10 mph, the dog can pretty well keep up with it.
At 15 mph, it may start lagging behind a little, with a discernible error.
But what if the car goes past at 75 mph? The dog might make a slight effort
to chase it, but the positional error or the speed error will be way
outside the range that any dog could possible correct. So does the dog
chase on down the road after the car at maximum speed, getting farther and
farther behind, miles and miles behind, for the next hour? No, it simply
turns and trots back home as if it didn't care about cars at all.

This can be handled as higher-level control, of course. A higher system
concerned with not wasting energy, or with controlling successfully, or
whatever else you want to propose, vetoes reference signals sent to lower
systems if this control system imagines that they will (or perceives that
they do) accomplish nothing but creation of huge errors and physical
exhaustion. The reference signals that would cause such results are
inhibited, or perhaps the systems that sent them are turned off at the
output function.

That's a straightforward proposal, but it does clutter up the model with a
lot of extra higher-level control systems that do nothing but inhibit
reference signals when errors get too large. It would be nicer if there
were a simpler (and dumber) way to account for the giving-up phenomenon.
This is where the universal error curve comes in.

Suppose that as the difference between a perceptual signal (any perceptual
signal) and a reference signal increases from zero, the error signal rises
at first (as we always assume in the simple basic control model). Beyond a
certain magnitude of difference, the error signal begins to fall below the
strictly linear representation of error. At some still larger discrepancy,
say when the car is going 30 mph and the dog is going all-out at 25 mph, a
further increase in the discrepancy does not increase the error signal at
all, and then for still greater discrepancies, the error signal begins to
_decrease_.

It's that final decrease that leads to the giving-up phenomenon. The sign
of the feedback loop is negative for small errors. The loop gain for
changes goes to zero where the error signal levels out, and then becomes
positive, creating positive feedback for further increases.

The result will be that resistance to a small disturbance will be normal,
but as the disturbance continues to get larger, the resistance to the added
amounts of disturbance will become weaker, until for the largest
disturbances the resistance will collapse to zero. You can see that if at
any level an increase of disturbance causes a _decrease_ in the opposing
action, that will allow the error to be come even larger, which will lead
to greater error, which will decrease the action more, and this
regenerative process must lead to a total collapse in the opposition. When
opposition ceases, we call that "giving up." There is a much larger error
than usual, and no output action at all.

How does HPCT explain the concept of priority in this context? When
the rat switches [or is switched] to controlling a variable creating sexual
sensations, what mechanism preempts the other control processes?

"Priority" implies (among other things) an ordering in time, I think -- a
sequence. If you want to enter a house, you need to open the door and step
over the threshold. But if you try to do these things at the same time, you
will be in conflict, pushing at the door at the same time you try to unlock
it. By generating the associated reference signals in a specific sequence
-- unlock, then enter -- you prevent the conflict.

Prioritizing also arises in straightforward conflicts. If we want to do two
things at the same time, and they are mutually exclusive, a higher-order
system can raise the loop gain in one of the two systems and/or decrease
the gain in the other, so one system will prevail over the other one. Then,
if the winning system is the sort that doesn't need continuous effort to
keep its error small, the other system will be allowed to correct its
error. Here the sequencing isn't explicit, but the effect is the same: the
system that is given the higher loop gain gets to correct its error first,
and then quite naturally other other system acts when the first system's
error, and action drops to zero (or gets small enough). This doesn't work,
of course, if the first system requires continuous action to keep its error
from getting large again. In that case, its action would continue to
overwhelm any attempt by the other system to act.

Some people think that prioritizing is a simple intellectual act that we
can do just by stopping and thinking for five minutes. The problem is that
we have many goals and they are all important to us, although for different
higher-level and lower-level reasons. We can't just decide offhandto give
one goal priority over another, because when we try to turn off the other
goal, even temporarily, we will be disturbing all the higher systems in
which that goal plays a part. And those higher systems, whether we are
conscious of them or not, will automatically oppose the changes.

In fact, the need to prioritize always implies that there is some kind of
conflict. So if we can develop an effective way of helping people resolve
conflicts, we will bypass the need for prioritizing, which to my mind is a
questionable way of resolving conflicts.

Oh, one more thing. It's possible that certain phenomena, such as sneezes
and sexual arousal, _do_ involve positive feedback.

Best,

Bill P.

[From Bill Powers (2002.02.21.1949)]

Note interesting date and time coming up tomorrow. My daughter Allie
informs me that a situation like this, in a different notation, has come up
only once before and will never come up again (because no month will ever
have 33 or more days):

"When the 24 hour clock ticks on 8:02 p.m. on Wednesday, 20 February 2002,
then time, day, month, and year will read in perfect symmetry 2002, 2002,
2002. To be more precise - 20:02, 20/02. 2002."

It has happened once before: 10:01, 10/01. 1001

How many date-time stamps can there be in which there is only one kind of
numeral? For example, "2002.02.22.2222" in our customary format.

Bruce Abbott (2202.02.21.1730 EST)]

The whole concept of pleasure/displeasure is missing from PCT. Once a
control system gets organized, it just controls what it controls; it seems
unnecessary to add another variable, correlated with the controlled
variable, to represent its experienced pleasantness or
unpleasantness.

Try that the other way around. What do you mean when you say a perception
is "pleasant" or "good"? Simply proposing that there's some sort of
pleasantness signal or goodness signal isn't an answer, because then you'd
have to ask why such a signal is experienced as pleasant. Or, as the
masochist asked, "What's so bad about pain?" If you don't ask that, then
pleasantness is completely arbitrary and meaningless. So the problem
remains unsolved.

However, we can follow the trail the other way by asking what conditions of
the human control system are usually appreciated as pleasant (or the
opposite). We could propose that perceptions can be in any state from
unpleasant to pleasant, with unpleasant being defined as "far from
reference level" or "increasing error" and pleasant being defined as "
close to reference level" or "decreasing error." If the errors in question
are exclusively high-level, then the pleasure is abstract; if errors in
bodily states are included, the total picture is commonly spoken of in
terms of emotions.

This way of defining pleasant/unpleasant really involves the whole
hierarchy: it is the reaction of all systems at all levels to the state of
a particular perception. If we seek that state we call it "pleasant," and
its implications and uses are "good." If we avoid that state, the opposite.
Sometimes, I think, we judge the value of a perception by observing our own
actions and deducing that _something_ in us, even if we're not conscious of
what it is, is trying to avoid or escape from having that perception
(fingernails on a chalkboard), and we assign a "bad" value, cognitively or
otherwise, on that basis. I think that similar "good" experiences happen.

Indeed, many variables we control all the time seem to
have no subjective quality of pleasantness/unpleasantness associated with
them.

Try picking one of them, and then imagining or remembering (or creating) a
situation in which it is subject to a very large and continuing or
repeating disturbance. I think that the perception which was neutral under
undisturbed conditions will begin to acquire value when one has to act
energetically or with great skill to maintain control of it. Increasing
error then becomes alarming, and decreasing error satisfying -- values and
emotions creep in. So perhaps those value terms simply reflect the
conditions under which we prefer and prefer not to maintain control. Seems
to me that this proposition could be tested by a clever experimenter.

Best,

Bill P.

[From Mike Acree (2002.02.22.1303 PST)

Bill Powers (2002.02.21.1949)--

Note interesting date and time coming up tomorrow. My daughter Allie
informs me that a situation like this, in a different notation, has come up
only once before and will never come up again (because no month will ever
have 33 or more days):

"When the 24 hour clock ticks on 8:02 p.m. on Wednesday, 20 February 2002,
then time, day, month, and year will read in perfect symmetry 2002, 2002,
2002. To be more precise - 20:02, 20/02. 2002."

It has happened once before: 10:01, 10/01. 1001

At 9:12 p.m. December 21, 2112, we will have 21:12, 21/12, 2112--no?

Mike

[From Fred Nickols (2002.02.23.0925)] --

Rick Marken (2002.02.21.0820)

Fred Nickols (2002.02.21.0637) --

>Hmm. I would assume that maintaining a reference specification, especially
>in the face of disturbances, would indeed be reinforcing, no matter the
>nature of the reinforcer or the reference specification. In this case, the
>action seems to be occurring at a very low level in the hierarchy.

Bruce Abbott (2002.02.21.0955 EST)

> I find it interesting that the rats do have a reference value for this
> unnatural stimulation and that it is well above zero. But I'm puzzled by
> your last sentence above. How do you define a "very low level"?

I'm puzzled by Fred's first sentence: "maintaining a reference
specification...would indeed be reinforcing". What is "reinforcing"?
Rewarding?
Strengthening? Isn't the maintenance of a perception in a specified reference
state just called "controlling"?

Best

Rick

My comments merely reflect my notion that a behaviorist would interpret and
label what was going on as reinforcing or reinforcement at work (with
reinforcement defined as strengthening a response or increasing its
occurrence). "Rewarding" will do just as well. But, now that you ask, I
wonder, do we act to control our perceptions because we find it rewarding?

Regards,

Fred Nickols
740.397.2363
nickols@att.net
"Assistance at A Distance"
http://home.att.net/~nickols/articles.htm

[From Fred Nickols (2002.02.22.0935)] --

[From Bruce Abbott (2002.02.21.0955 EST)]

>Fred Nickols (2002.02.21.0637) --

>Hmm. I would assume that maintaining a reference specification, especially
>in the face of disturbances, would indeed be reinforcing, no matter the
>nature of the reinforcer or the reference specification. In this case, the
>action seems to be occurring at a very low level in the hierarchy.

I find it interesting that the rats do have a reference value for this
unnatural stimulation and that it is well above zero. But I'm puzzled by
your last sentence above. How do you define a "very low level"?

I was thinking of a low level in the PCT hierarchy and, darnit, you've
forced me to go back to the bible. (Okay, I'm back.) I guess I was
referring to "low level(s)" i.e., first and second-order (Levels 1 &
2?). Intensity control and sensation control.

Regards,

Fred Nickols
740.397.2363
nickols@att.net
"Assistance at A Distance"
http://home.att.net/~nickols/articles.htm

[From Bill Powers (2002.02.23.1107 MST)]

Rick Marken (2002.02.23.0830)--

>Fred Nickols (2002.02.23.0925)--
` >

My comments merely reflect my notion that a behaviorist would interpret and
label what was going on as reinforcing or reinforcement at work (with
reinforcement defined as strengthening a response or increasing its
occurrence). "Rewarding" will do just as well. But, now that you ask, I
wonder, do we act to control our perceptions because we find it rewarding?

Rick:

Yes, of course. We control perceptions that we have been rewarded for

controlling

in the past. Control of perception is a result of our history of

reinforcement.

This has been tested and shown to be statistically significant at the .01

level.

:wink:

Now, now. A little smiley doesn't make it all right to be sarcastic, does it?

The problem with the idea that reinforcements strengthen the response that
produces them is that you immediately have to introduce an ad-hoc patch on
this model to keep it from running the behavior rate up to the maximum
possible level. Something has to start inhibiting the effect of the
reinforcer just in the nick of time, so everything comes to the same level
of activity that is observed. There's no basis for predicting what that
level will be.

A behaviorist might say these things, Fred, but of course we're not
behaviorists, are we? We know that the behavior we observe is simply varied
as required to make the controlled consequence comne to the state that the
organism wants for it. Behavior is unimportant, except to an observer. To
the organism, it's the controlled perception that's important. This would
have become evident to behaviorists if they had made explicit disturbances
a standard part of their experimental conditions.

At some high level of organization, where we can perceive how well we are
controlling, there might be some sort of reference signal specifying good
control as a goal, in some contexts.

But I wouldn't say that we control well because controlling well increases
the probability of controlling well. Most PCTers I know would boggle at
that. It sort of skips over all the hard parts of devising a theory of
behavior and learning, doesn't it?

Would a behaviorist say that animals find that being reinforced by rewards
is reinforcing?

Best,

Bill P.

[From Norman Hovda (2002.02.21.1812 MSI)]

At 17:43 Bill wrote about Re: Controlling Rats on 21 Feb 2002

[From Bill Powers (2002.02.21.1641 MST)]

Bill Curry (2002.02.21.1220 EST)--

>I find it curious that the sexual sensation control system can be happily
perking
>along (presumably at zero error with zero sensation), and upon
stimulation, shift
>to a reference for high stimulation and shed other important control
>processes.

Me, too. But it does seem to work that way, doesn't it? I once proposed a

[snip]

Would not the addict, chasing the "endless orgasm", be an example?

Best,
nth