VARINT: quick comments

[From Bruce Abbott (941019.1650 EST)]

Bill Powers (941018.2250 MDT) --

Bill, I have compiled and run your VARINT program. I like your graphic
display--I was looking up the Pascal to do something similar in my program
(talk about RUSTY, it's been a while since I tried anything graphical!) when
yours arrived. Nice to have an example...

I see that your pigeon gets hungry if it doesn't get fed. This is certainly
an improvement on my pigeon which, oblivious to its own state of nutrition,
just goes on generating responses. In thinking about this simulation, I would
see this as the top level control in a hierarchy, the one that gets the pigeon
pecking at the keys. I guess you'd call the error signal of this level
"hunger." As hunger increases, so does the overall rate of pecking. However,
we'll be needing something more sophisticated to properly model the perceptual
variables being controlled by the lower level(s). The poor animal is still
switching keys at random and doesn't seem to care about the length of time
that has elapsed since last reward on a key, the increasing likelihood that
switching to the other key will bring instant error-reduction, or the effort
expended per reward on a key. In fact, the more I think about it, the more
complicated the pigeon gets. A really good simulation may even have to model
conflict from competing systems that "want" the pigeon to do something else
besides keypeck.

The current version (I know this is just a start) manages to let the pigeon
satiate during the experimental session, after which the pigeon only responds
as need be to make up for losses. Although satiation effects sometimes appear
in real experiments, the size of the food reward is usually small enough to
prevent all but small changes in hunger. This level of control definitely
belongs in the model, but it is the lower levels that will produce patterns of
behavior that resemble those of the real pigeons in real experiments. The
question is, just what perceptions are those critters controlling at that

Rick Marken suggested using a two-level model in which controllers at the
lowest level deal with perceptions of reinforcement rate and effort. I'm
thinking that something along these lines may work, but that something more
immediate than average reinforcement rate may be needed, such as estimated
time to next reinforcement being minimized.

Tom Bourbon [941019.1301]

A self-correction for part of my reply:

Tom Bourbon [941019.1000]

Tom, I'm really pleased to see your correcting yourself; it saves me from
having to do it! Seriously, by an odd coincidence I have not received the
aforesaid message that your were correcting. Meanwhile, quoting from you
quoting yourself:

I did not have a clear idea about how you think behavior is generated. I
still don't.

It's probably just my old EAB habits getting in the way of clear
communication. Does my above discussion about the pigeon simulation help?

Oops, I just checked and your missing earlier posting has finally arrived--
I'll reply later when I've had a chance to read it.

Until next time,