VI schedule performance

[From Bruce Abbott (990406.1500 EST)]

Bill Powers (990406.0926 MDT) --

Bruce Abbott (990405.1255 EST)

I need to regroup here. It's been too long since I had any data in front of
me and I'm misremembering things. Let's focus on the VI data you sent in
'97 and see if we can't at least talk about the same thing. I started
looking at it yesterday and it will take some time to figure out again what
it's about.

When I first recieved the data and your program, I wrote a modification of
your "cumulative record" display so I could look directly at the
interresponse times (the reciprocal of rate) versus time. Here's what I
read from the plots (eyeball estimates):

Run 97001.145 97001.149

Lower band 0.067-1 sec 0.067-0.7 sec
Upper band about 3-4 sec about 2 sec
total reinf. 210 87
total presses 4571 4997

The "lower band" and "upper band" are distributions of interresponse times
that remain essentially the same during the whole run. They are
distinguishable but not sharply separated. The 0.067-sec lowerf limit
reflects the 30/sec sampling rate. The upper band, as I interpret it,
reflects the collection time, which is clearly less when the total
reinforcements are smaller. This indicates to me that the rats are
consuming the pellets in more of a hurry when they are getting less food.
Also, there is an increase in mean rate of pressing, indicated by the
general shift to shorter interresponse times in the lower band.

On the other hand, I don't even know if these plots are for the same rat,
so all that may be spurious. How about reviewing for me the experimental
conditions and the meaning of the data?

It's been a while since I looked at those data, too; in fact, I'll have to
do some searching to find the folder in which I placed the information
giving which schedule values were programmed on which days (although I
believe that the schedule value does show up in the header of the computer
file of each day's run).

The data you show are from the same animal (97001) on sessions 145 and 149.

I'm interested in pursuing this with you, Bill, but at the moment I'm bogged
down with school-related work and with getting prepared for presenting the
cyclic ratio data at the Eastern Psychological Society convention in
Providence, RI next week. As you will recall, these are the data which show
that rats working on fixed ratio schedules (FR-2 up to FR-64) simply press
away at a relatively constant rate regardless of the ratio, with pauses in
lever-pressing following pellet delivery remaining about the same length on
average across ratios. So if you don't mind shelving this for a short
while, I'll take it up as soon as time permits. (I'm annoyed that it must
be put off as I really do want to go over this carefully with you.)

Regards,

Bruce

[From Bill Powers (990406.1640 MDT)]

Bruce Abbott (990406.1500 EST)--

I'm interested in pursuing this with you, Bill, but at the moment I'm bogged
down with school-related work and with getting prepared for presenting the
cyclic ratio data at the Eastern Psychological Society convention in
Providence, RI next week. As you will recall, these are the data which show
that rats working on fixed ratio schedules (FR-2 up to FR-64) simply press
away at a relatively constant rate regardless of the ratio, with pauses in
lever-pressing following pellet delivery remaining about the same length on
average across ratios. So if you don't mind shelving this for a short
while, I'll take it up as soon as time permits. (I'm annoyed that it must
be put off as I really do want to go over this carefully with you.)

OK -- when you get back we can look at the VI data.

A preliminary look at data histograms indicates that there is

(1) one peak, almost constant in amplitude over all interval settings, at
0.1 sec delay after reinforcement,

(2) a second peak with a variable height, at a constant delay between 0.2
and 0.3 sec for all conditions, and

(3) a third very low peak that is variable betwen 2 and 4 seconds, becoming
shorter as the interval increases.

The third peak delay is probably the collection time, which shortens as the
amount of food obtained becomes smaller.

It will be interesting to figure out what these fine-grained data mean. It
looks very much to me as if these rats just bang away at a constant rate,
too. It's harder to tell because of the superimposed random variations due
to the variable schedule.

Best,

Bill P.

[From Rick Marken (990406.2100)]

Bruce Abbott (990406.1500 EST)--

I'm bogged down with school-related work and with getting prepared
for presenting the cyclic ratio data at the Eastern Psychological
Society convention in Providence, RI next week. As you will recall,
these are the data which show that rats working on fixed ratio
schedules (FR-2 up to FR-64) simply press away at a relatively
constant rate regardless of the ratio, with pauses in lever-pressing
following pellet delivery remaining about the same length on average
across ratios.

I'm really glad to hear that you are doing this. Do you have a
copy of the paper? I'd would really like to read it. Also, have you
considered submitting it to JPCT? I would sure like to hear how
people at EPS react to the discovery that the results of over 60
years of operant research are spurious?

Best

Rick

···

--
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken/

[From Bruce Abbott (990407.1050 EST)]

Rick Marken (990406.2100) --

Bruce Abbott (990406.1500 EST)

I'm bogged down with school-related work and with getting prepared
for presenting the cyclic ratio data at the Eastern Psychological
Society convention in Providence, RI next week. As you will recall,
these are the data which show that rats working on fixed ratio
schedules (FR-2 up to FR-64) simply press away at a relatively
constant rate regardless of the ratio, with pauses in lever-pressing
following pellet delivery remaining about the same length on average
across ratios.

I have to laugh at myself here. The name of the organization is "Eastern
Psychological _Association_ (EPA). I have no idea why I typed "Society." (:-<

I'm really glad to hear that you are doing this. Do you have a
copy of the paper? I'd would really like to read it. Also, have you
considered submitting it to JPCT?

No copy yet -- as I said, I'm just getting around to preparing for this.

I would sure like to hear how
people at EPS react to the discovery that the results of over 60
years of operant research are spurious?

They are not going to discover that, because the results do not show that.
What the results show is that John Staddon's "regulatory" (control-system)
model of behavior on FR schedules is incorrect -- because the apparent
increase in overall rate of lever-pressing with increasing ratio requirement
is an artifact of the way average rate is computed. Staddon subsequently
proposed another model he calls "linear waiting time" that provides an
alternative account of the same data, without discounting the possibility
that either model (or indeed some other model) might be correct. (John likes
to throw out alternatives and then let subsequent research determine which
should be accepted or rejected.) However, my data do not support linear
waiting time either.

Regards,

Bruce

[From Rick Marken (990407.0930)]

Me:

I would sure like to hear how people at EPS react to the discovery
that the results of over 60 years of operant research are spurious?

Bruce Abbott (990407.1050 EST)

They are not going to discover that, because the results do not
show that.

Well, that's a relief!

What the results show is that John Staddon's "regulatory"
(control-system) model of behavior on FR schedules is incorrect --

But don't they also show that "reinforcement" has no effect on
"response" rate (which is the same in all conditions)? That
would be the conclusion if the rat is producing more food
("reinforcement") for itself in the low ratio than in the high
ratio condition, right?

Best

Rick

···

--
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken

[From Bruce Abbott (990407.1335 EST)]

Rick Marken (990407.0930) --

What the results show is that John Staddon's "regulatory"
(control-system) model of behavior on FR schedules is incorrect --

But don't they also show that "reinforcement" has no effect on
"response" rate (which is the same in all conditions)? That
would be the conclusion if the rat is producing more food
("reinforcement") for itself in the low ratio than in the high
ratio condition, right?

Not if you assume that response rates are "on the ceiling" of the rate
measure even at the highest ratio (FR-64). Reducing the ratio (and thus
reducing the minimum delay to pellet delivery) would be expected under
reinforcement theory to increase the lever-press rate, but not if it is
already at maximum to begin with. Supposedly, reinforcement has a stronger
effect on the "strength" of the operant the more immediately it is delivered
following the operant. This presumption implies a positive feedback
situation, in which higher rates of lever-pressing ----> shorter delay
-----> stronger reinforcement effect ----> higher rates of lever-pressing,
so that lever-press rates quickly ramp up to the maximum sustainable rate.

Of course, under PCT one would expect reducing the ratio requirement to have
the opposite effect: less delay ---> reduced error ---> lower lever-press
rate (assuming that the rat is controlling for a near-zero delay of
pellet-delivery), but we don't see that either, and could evoke the its own
form of "saturation" argument to explain why not (error remains high enough
even at FR-2 to generate maximum rates). I think there's a way to tease out
which account (if either) is correct, but this study leaves the issue
unresolved.

Bruce

[From Bill Powers (990407.1247 MDT)]

Rick Marken (990406.2100)--

I would sure like to hear how
people at EPS react to the discovery that the results of over 60
years of operant research are spurious.

That's rushing it. We haven't proven anything yet. I'm beginning to
suspect, however, that one model that might work over all schedules is

low hunger ---> miscellaneous spontaneous activities

medium hunger ---> press bar as fast as possible

high hunger ----> go into search mode for other sources of food

All degrees of hunger:
  food delivey ---> stop other actions long enough to eat it.

I don't think the rat's organization changes just because the schedule
changes from FR to VR to FI to VI. It's the same organization interacting
with different environmental properties to give what looks like different
behavior.
The goal, therefore, is to find a single model that explains behavior under
all schedules.

That's just what I suspect. I don't know anything yet.

Best,

Bill P.

[From Rick Marken (990407.2040)]

Me:

But don't they also show that "reinforcement" has no effect on
"response" rate (which is the same in all conditions)?

Bruce Abbott (990407.1335 EST) --

Not if you assume that response rates are "on the ceiling" of
the rate measure even at the highest ratio (FR-64).

In order to know this you would have to know that there _is_ a
relationship between response rate and reinforcement rate, right?
So there must be some "reference" data that shows this relationship,
right? But how do know that this "reference" data is free of the
very averaging artifact that you discovered?

Supposedly, reinforcement has a stronger effect on the "strength"
of the operant the more immediately it is delivered following the
operant.

Isn't this supposition based on data that is now highly suspect
based on the artifact that you discovered?

Of course, under PCT one would expect reducing the ratio
requirement to have the opposite effect: less delay ---> reduced
error ---> lower lever-press rate (assuming that the rat is
controlling for a near-zero delay of pellet-delivery)

I bet that the same relationship between ratio and press rate can be
produced by models that control all kinds of different perceptual
variables; rate of pellet delivery, probability of pellet delivery,
stomach fullness, etc. Indeed, the relationship between ratio and
press rate could even be produced by a cause-effect model that simply
generates presses at a rate dependent on some sensed aspect of the
schedule. I don't see how you can know whether or not a control
model applies to behavior until you _determine_ (not assume) whether
any variables are under control and, if so, what they are.

Best

Rick

···

---
Richard S. Marken Phone or Fax: 310 474-0313
Life Learning Associates e-mail: rmarken@earthlink.net
http://home.earthlink.net/~rmarken/

[From Bill Powers (990408.0253 MDT)]

Bruce Abbott (990407.1335 EST)]

Attached is my modification of your CUMREC.PAS, called CUMREC3P.PAS. I have
picked out two significant ways of presenting the data. One is a plot of
average interpress time versus time after food delivery. The other is a
histogram of number of presses per 1/30 sec bin as a function of time after
a food delivery.

The first one shows, I believe, the interval between presses as the delay
to the next food delivery (after the last one) gets longer. The average is
the appropriate measure because there are fewer long delays to the next
food delivery than there are short delays. What this graph shows is that
the rats begin delaying longer when the next food delivery doesn't show up
for a minute or two. I would interpret these long interpress times as time
spent away from the lever, rather than the rat standing in place with one
foot poised over it.

The second plot shows a histogram of number of responses in 1/30 sec bins
versus time since the previous food delivery. We get two distinctly
different plots for the two rats.

For 97002, there is a single peak at about 0.133 sec after food delivery,
showing an average rate of pressing of about 7.5 presses per second (if
I've figured that out right). This peak does not shift appreciably across
runs 145, 148, 149, and 150, the four I have (I trust that Bruce will copy
the data files to anyone who requests them). In this rat, an adequate model
would be one that presses steadily at a mean rate of 7.5 presses per second
with some statistical scatter about this mean, pausing only to collect the
food. The collection time varies with the amount of food being obtained,
from about 2 to 4 sec, with the fastest time occurring when the rat is
getting the least food. The only effect of the schedule is on collection time.

The other rat, 97001, shows similar effects except that there is a
pronounced second peak spanning 0.2 to 0.3 seconds delay, the amplitude of
which varies with the schedule. I would guess that this second peak
represents some idiosyncratic way of pressing the bar -- perhaps pawing at
it first with one forepaw (the first peak), then the other (which I have
seen in videos of rats pressing bars). A video would help. This second peak
stays at about the same delay across schedules, but the number of presses
at this delay changes with the schedule, the first peak rising and falling
the opposite way.

For both rats, I take the peak at 0.133 seconds to be caused by an
"overrun" -- the rats can't stop their repetitive pressing instantly when
the food appears, so there's always one extra press at the mean interpress
time later.

The peaks are spread out by (1) the fact that the apparatus enables the
reward trigger independently of the rat's lever presses, so that the press
that releases the food might occur at any time between one microsecond
after the trigger is enabled to one mean interpress time later, and (2) any
natural variation in the rats' mean rate of pressing. To test this model we
would need to generate artificial presses with an adjustable mean rate, and
an adjustable distribution, and run them through the same schedules. I
suspect that the pressing-rate distribution can be rather narrow, with most
of the scatter being provided by the variability in the interval schedule.
As a first approximation we could set the collection time to 3 seconds and
see how the histogram looks.

There is certainly a collection-time effect in these data, and very little
effect of schedule on rate of pressing, if any. Or so it appears now.

When you get back from the meeting, I hope we can work on refining this
preliminary interpretation of the data.

Best,

Bill P.

program CumRec3p;
{ Plots a cumulative record based on data files created by the FRATIO
  and RANDOMVI real-time experiment programs.

  Created 1/2/98 by B. Abbott

  More plots added by WTP
}

uses dos, crt, frdata, graph, grutils;
const MAXDATA = 4500;

var
  ch: char;
  Filename, Path: pathstr;
  bin: array[0..MAXDATA] of integer;
  irtavg: array[0..MAXDATA] of longint;
  nirt: array[0..MAXDATA] of longint;
  R, P: dataptr;
  RTon, RToff, PTon, PToff, Event: timesptr;
  i, j, nR, nP: word;
  NWords, SubjYr, SubjNo, Year, Month, Day, Session, Ratio,
  Hour, Minute, Resp, Rft: word;

procedure InitScreen;
begin
  initgraphics;
  graphmode := getmaxmode;
  setgraphmode(graphmode);
  hsize := getmaxx + 1;
  vsize := getmaxy + 1;
  hcenter := hsize div 2;
  vcenter := vsize div 2;
end;

procedure WriteData(nD: word; On, Off: timesptr);
var
i: word;

begin
  writeln(nD:10);
  for i := 1 to nD do
    begin
      write(On^[i]:8, Off^[i]:8);
      if (i Mod 4 = 0) then writeln;
    end;
  writeln('Successfully wrote ', nD, ' values');
end;

function IRT(i: word; On, Off: timesptr): longint;
begin
  if i = 1 then IRT := On^[i] else IRT := On^[i] - On^[i-1];
end;

function DUR(i: word; On, Off: timesptr): longint;
begin
  DUR := Off^[i] - On^[i];
end;

procedure PlotlogIRT;
var
  i, j, x, y,color: word;
  yr: real;
  istr: string;
begin
  str(Session, istr);
  outtextxy(10,35, 'Sess = '+istr);
  str(nR, istr);
  outtextxy(10,50, 'Rsp = '+istr);
  str(nP, istr);
  outtextxy(10,65, 'Rft = '+istr);
  y := Vsize;
  j := 1;
  for i := 1 to nR do
    begin
      y := vsize - round(50.0*ln(IRT(i,RTon,RToff)));
      x := (RTon^[i] * Hsize) div 72000;
      putpixel(x,y, white);
    end;
  j := 1;
  for i := 0 to 8
    do
    begin
      if i > 0 then j := j + j;
      y := j;
      str(y/30:4:3,istr);
      yr := ln(j);
      outtextxy(hsize - 100,vsize - 8 - round(50*yr),istr);
    end;
    outtextxy(hsize - 100, 100, 'IRT, sec');
    outtextxy(vcenter,0,'LOG IRT VERSUS TIME');
end;

procedure IRTHisto;
var basetime,maxtime: longint;
    x,y,i,j,k: integer;
begin
  clearviewport;
  basetime := RTon^[1];
  maxtime := 0;
  j := 1;
  for i := 0 to 3000 do bin[i] := 0;
  for i := 1 to nR do
    begin
      if RTon^[i] >= PTon^[j] then
        begin
          inc(j);
          basetime := RTon^[i];
        end;
        k := IRT(i,RTon,RToff);
      inc(bin[k]);
      if maxtime < k then maxtime := k;
     end;
  clearviewport;
setcolor(darkgray);
for i := 1 to 30 do
  line(round(100*ln(i)),vsize,round(100*ln(i)),vsize - 360);
  setcolor(white);
  for i := 1 to 320 do
  begin
   y := bin[i];
   if i = 1 then moveto(round(100*ln(i)),vsize - y)
   else
   lineto(round(100*ln(i)),vsize - y);
  end;
setcolor(white);
line(0,vsize - 100,360,vsize - 100);
line(0,vsize - 200,360,vsize - 200);
line(0,vsize - 300,360,vsize - 300);
outtextxy(370,vsize - 100,'100');
outtextxy(370,vsize - 200,'200 PRESSES');
outtextxy(370,vsize - 300,'300');
outtextxy(0,vsize - 370,'0.033');
outtextxy(round(100*ln(3)) - 11,vsize-370,'0.1');
outtextxy(round(100*ln(6)) - 11,vsize-370,'0.2');
outtextxy(round(100*ln(9)) - 11,vsize-370,'0.3');
outtextxy(round(100*ln(15)) - 11,vsize-370,'0.5');
outtextxy(round(100*ln(30)) - 11,vsize-370,'1.0');
outtextxy(360,vsize - 370,'sec');
outtextxy(100, 0,' HISTOGRAM');
outtextxy(100,20,'IRT, 1/30 SEC INTERVALS AFTER RFT');
outtextxy(100,40,' LOG TIME SCALE');
outtextxy(100,60,filename);
end;

procedure IRTTime;
var basetime,maxtime: longint;
    x,y,i,j,k,m,maxm: word;
    avgrate: real;
    firstp: boolean;
begin
  basetime := RTon^[1];
  maxtime := 0;
  maxm := 0;
  firstp := false;
  j := 1;
  for i := 0 to MAXDATA do irtavg[i] := 0;
  for i := 0 to MAXDATA do nirt[i] := 0;
  for i := 1 to nR - 1 do
    begin
      if RTon^[i] >= PTon^[j] then
        begin
          inc(j);
          basetime := RTon^[i];
          if not firstp then firstp := true;
        end;
     if firstp then
     begin
      {Fill in all histogram slots during IRT}
      for m := RTon^[i] - basetime to RTon^[i+1] - basetime - 1 do
       begin
        if m > 2999 then continue;
        irtavg[m] := irtavg[m] + RTon^[i+1] - RTon^[i];
        inc(nirt[m]);
       end;
      if maxtime < (RTon^[i] - basetime)
      then maxtime := (RTon^[i] - basetime);
     end;
    end;
  for i := 1 to maxtime do
  if nirt[i] > 0 then
   irtavg[i] := round(irtavg[i]/nirt[i]);
  for i := 1 to maxtime div 5 do
  begin
   if nirt[5*i] > 0 then
    begin
     if i = 1 then moveto(i,vsize - irtavg[5*i] div 4)
     else lineto(i,vsize - irtavg[5*i] div 4);
    end;
    putpixel(i,vsize-1,white);
    if (i mod 30) = 0 then line(i,vsize,i,vsize - 10);
  end;
  outtextxy(2*hsize div 3, vsize - 20,'10 second marks');
end;

procedure PlotIRTs;
var
  i, j, x, y: word;
  mark: boolean;
  istr: string;
begin
  outtextxy(hcenter-80, 10, 'IRT vs Response Count');
  outtextxy(hcenter-75, 25, 'red = post reinf IRT');
  str(Session, istr);
  outtextxy(10,35, 'Sess = '+istr);
  str(nR, istr);
  outtextxy(10,50, 'Rsp = '+istr);
  str(nP, istr);
  outtextxy(10,65, 'Rft = '+istr);
  outtextxy(hcenter-30, vsize-80, 'Time');
  line(0, vsize-99, hsize, vsize-99);
  line(0, vsize-400, 0, vsize-100);
  for i := 0 to 300 do
    if (i mod 30 = 0) then line(0,vsize-100-i, 6, vsize-100-i);
  j := 1;
  mark := false;
  for i := 1 to nR do
    begin
      x := (RTon^[i] * Hsize) div 72000;
      y := vsize - 100 - (IRT(i, RTon, RToff) div 1);
      if (mark and (IRT(i, RTon, RToff) > 60)) then
        begin
          setcolor(lightred);
          circle(x,y,3);
         { putpixel(x, y, lightred );}
          mark := false;
        end else putpixel(x, y, white);
      if RTon^[i] = PTon^[j] then
        begin
          inc(j);
          mark := true;
        end;
    end;
end;

procedure PlotIt; { cum record }
var
  i, j, x, y: word;
  istr: string;
begin
  str(Session, istr);
  outtextxy(10,35, 'Sess = '+istr);
  str(nR, istr);
  outtextxy(10,50, 'Rsp = '+istr);
  str(nP, istr);
  outtextxy(10,65, 'Rft = '+istr);
  y := Vsize;
  j := 1;
  for i := 1 to nR do
    begin
      dec(y); if y = 0 then y := Vsize;
      x := (RTon^[i] * Hsize) div 72000;
      putpixel(x,y, white);
      if RTon^[i] = PTon^[j] then
        begin
          inc(j);
          line(x,y, x+5,y+5);
        end;
    end;
end;

begin
  ClrScr;
  Path := 'd:\tp\ratexp\vidata\';
  Filename := '97002.145';
  Filename := Path + Filename;
  if FileExists(Filename) then
    begin
      InitScreen;
      new(R); new(P);
      new(RTon); new(RToff);
      new(PTon); new(PToff);
      writeln('Reading File ', Filename, ' . . .');
      ReadData(Filename, SubjYr, SubjNo, Session,
            Ratio, Year, Month, Day, Hour, Minute, Nwords,
            Resp, Rft, R, P);
      writeln('Decoding data . . .');
      Decode(R, Nwords, RTon, RToff, nR);
      Decode(P, Nwords, PTon, PToff, nP);
      IRTTime;
      ch := readkey;
      IRThisto;
      ch := readkey;
      dispose(PToff); dispose(PTon);
      dispose(RToff); dispose(RTon);
      dispose(P); dispose(R);
     CloseGraph;
    end
  else
    begin
      write('File Not Found . . .');
      ch := readkey;
    end;
end.