Belated reply to Tom Hancock

[From Dick Robertson] (930803.1730CDT)

Tom Hancock You asked me a few weeks back if I had done any

higher level testing lately. Sorry I couldn't answer at the time,
my modem phone line was getting static for a couple of weeks there.
Anyway, the next project I attempted was a "grade control" study in
the intro psych course that I taught with my Intro to Modern Psyc
text. Bill Powers helped me analyse the loop that seemed to be
involved in a student's attempt to control their course grade, and
I set it up so that students would get a tutorial answer sheet to
grade their own tests after taking each one, and in addition, I
asked them to give three ratings when asking for each new test.
(They had to pass a test for each chapter before moving to the next
chapter; a different alternate form was given as often as needed.)
The three ratings were: How well prepared do you feel you are? How
well do you understand the material? How many questions do you
predict you will get right? -- All on 5-pt scales. (This is the
second year's questions, slightly modified from the first year.)

I used various combinations of the preparedness ratings, looking
for any combination that would correlate highly with the error
signal from each previous test. (ES defined as difference between
score they indicated they wanted & the score actually attained.)

I'm still analysing the data, but without any great enthusiasm.
The reason for that is that I can't find any hint of a common
pattern for all subjects, such as the guys who do tracking
experiments find. Some students do attempt to control their grades
in a definite way, according to what they told me. But different
ones use different strategies: some lower their reference level
for the course when they find it will take more study than they
have allocated time for; others do increase their study time; still
others attempt some compromise. Still others seemed to adopt a
strategy of "I'll take whatever I get," or what I called the
lottery approach to pursuing a course grade.

It seems subjectively clear to me that some students did attempt to
control their grade in the course by adjusting their study for each
next exam in terms of the overall exam average they needed and
taking into account the error score from each last exam. We got an
average of about 45 data points per student. But there was very
little variation in most of the measures. Their estimates of how
hard they studied tended not to vary much, nor did their
predictions about how many they would get right. So there isn't
much to work with.

As in the Self-as-control system study, a set of successive
approximations moving from a compromise "traditional/PCT mixed"
approach might eventually get to a real PCT experiment, as it did
in that case. I have some hunches about what it should be -- I
would say a tutor-text arrangement, in which each student can
answer a question after each bit of information and get referred to
a tutorial each time they get a question wrong, and also get a
running average of their percent correct to compare with their
stated RS. We examined this project at last year's conference and
agreed that it would need a lot of interviewing of individual
students during the course to get a better idea of how they went
about controlling their course grades. But, I didn't have the
manpower to do that in this year's courses. The compromise I
described about didn't come close enough. (And I have now retired
from Northeastern, so it's up to guys like you, if you want to try
it.)

#2 > also Tom Hancock. Greg Williams told me you are going to

use the textbook in a course this coming term. Congratulations and
felicitations! Did you know that I have at least 4 alternate 20-
items quizzes for each chapter, along with tutorial sheets that
explain the "right" answers? They have some bugs, but you are
welcome to them, if you are interested. Send me an address and a
blank disk for IBM, if you want them (preferably 1.4 meg). Best Wishes, Dick Rob
ertson