PCT instructional design --- a proposal

CSGnet Archive CSG1997
1

[From Robert Kosara, 970131.18:30 MET]

in Re: Bruce Gregory, Bill Powers

What I had to first confront is the fact that I am
dealing with twenty autonomous control systems with very
different reference levels and very different histories.

  I am not sure if that is what you are looking for, but ...
being a student, and I want to propose an entirely different
approach, throwing together ideas from many different places: let's
see what this leads to ...

  Let's face it, students' main controlled variables are their marks. And
even if they're interested (many are, of course), they don't know enough
about the subject to be able to answer the question 'What do you want to
know?', especially in a course on such a basic subject. You are the one to
decide.
  So let's turn the model around, and make you the environment, and the
students control their perceptions (= marks and knowledge). You start
with giving every single student an A, all they have to do is keep it
(thus keeping the error [:= 'distance from A'] at zero [this is stolen from
'Zen and the Art of Motorcycle Maintenance' by Robert Pirsig]). You
provide the facts and the feedback, with as little delay as possible (a day
or two, but not two weeks), and provide the students with feedback in
short intervals (so that they can start controlling when the error is
still small).

easiest to learn something when the subject is clear, and there is no
'we will eventually come to that, but I can't promise'. No matter how
difficult the subject is, when I get feedback continuously, instead of
once (at the very end of the term), it is a lot easier to identify
difficulties and see the holes in my knowledge. Thus, I learn more and
get better marks!
  I really like what I am studying, and I want to know things. But I
don't know enough yet to know what I want to know. The lecturers have
to provide the subjects, I control by selecting the lectures I take!
That, of course, means I have to select what I want to know anyway, but
there are compulsory lectures, and my selecting is easier when I can choose
a field instead of a single (small) part of one.

  In the first lecture on graph theory, a student asked what subjects will
be covered in this lecture. The lecturer answered that it wouldn't make sense
telling him, for he wouldn't understand what directed and weighed graphs are,
what a path is and what you can use the methods of graph theory for. If he
had asked us to tell him what we wanted to know, that would have been a very
short lecture ...

  This may be complete nonsense, I can only speak from my personal experience;
but if it is, please don't just delete this message, but tell me!

   Robert

···

From my experience (that also applies to many of my colleagues), it is

***************************************************************************
Remember: Do not underestimate the power of the Farce.
***************************************************************************
   _ PGP welcome! email: rkosara@wvnet.at
  /_)_ / __ __ 7 //_ _ __ __ __ or: e9425704@student.tuwien.ac.at
/ \(_)/_)(- / / /\(_)_\(_// (_/ http://stud2.tuwien.ac.at/~e9425704/
Student of Computer Science at the University of Technology Vienna, Austria
***************************************************************************

2

[From Bruce Gregory (970201.0735 EST)]

Robert Kosara, 970131.18:30 MET

  So let's turn the model around, and make you the environment, and the
students control their perceptions (= marks and knowledge). You start
with giving every single student an A, all they have to do is keep it
(thus keeping the error [:= 'distance from A'] at zero [this is stolen from
'Zen and the Art of Motorcycle Maintenance' by Robert Pirsig]). You
provide the facts and the feedback, with as little delay as possible (a day
or two, but not two weeks), and provide the students with feedback in
short intervals (so that they can start controlling when the error is
still small).

An associate taught a high-school course in physics. In one class he passed
back homework assignments and tests the next day. In the others he passed back
homework assignments and tests after a week to ten days. He was dismayed to
find that both groups did equally well (or poorly). It makes sense that rapid
turnaround would be superior, but he could find no evidence to support the
notion. My motto is, "the feedback comes from the task, not the teacher."
This seems to hold whenever we really learn something and is consistent
with "feedback" in the PCT sense.

From my experience (that also applies to many of my colleagues), it is
easiest to learn something when the subject is clear, and there is no
'we will eventually come to that, but I can't promise'. No matter how
difficult the subject is, when I get feedback continuously, instead of
once (at the very end of the term), it is a lot easier to identify
difficulties and see the holes in my knowledge. Thus, I learn more and
get better marks!

  I really like what I am studying, and I want to know things. But I
don't know enough yet to know what I want to know. The lecturers have
to provide the subjects, I control by selecting the lectures I take!
That, of course, means I have to select what I want to know anyway, but
there are compulsory lectures, and my selecting is easier when I can choose
a field instead of a single (small) part of one.

The students I have are somewhat of a special case. They are preparing
to teach precollege science and have undergraduate degrees in a science.
I makes sense to ask them what they want to know because my goal is
to support them in making the transition from student to teacher and to
reflect on what teaching science means to them. Otherwise I agree with
your comments.

Bruce Gregory

3

[From Bill Powers (970201.0745 MST)]

Bruce Gregory (970201.0735 EST)--

An associate taught a high-school course in physics. In one class he
passed back homework assignments and tests the next day. In the others he
passed back homework assignments and tests after a week to ten days. He
was dismayed to find that both groups did equally well (or poorly).

Why dismayed? He had found that if there is any benefit from letting the
students see their graded assignments quickly, a delay as short as one day
is still too long to reveal it. And in both cases, the assumption was that
the students looked at something on the returned papers beside the grade --
perhaps he was acquiring evidence that students were not reviewing their old
homework assignments to study and correct their errors, or at least not
changing whether they did this or not. Students who reviewed their homework
errors did so in either case, and those who didn't, didn't.

If you really want to study the effect of "delayed feedback" on quality of
work, you have to explore all delays down to zero, or as close as you can
get to zero. One day is not the minimum possible time. Perhaps the critical
amount of delay is 10 seconds. One of the data points should involve having
a student solve one problem, then grading it immediately and handing it
back. Then the next, and so on.

The problem with doing a REAL research project is that when teachers see
what it entails, their natural response will be "But that's impossible, I
can't do that with a class of 30 people! We'd spend all our time grading
papers and talking about them, instead of getting on with the course." When
I suggest research procedures to people, it often turns out that if my
proposals are really what is necessary to arrive at a good answer, the
people lose interest in the answer. They have all sorts of reasons for going
on doing what they were doing before. If they can't get the answer easily,
they don't want to know it. And of course the last possibility they will
consider is that they're boring the students to death, and that all most of
the students want is a C and out. Maybe they should consider a different
profession. Of course the teachers' union wouldn't let you do a research
project that might result in that answer.

It makes sense that rapid
turnaround would be superior, but he could find no evidence to support the
notion.

Why should rapid turnaround make any difference? Behind any such assumption
are others, such as that the students actually try to find their errors and
re-do the problems until they know they can get the right answer every time.
If they aren't doing that, then there's no reason to suppose the turnaround
time will have any effect at all, even if instantaneous. These things don't
work by magic.

From my experience (that also applies to many of my colleagues), it is
easiest to learn something when the subject is clear, and there is no
'we will eventually come to that, but I can't promise'. No matter how
difficult the subject is, when I get feedback continuously, instead of
once (at the very end of the term), it is a lot easier to identify
difficulties and see the holes in my knowledge. Thus, I learn more and
get better marks!

That is, provided you want to learn better. If all your students really want
to understand a subject, you don't have much of a problem to begin with --
anything will work. But if they don't, what does any theory of instruction
tell you to do about that? Threaten them?

The students I have are somewhat of a special case. They are preparing
to teach precollege science and have undergraduate degrees in a science.
I makes sense to ask them what they want to know because my goal is
to support them in making the transition from student to teacher and to
reflect on what teaching science means to them. Otherwise I agree with
your comments.

If I were talking to kindergartners with the idea of teaching them to read,
I would start by asking them "What stories do you think are really neat?"
I'd ask them if there's anything they like to be read to out of. I'd ask
them what they do when there's nobody to read it to them. I'd ask them if
they think reading would be hard to do. I'd ask them how they think reading
works. And of course I'd ask them if they'd like to know how to read. If you
could just get a discussion going, I think it would be pretty obvious what
various people need to learn next.

At least that's more or less what I THINK I'd do. I have no idea, to be
honest, what I'd be doing after a semester of trying to teach kindergartners
to read. I might be recuperating in a hospital. My only experience as a
language teacher occurred when I was about five, and wanted my little sister
to quit pestering me to read the comics to her. So I taught her to read them
herself. She picked it right up, for which I can take no credit; she always
was a pretty smart kid, and still is.

Best,

Bill P.

4

[From Bruce Gregory 9970201.1210 EST)]

Bill Powers (970201.0745 MST)

Why dismayed? He had found that if there is any benefit from letting the
students see their graded assignments quickly, a delay as short as one day
is still too long to reveal it. And in both cases, the assumption was that
the students looked at something on the returned papers beside the grade --
perhaps he was acquiring evidence that students were not reviewing their old
homework assignments to study and correct their errors, or at least not
changing whether they did this or not. Students who reviewed their homework
errors did so in either case, and those who didn't, didn't.

I agree. He was dismayed because the students had pleaded with him to
return their work rapidly, and that was as rapidly as he could manage.

If you really want to study the effect of "delayed feedback" on quality of
work, you have to explore all delays down to zero, or as close as you can
get to zero. One day is not the minimum possible time. Perhaps the critical
amount of delay is 10 seconds. One of the data points should involve having
a student solve one problem, then grading it immediately and handing it
back. Then the next, and so on.

Yes. I heard someone describe his high school math course that was
conducted in exactly this way. It worked fine in his case!

The problem with doing a REAL research project is that when teachers see
what it entails, their natural response will be "But that's impossible, I
can't do that with a class of 30 people! We'd spend all our time grading
papers and talking about them, instead of getting on with the course." When
I suggest research procedures to people, it often turns out that if my
proposals are really what is necessary to arrive at a good answer, the
people lose interest in the answer. They have all sorts of reasons for going
on doing what they were doing before. If they can't get the answer easily,
they don't want to know it. And of course the last possibility they will
consider is that they're boring the students to death, and that all most of
the students want is a C and out. Maybe they should consider a different
profession. Of course the teachers' union wouldn't let you do a research
project that might result in that answer.

I'm somewhat more optimistic. I think most people want to learn _something_.
I agree that most don't want to learn what we want to teach!

At least that's more or less what I THINK I'd do. I have no idea, to be
honest, what I'd be doing after a semester of trying to teach kindergartners
to read. I might be recuperating in a hospital. My only experience as a
language teacher occurred when I was about five, and wanted my little sister
to quit pestering me to read the comics to her. So I taught her to read them
herself. She picked it right up, for which I can take no credit; she always
was a pretty smart kid, and still is.

In the good ole days many people learned to read by sitting on
a parents lap who read the Bible to them. Talk about instant "grading".
It seemed to work out fine.

Bruce Gregory