I've redesigned the homework a couple of times since Wednesday. Here's
what Dr. Kaner and I have come up with.

1) For class on Wednesday, evaluate the Johnson, et. al. paper using the
enclosed "Points to Notice" (or inspection checklist if you prefer that
term.) We'll discuss it in class.

2) FOr class on Wednesday, evaluate two additional papers that I just selected.
I'll leave a few copies in the Ph.D. lab (273 EC) on one of the tables if you
want them this weekend. I'll give the other copies to Karen on Monday morning.
The papers are:
Results of Applying the Personal Software Process, Pat Ferguson, et. al, IEEE
Computer, May 1997, page 24.
Automated incremental improvement of software product quality: a case history,
by Les Hatton. (It's out of a book that I borrowed from Dr. Kaner, and forgot
to
write down the title. Tsk, tsk.)

3) When I read them in detail, I realized that the other two papers I
passed out (Rothermel, et. al., and Graves, et. al.) were not case studies.
They were controlled experiments. So, we're not going to review them in
detail. But, please read the sections in each on threats to validity. Many
papers do not include such sections, and I want to hold these two up as
"shining star" good examples. But, while you are reading them, see if you
can find some other threats that they do not mention.

4) Sometime during the semester, read "The Soul of a New Machine", by Tracy
Kidder. It's a classic work, as well as a classic case study. Write an
evaluation of it using the "Points to Notice". We'll decide later when to
turn this in, but it will be later in the semester. If you want to buy it,
Barnes and Noble will probably have a couple of copies in a week or so. The
two copies that they had yesterday are now available to borrow from Dr.
Kaner. I checked out the copy from our library, and will keep it in 238. If
you want to read that one, come by and ask me.


Points to Notice for case studies:

1) Does it have data? (Remember that factual observations can be data, as
well as numbers.)
2) Can you get a copy of the data?
3) Does it identify a theory that is being examined? (Question and
propositions.)
4) Does it identify rival plausible theories that it is trying to distinguish?
5) Does the process of distinguishing the theories illuminate or cloud the
observations?
6) Does it identify one or more units of analysis?
7) Is there logic linking the data to the theory?
8) Does it state which, if any, theories are supported or counter-supported?
9) Does it clearly state the criteria for interpreting the observations and
findings?
10) Do you believe that the data supports (or counters) the theories?
11) For each conclusion, do you believe that the data supports it?
12) Is the theory relevant enough to a real problem that it is worth your
time to read the case study?
13) Can you think of alternate theories that were not presented that might
also account for the data?
14) Do the author(s) explicity address various threats to validity:
construct, internal, external, and reliability?
15) Do you see other threats to validity that the author(s) did not address?
16) Are any results of interest independent of the stated theories?
17) Wast here enough richness of detail?
18) In the methodology: What did they do? How did they do it? Why?
19) Did you learn anything?


---
Pat McGee