CS 564, Winter 2021
CS 564 FIELD STUDIES IN SE AND HCI
Instructor: Dr. Burnett
Dr. Burnett's Office Hours and Office Location are listed on my home page
Class location: Canvas and Zoom
The course is about a particular empirical method in understanding humans' use of languages, environments, etc., in software development.
There are many possibilities about empirical methods, and we can't cover them all. Thus, this course will focus on a method that is very useful for certain kinds of questions about the HCI of software development, but
is not well understood in computer science:
how to scientifically conduct and analyze case studies.
Note that this is not a statistics course, although we might touch on a couple of basic stats. If what you really want is a statistics-for-experiments course, I recommend the Stats 511 and/or 515 courses, which are excellent, and are specifically targeted to non-stats grad students.
You can think of this as a "research methods" course, focusing on the research method of doing the type of empirical work known as case studies. The goals are that by the end of this course, you will be able to:
- Choose when a case study is the right choice of empirical work.
- Design, conduct, and gather data in case studies...
- ... according to accepted ethical principles of dealing with human subjects.
- Analyze data in case studies using qualitative methods.
- Report case study empirical work in research publications.
Every year that I've taught this, one or two of the teams have decided to submit revised versions of their final case studies to research conferences. It's my understanding that everyone who has done this has eventually had their papers accepted (but not always on the first try).
Contrary to what the catalog or scheduling system might say, the only prerequisite for this course is grad standing in Computer Science.
Mastery of English and Qualitative Empirical Work
This course's data will be qualitative -- examples are what people said during a design meeting, what they wrote in their bug reports, and so on. Not quantitative (i.e., not numbers of log-ins, etc.)
This type of data will often be subtle and nuanced.
Because of this, strong English skills are a must.
Those who struggle with English will find this course problematic.
How the course will be conducted, method of instruction
You'll actually do a case study involving some aspect of software development, with a team member.
I'm anticipating no programming in this course.
There will be some lectures by me, but probably fewer than half of the classes will be like this. Much of the class will be studio style, with teams critiquing each others' work, and based on team presentations in which the class jointly provides feedback on some aspect of a team's case study. In short, it will be highly interactive.
Due to the interactive style, class size is limited.
There will be one midterm, a number of interim assignments, and a final presentation of your case study.
There is no final exam.
I have high expectations, and expect performance worthy of graduate students in computer science. Thus, in this class, "A" does not mean "adequate" or "nothing wrong" -- it means "excellent". For an A, you should expect to dig deep and get the most you can out of the class.
- Midterm: total = 37.5%
- Assignments, presentations, other contributions to class discussions = 25%
- Final case study = 37.5%
- Required, ISBN/SKU 978-1118104354, Per Runeson et al., Case Study Research in Software Engineering: Guidelines and Examples, 2012, Publisher: Wiley. Hardcopy on order at the OSU Bookstore. Hardcopy and kindle also available at amazon at a variety of prices. E-versions also available from other vendors, but --
electronic version is available free via the OSU Library. Check the course reserves if you have trouble finding it.)
We will also have selected readings from other sources, but you don't have to buy those.
- Example single holistic case study (conducted in cs569/cs564) that got published: Software Visualization for End-User Programmers: Trial Period Obstacles, Neeraja Subrahmaniyan, Margaret Burnett, and Christopher Bogart, ACM Symposium on Software Visualization, Herrsching am Ammersee, Germany, Sept. 2008, pp. 135-144.
- Classic case study example: the Seaman/Basili ICSE'97 paper
- A selection of good case study papers.
- Example think-aloud study described in week #1 lecture.
- Examples of multiple case studies.
- Field observation examples: Observing people in E-cafe (done as a cs 564 assignment), Observing teachers in system tutorial.
- Beki Grinter's example recordings.
- Beki Grinter's examples of open, axial, and selective coding.
- Steve Krug's demo (pertains to both interviewing and prototype evaluation). (local WMV. local MOV)
- Clarification on propositions vs hypotheses.
- Self-efficacy questionnaire (as instantiated for spreadsheets).
Explaining Debugging Strategies to End-User Programmers,
Neeraja Subrahmaniyan, Cory Kissinger, Kyle Rector, Derek Inman, Jared Kaplan, Laura Beckwith, and Margaret Burnett,
IEEE Symposium on Visual Languages and Human-Centric Computing, Couer dAlene, Idaho, Sept. 23-27, 2007, pp. 127-134.
Supporting End-User Debugging: What Do Users Want to Know?
Cory Kissinger, Margaret Burnett, Simone Stumpf, Neeraja Subrahmaniyan, Laura Beckwith, Sherry Yang, and Mary Beth Rosson, Proceedings Advanced Visual Interfaces, ACM Press, Venice, Italy, May 2006.
Margaret M. Burnett
Date of last update: Jan. 4, 202