Your mission is to design a substantial user interface, carefully and thoroughly following the methods and principles from CS 352.
Guide your efforts by following the PRICPE process.
Project Parts 1, 2, and 3: Proposal: (PRICPE)
Due date: See the main class web page. For the electronic part, use Canvas.
Your proposal should include the following:
- Name of team members
- Project description (what do you want to do)
"P": predispositions. (From the perspective of factors that could affect USABILITY, what do you know, what do you NOT know?)
- This should include a description of this problem as a
- A justification why this is a good/interesting project from the standpoint of there being a reasonable amount of USABILITY work to think about
- Brief description of the target users
- What potential users will you have access to (being able to talk to potential users is REQUIRED).
Note: If you also have access to other stakeholders, that's a good plus, but note it doesn't replace being able to watch/talk to users who are like the application's target audience.
- Reasons you think you are the best team for this, and why
you'll be able to complete this before the end of the term
Typical length: About 1-2 pages for parts #1-#2, and about 1-2 pages for part #3.
Samples: Parts #1-#2, and Part #3.
Note about part #2: If you have doubts about the suitability of your project idea for this class, feel free to ask during office hours before the due date, so that you don't waste time developing an idea that I'm not likely to approve.
Grading Criteria for Proposal Parts 1 and 2
Graded by: Dr. Burnett, Total of 20 points
- 2 points: Name of team members
- 6 points: Description shows well how project is a USABILITY problem
- 6 points: Justification of why this is a good/interesting project from the standpoint of amount of usability work. (Common error: saying why interesting/important but not justifying reasonable amount of work: -4)
- 2 points: Description of target users
- 2 points: What potential users will you have access to (being able to talk to potential users is REQUIRED)
- 2 points: Reasons why you think you are the best team for this, and why you'll be able to complete this before the end of the term
Grading Criteria for Proposal Part 3
Graded by: Charles Hill, Total of 20 points
- 4 points: Do you actually KNOW what you say you know?(-2 for one wrong answer and -4 for 2 or more wrong answers)
- 12 points: At least 6 examples of knowing or not knowing useful UI/user related issues.
- 4 points: Completeness, did you miss any critical issues?
- -2 points for listing predispositions not about users or the UI the team is working with, e.g., generic statements like "increasing usability makes UIs easier to use."
Project Part 4: Your users (PRICPE)
Due date: See class schedule. For the electronic part, use Canvas
This is the "R", mainly from your formative empirical work, stemming from the "P" part of your proposal above. Note that since the process is iterative, it is indeed allowable to expand the "P" (research questions) as you progress with finding out more about your users.
The "R" will consist of in-the-field observations or in-the-field interviews of people who could be your users doing the task you are trying to support. (For example, if your users are supposed to be students planning their classes, then you should observe or field-interview students planning their classes. If your users are supposed to be grocery shoppers shopping, then you should observe or field-interview grocery shoppers shopping. Etc.) In addition, you may want to include other forms of research such as web research for additional concept ideas and so on, but we will not grade these other forms of research.
NOTE: Please avoid CS students as the users you observe/interview. You need experience learning to understand users not so much like you.
What to turn in:
- What are the research questions/goals (initially derived from the "P" in your proposal).
- Process: How did you proceed to answer these questions? Be detailed. For example, if you conducted an interview, list all the questions. Where and when did you collect the data? How did you set up to allow triangulation, etc.
- An exhaustive inventory of the elements we discussed in class:
- The people in the space
- 1. Who are they, what are they like?
- 2. What are they doing?
- 3. How are they doing it?
- 4. What do their emotions, purposes, reactions seem to be?
- 5. What problems do they encounter with their activities?
- The objects (technological and otherwise) in the space and with the people
- 1. What are the functional elements of the objects?
- 2. What are the decorational elements?
- 3. Which objects do people look for (perhaps to somehow interact with)?
- 4. Which objects do people bring with them that matter to the activities they are trying to do?
- The environment: spaces, architecture, lighting etc
- 1. What is the layout?
- 2. What is the environment like?
- 3. How does it influence the activities people engage in?
- 4. How does the environment support the objects above?
- Attach your raw data: detailed observations or interview responses (verbal and non-verbal)
- With the detailed observations/responses, point out the places that provide Results/Insights and say what they are ("I"):
- What are the answers to your research questions?
- What other insights did you get from this that are relevant to your design?
What am I looking for:
- A) That you included all the elements I asked you to.
- B) That your observations/interview data contains sufficient detail to notice something interesting, or which the average visitor would not normally see.
- C) That you derive some interesting design Insights from the data. Should be related to some real user need or activity that deserves support/attention.
- D) That you describe the experience factually, clearly, and in enough detail so that someone who has never been in the setting could clearly appreciate and understand what you saw.
Expected length: 6-10 pages, including words and sketches. For sketches, feel free to scan them in instead of drawing them on the computer.
Note: You can combine things into one file, or turn in separate files if it's easier (eg, a separate one for sketches?). The Canvas site will support several separate files.
Grading Criteria for User Data Assignment
Graded by: Charles. Total of 30 points
- 5 points: Suitable RQs/Goals
- -2.5 for no specific research questions/goals
- -2.5 if research questions weren't fairly thoroughly answered by the insights.
- 5 points: Process:
- -1 to -3 points for not enough detail
- -1 point for not listing questions on an interview
- -1 point for not mentioning triangulation
- -2 for not using a representative user
- If observation: 15 points: Observation that covers people, objects and environments as listed in assignment description. The write-up should have a good amount of detail (so I felt like I could picture the observation/interview).
- -1 Environment not suitable for observation/field study
- -1 user not performing task in ways they would normally
- -1 to -5 observation/interview not detailed enough to picture interview/observation
- If field interview: 5+5+5 points: pertinent questions, good field interview notes, good information about context & artifacts. Overall, the interview should involve the subject recounting his previous experience or perform some aspect of your project, with as much data as possible about the materials they used and the context they were in.
5 points: Good insights/understanding derived from your observation/interview. One point for each non-obvious insight/understanding.
- -2 for using mainly present tense
- -1 for not reporting non-verbal details
- 5 bonus points: Doing both field-interview and observation(Points will be reduced, based on the content, if additional interview/observation doesn't seem complete)
- As explained in class, we recommend 1 or 2 users, not large numbers of them; the term is not long enough for large numbers.
- If interview: Must be field interview, so your interview should involve the subjects recounting their previous experience or perform some aspect of the project. At least 2 points reduced if this requirement not met.
Project Part 5: Concepts and early Prototype #1 (PRICPE):
Your team will prepare a poster with the information below, to be presented at Design Studio 1, for brainstorming and feedback. See main class web page for the date.
Also turn in the pages from your poster electronically by 11:59 pm. Make sure everything turned in is readable. Use the Canvas hand-in page.
Present the following:
- A very brief summary of the problem your system is addressing and its users.
- At least 3 of your concepts (sketches, labeled with what Aspect each concept is exploring, as in lecture slides)
- A sketched screen transition diagram of the prototype
(see lecture slides for examples). Must have as at least part of it a (probably updated) concept you decided to pursue that you turned in as a concept sketch. Must include at least 4 screens that connect with transition arrows, but the screens do not need to be complete.
- For the design decisions you have made, include a justification based on your users and the user Research of each of your design decisions in the prototype.
Notes on constraints on your concepts/prototypes:
- Do not artificially constrain your interface ideas to the way a previous system worked or things that you know how to implement easily.
Instead, your design choices should have reasons. Should it be desktop/WIMP? Should it be Wearable? Should it be Tangible? Should it be in a Robot? Your concern is your users, not your developers! Remember to defend your decisions with Usability reasons. And remember to think about trade-offs: eg, the cost of a robot might not be feasible for on-line grocery shopping, so in this example you might eliminate a robot-based interface for reasons of the user's system purchase cost.
- Your UI does not have to be implemented in some other world/project/class. For example, if you're doing this on a senior design project, realize that this UI might be totally usable in that project, totally unviable for that project, or have influences on that project but not transfer directly. Do not constrain your UI to be a "deliverable" that another project could drop into place and use.
How you will be graded: This prototype will be graded based on how well you addressed the details I asked you for (see list in this subsection). This score will contribute a portion of your eventual grade for the prototype portion of the project.
Expected length: enough to fill up a poster, which will probably be 6-8 pieces of paper containing sketches, justifications, explanations, background. As usual, feel free to scan in sketches instead of drawing them on a computer.
Sample from a prior year (not exactly the same as this year's specs, but gives some reasonable ideas).
Give feedback to other teams here from Design Gallery #1 (until Feb. 12)
Grading Criteria for Design Gallery #1
Graded by: Charles, Total of 100 points
- 5 points: Description of problem.
- 10 points: At least 2 significantly different concept ideas. (-5 for similar designs, -5 for each missing design)
- -5 for similar designs
- -5 for each missing design
- 50 points: Sketches include the majority of the interface. Depending on how much of the interface is covered, partial points are given.
- 35 points: Justifications for major design decisions included. ( -15 if you only used general design principles to justify decisions. You need to justify
decisions based on YOUR users.) One important justification is why you chose to prototype that specific concept and not the other concepts.
- -15 if you only used general design principles to justify decisions. You need to justify decisions based on YOUR users.
- -5 for justifications not based on users OR design principles
- -10 if you didn't include why you chose to prototype that specific concept and not the other concepts.
- -5 for i-centered language (up to the first 4 times for a total of -20)
Project Part 6: Evaluation preparation (PRICPE):
Due date: See main class web page for the date. Use Canvas hand-in page
In the assignment after this, you'll need to evaluate an updated version of the prototype you presented in Design Gallery #1. This assignment is to do some preparatory work to help make that possible.
Heads-up: By the time of your actual analytical evaluation (next assignment), your prototype will need to be in Mockups (even if it is just a scan in of sketches with widgets/transitions added).
The updated prototype will need to support some user in your target population doing some task that's fairly central for your prototype (which we will call a scenario here). For example, if you are EmpCenter, your scenario might be "enter my hours for this week".
But for this assignment, all you need to turn in is:
- The scenario you plan to evaluate in the actual evaluation assignment. It should be a central task for your prototype (e.g., "enter my hours for this week"). Note: the scenario will need to be complex enough to eventually involve the user acting upon 4 different screen states (eg, pushing a button that changes the state, then filling in a text field and pressing ok, which changes the state, then clicking a link to a new screen, then pressing ok again).
- The following information about a member of your target population we will call "Abby":
- How old do you want your Abby to be? (Make her very different from your teams' ages unless the only people using your prototype are people your age. But under the age of 55.)
- What should her job title be? (Note: she is not a CS student unless you have special permission.)
- Where does she live?
Here is more information that will be true for every team's Abby:
- Abby has always liked music. When she is on her way to work in the mornings, she listens to music that spans a wide variety of styles.
- She likes puzzles and puzzle games (e.g., crosswords, Sudoku, pic-a-pix, etc.). She likes them on computers and she also likes working puzzles on paper.
- When she sits down to do a task, she likes to gather information first. For example, when she checks her email, she reads through all of the new ones before answering any to get an overall picture of what's going on.
- She is comfortable with the technologies she uses regularly, but she just moved to this employer/school/club/vehicle/state/etc 1 week ago, and the system you're prototyping is new to her.
- Abby says she's a "numbers person". She likes Math and knows how to think with numbers.
Looking ahead: You will be using this scenario, this version of Abby, and your updated Mockups prototype in your upcoming analytical evaluation assignment.
Grading Criteria for Evaluation Prep
Graded by: Dr. Burnett, Total of 3 points
- 1 point: Your scenario is a scenario that a user would be expected to perform using your prototype, and is fairly central to your project, as per the example above.
- 2 points: All the information I asked for about your Abby is provided and makes sense for your target audience.
Project Part 7: Analytical Evaluation with GenderMag CW (PRICPE):
Due date: See main class web page for the date. Use Canvas hand-in page
Perform a GenderMag Cognitive Walkthough (CW) on an updated version of the prototype you presented in Design Gallery #1
that is now in Mockups (even if it is just a scan in of sketches with widgets/transitions added).
If your scenario did turn out to involve at least 4 user actions that changed screen state, you'll be evaluating the scenario you turned in for the Evaluation Prep, from the perspective of the Abby you also turned in for the Evaluation Prep. (Otherwise, fix the scenario and Abby first.)
For example, if you are Augmented Reality Navigation team, you might be evaluating Abby driving from class to Starbucks on her bike using your system, which might involve the following 5 user actions:
- Abby getting the map going (eg, with a single voice command),
- Abby then pedaling her bike as per the map to the first turn,
- Abby then pedaling her bike as per the map to the second turn,
- Abby then having to turn quickly to avoid a huge obstacle she can now see through the glasses,
- Abby then turning her bike according to the map's rerouting.
To do the evaluation, follow the instructions in the highlighted GenderMag kit (Canvas).
Opportunity: Have this assignment be part of GenderMag research
Summary: If you wish, your work on this assignment can become part of the GenderMag research, by scheduling your work session in a particular lab in KEC. In exchange, each team member who does this will be paid $20 at the time they do the assignment. I won't be told who participated in the research vs. who just did the assignment on their own until after I turn in grades for Winter term. Details are here.
Hand-in instructions, what I'm looking for
Either way, here's what to turn in:
- List the team members who conducted this evaluation (must be at least two, but this is an extremely valuable exercise, so if all team members can do it, all team members will have their eyes opened).
- Your team's Abby (probably the same as the previous assignment, but updates allowed if needed)
- Your team's scenario (probably the same as the previous assignment, but updates allowed if needed)
- Your team's filled-out Cognitive Walkthrough (CW) forms showing your walk-through's results:
- If you record your CW on paper, then you can just scan in the paper forms.
- If you instead record using a computer, then you can still print+annotate+scan it if you want, or turn in whatever you recorded into during your evaluation: eg, Word document, pdf, Excel document, and/or any (optional) graphics files of screen shots that you took along the way of your evaluation (eg: jpg, tiff, png) if they are needed to make sense of what you wrote on your forms, ... But please do not turn in a zip file, as this may make things harder to grade using Canvas: upload each file separately.
- Your team's saved Mockups prototype that contains the part of the prototype you walked through.
What I am looking for:
- That you turned in everything I asked for, that it is a main-stream scenario for your project, and that it does show at least 4 user actions that each change the system's state
- That your forms contain enough detail that, in case someone later decided to make changes based upon your CW, the CW forms clearly show where you found the problem, why you think it's a problem. (And that your reason makes sense, of course.)
- That the forms show that you're definitely using Abby's perspective in your responses, not some other user. (Why: Because in a GenderMag session you represent only the one persona you chose for that session.)
Grading Criteria for Analytical Evaluation
Graded by: Charles, Total of 50 points
Note: the 4 highest-graded actions were considered for the assignment's final grade. All actions were graded; even if your team scored 100% on the CW portion, there may be actions within the walkthrough about which you received some feedback.
- 5 pts: At least 2 team members listed as having conducted the GenderMag session.
- -2.5 for not mentioning who performed evaluation
- 5 pts: Your team's Abby
- 10 pts: Your team's scenario
- -10 points if scenario isn't integral to prototype's function
- 30 pts: Your team's filled-out Cognitive Walkthrough (CW) forms showing your walk-through's results.
- -10 points for every action missing (must be at least 4 actions)
- -2 points for each action or subgoal whose response isn't focused on the user
- -X points for not enough detail (max -5)
Project Part 8: Empirical Evaluation (PRICPE):
Due date: See main class web page for the date. Use Canvas hand-in page
Perform a usability study, using the think-aloud technique, in which a user performs one or more tasks you give them on a portion of your Mockups prototype. As in other usability studies, your goal is to find out what usability problems this part of your prototype has. The user's task(s) must involve a total of at least 8 user actions.
What to turn in:
- The user(s):
- Turn in a brief description of the user(s) who performed the task, and how they fit your target population. (e.g., "A male college student (non-CS major) who likes to go to physical bookstores.")
- Turn in the think-aloud practice exercise you had the user do.
- Turn in the task instructions you gave the user (e.g., "Find one of the Jim Chee mysteries by Tony Hillerman").
- How you actually performed the study (eg, "We sat on the couch at the front of the bookstore to do the think-aloud exercise, and then we explained the task and handed our user a tablet computer with the prototype in the right state to begin the task. We all got up, and one of us walked with the participant around the bookstore as he performed the task using the prototype. That team member tried to take notes on paper at the same time, but that was hard since we were walking around. Since following the participant around made it hard to take notes, a second team member followed behind to take additional notes. This note taking was really difficult in these circumstances (walking around a bookstore, watching the user and the prototype, writing, and trying not to run into things), so we had rehearsed it first to work out who was responsible for gathering what kind of data."
- If you asked follow-up questions, used additional materials, etc., list them.
- Your prototype (so that we can make sense of your study and its results):
- Turn in your Mockups prototype.
- Also turn in drawings or a list of 8+ actions you were expecting/hoping the user to perform the task. (The users don't have to perform those exact same 8+ actions, but at least we can tell what you had in mind as a reasonable way to do the task(s).) You can print the screen sequence and annotate it with your pencil, or just write up a list of the actions like this: (1) Enter email address, (2) Click "ok" to give permission to share location data (3) Select "by author" on the next screen, (4) Type "Tony Hillerman", (5) Click "More criteria" on the next screen, (6) Click "By main character" on the next screen, (7) Enter "Jim Chee" on the next screen, (8) Follow the map to the first highlighted location it shows, (9) Click the location to show that they've arrived there, (10) Look at location on the shelf it highlights, (11) Grab the Tony Hillerman book and start browsing it.
- Data and results:
- A list of the problems you found. Be specific about what feature(s)/screen(s) were involved, and at what portion of the user's task.
- Insights from this study on fixes you need to make to your prototype.
- All the raw data you collected.
Samples: Sample #1. Sample #2. Note: the requirements for this assignment were different for these samples, but they will still give you some ideas.
Prototype #2 (Mockups) (PRICPE):
For brainstorming, feedback at our Design Gallery. See main class web page for the date.
Also turn in the pages from your poster electronically the same day.
Use the Canvas hand-in page.
(Although the runnable Mockups prototype needs to exist for your poster, you do not need to turn in the actual prototype.)
Present the following. All materials must be READABLE by human eyeballs. :-)
- A very brief summary of the problem your system is addressing and its users.
- A Mockups screen transition diagram of the complete prototype (printed out). (How to do this: "export to pdf" to get all your screens, draw arrows with your pen.)
Your users' "main" tasks should all be part of the prototype.
At least half of it should now be high-fidelity.
- A laptop with your Mockups prototype live.
- For the design decisions you have made since the last Design Gallery, include a justification based on your users, research and evaluation activities, and/or principles you've learned in class of each of your design decisions in the prototype. (Some of these you may have given before; if they came after Design Gallery #1, please include those here too.)
Sample. (Somewhat different specs that year and different prototyping tool, but still gives an idea.)
Give Design Gallery #2 feedback to other teams here (until March 12)
Final Prototype (Mockups) and Team Presentation:
Due date is on the main class web page.
Turn in your "runnable" Mockups prototype AND supporting documents electronically on the due date. Use the Canvas hand-in page.
In addition to the "runnable" Mockups prototype, the supporting documents are:
- An updated list of the elements from prototype #2 above (brief summary, Mockups screen transition diagram, justifications of decisions).
- Your powerpoint file from your 10-minute team presentation (you can update it to improve it before you finally hand it in), which emphasizes the above items. Namely:
- (Brief, eg 1-2 minutes): Start with the brief summary of the problem you are trying to solve and generally what kind of users you're targeting.
- (Brief, eg 2-5 minutes): Familiarize us with your Mockups prototype (can be a live demo, or can be static pictures in your powerpoints).
- (Most of your presentation): Spend most of your time telling us about the design decisions you made, explaining why you made each one.
Tie the decisions to the various Research/Feedback/Evaluations you have done and to CS 352 principles. Be specific about things you saw (eg in a GenderMag session or in a user observation) that led to your decisions, and specific about which principles and feedback led to what decisions.
- At the end of the 10 minutes, there will be a couple of minutes for the rest of the class to get their questions/notes together to give you feedback, while we transition to the next team.
Give teams feedback on their presentations here (until March 17)
Your final prototype will be graded on the strength of connection between usability design principles and your users with the decisions you made.
Be sure to include plenty of justifications from all three of: (1) design principles, (2) your analytical work (GenderMag), and (3) your user-based empirical work (early observations and later evaluation). Justifications based on other kinds of research/feedback are fine too, but don't neglect those first three.
In summary, the more justification for each design decision based on HCI principles and the "Research" and "Evaluation" aspects of PRICPE you have experienced, the better your grade.
Date of last update: March 9, 2017