Your mission is to design a substantial user interface, carefully and thoroughly following the methods and principles from CS 352.
Guide your efforts by following the PRICPE process.
Project Parts 1, 2, and 3: Proposal: (PRICPE)
Due date: See the main class web page. For the electronic part, use TEACH.
Your proposal should include the following:
- Name of team members
- Project description (what do you want to do)
- This should include a description of this problem as a USABILITY problem
- A justification why this is a good/interesting project from the standpoint of there being a reasonable amount of USABILITY work to think about
- Brief description of the target users (they CANNOT be Computer Science students
- What potential users will you have access to (being able to talk to potential users is REQUIRED).
Note: If you also have access to other stakeholders, that's a good plus, but note it doesn't replace being able to watch/talk to users who ARE like the application's target audience and ARE NOT Computer Science students.
- Reasons you think you are the best team for this, and why you'll be able to complete this before the end of the term
- "P": predispositions. (From the perspective of factors about your USERS that could impact the USABILITY of the system: what do you know, and even more important, what do you NOT know?)
Typical length: About 1-2 pages for parts #1-#2, and about 1-2 pages for part #3.
Samples from prior years (under slightly different requirements): Parts #1-#2, and Part #3.
Note about part #2: If you have doubts about the suitability of your project idea for this class, feel free to ask during office hours before the due date, so that you don't waste time developing an idea that I'm not likely to approve.
Tentative Grading Criteria for Proposal Parts 1 and 2
Graded by: Doshna, Total of 20 points
- 2 points: Name of team members
- 6 points: Description shows well how project is a USABILITY problem
- 6 points: Justification of why this is a good/interesting project from the standpoint of amount of usability work. (Common error: saying why interesting/important but not justifying reasonable amount of work: -4)
- 2 points: Description of target users
- 2 points: What potential users will you have access to (being able to talk to potential users is REQUIRED)
- 2 points: Reasons why you think you are the best team for this, and why you'll be able to complete this before the end of the term
Tentative Grading Criteria for Proposal Part 3
Graded by: Doshna, Total of 16 points
- 6 points: Do you actually KNOW what you say you know?
- 6 points: At least 6 examples not knowing useful UI/user related issues.
- 4 points: Completeness, did you miss any critical issues? Did you think about the users and the UI?
- -1 For generic statements like "increasing usability makes UIs easier to use" will lose points.
Project Part 4: Your users (PRICPE)
Due date: See class schedule. For the electronic part, use TEACH
This is the "R", mainly from your formative empirical work, stemming from the "P" part of your proposal above. Note that since the process is iterative, it is indeed allowable to expand the "P" (research questions) as you progress with finding out more about your users.
The "R" will consist of in-the-field observations or in-the-field interviews of people who could be your users doing the task you are trying to support. (For example, if your users are supposed to be students planning their classes, then you should observe or field-interview students planning their classes. If your users are supposed to be grocery shoppers shopping, then you should observe or field-interview grocery shoppers shopping. Etc.) In addition, you may want to include other forms of research such as web research for additional concept ideas and so on, but we will not grade these other forms of research.
NOTE: Please avoid CS students as the users you observe/interview. You need experience learning to understand users not so much like you.
What to turn in:
- What are the research questions/goals (initially derived from the "P" in your proposal).
- Process: How did you proceed to answer these questions? Be detailed. For example, if you conducted an interview, list all the questions. Where and when did you collect the data? How did you set up to allow triangulation, etc.
- An exhaustive inventory of the elements we discussed in class:
- The people in the space
- 1. Who are they, what are they like?
- 2. What are they doing?
- 3. How are they doing it?
- 4. What do their emotions, purposes, reactions seem to be?
- 5. What problems do they encounter with their activities?
- The objects (technological and otherwise) in the space and with the people
- 1. What are the functional elements of the objects?
- 2. What are the decorational elements?
- 3. Which objects do people look for (perhaps to somehow interact with)?
- 4. Which objects do people bring with them that matter to the activities they are trying to do?
- The environment: spaces, architecture, lighting etc
- 1. What is the layout?
- 2. What is the environment like?
- 3. How does it influence the activities people engage in?
- 4. How does the environment support the objects above?
- Attach your raw data: detailed observations or interview responses (verbal and non-verbal)
- With the detailed observations/responses, point out the places that provide Results/Insights and say what they are ("I"):
- What are the answers to your research questions?
- What other insights did you get from this that are relevant to your design?
What am I looking for:
- A) That you included all the elements I asked you to.
- B) That your observations/interview data contains sufficient detail to notice something interesting, or which the average visitor would not normally see.
- C) That you derive some interesting design Insights from the data. Should be related to some real user need or activity that deserves support/attention.
- D) That you describe the experience factually, clearly, and in enough detail so that someone who has never been in the setting could clearly appreciate and understand what you saw.
Expected length: 6-10 pages, including words and sketches. For sketches, feel free to scan them in instead of drawing them on the computer.
Note: You can combine things into one file, or turn in separate files if it's easier (eg, a separate one for sketches?). The TEACH site will support several separate files.
Sample from a prior year (similar but not exactly the same assignment criteria).
Tentative Grading Criteria for User Data Assignment
Graded by: Doshna, Total of 30 points
- 5 points: Suitable RQs/Goals
- -2.5 for no specific research questions/goals
- -2.5 if research questions weren't fairly thoroughly answered by the insights.
- 5 points: Process:
- -1 to -3 points for not enough detail
- -1 point for not listing questions on an interview
- -1 point for not mentioning triangulation
- -2 for not using a representative user
- If observation: 15 points: Observation that covers people, objects and environments as listed in assignment description. The write-up should have a good amount of detail (so I felt like I could picture the observation/interview).
- -1 Environment not suitable for observation/field study
- -1 user not performing task in ways they would normally
- -1 to -5 observation/interview not detailed enough to picture interview/observation
- If field interview: 5+5+5 points: pertinent questions, good field interview notes, good information about context & artifacts. Overall, the interview should involve the subject performing (or reconstructing) some aspect of your project in the field, with as much data as possible about the materials they used and the context they were in.
- -2 for using mainly present tense
- -1 for not reporting non-verbal details
- -2 for quality of questions (accepting a yes/no answers, no proper phrasing)
- -1 for interview not in the format mentioned in your report. (Ex: If you say you conducted semi-structured interview it has to be semi-structured interview)
- -5 for not doing a field interview
5 points: Good insights/understanding derived from your observation/interview. One point for each non-obvious insight/understanding.
- 5 bonus points: Doing both field-interview and observation(Points will be reduced, based on the content, if additional interview/observation doesn't seem complete)
Additional notes:
- As explained in class, we recommend 1 or 2 users, not large numbers of them; the term is not long enough for large numbers.
- If interview: Must be field interview, so your interview should involve the subjects recounting their previous experience or perform some aspect of the project. At least 2 points reduced if this requirement not met.
Project Part 5: Concepts and early Prototype #1 (PRICPE):
Your team will prepare a poster with the information below, to be presented at Design Studio 1, for brainstorming and feedback. See main class web page for the date.
Also turn in the pages from your poster electronically by 11:59 pm. Make sure everything turned in is readable. Use the TEACH hand-in page.
Present the following:
- A very brief summary of the problem your system is addressing and its users.
- At least 3 of your concepts (sketches, labeled with what Aspect each concept is exploring, as in lecture slides)
- What a concept LOOKS like: see slides 4-9 of the Concepts/Prototypes lecture
- A sketched screen transition diagram of the prototype
(see lecture slides for examples). Must have as at least part of it a (probably updated) concept you decided to pursue that you turned in as a concept sketch.
Must include at least half of your interface, so at least 6 screens that connect with transition arrows.
However, this is lo-fi, so each individual screen does not need to be complete. Diagram must have a user task flow.
- What a lo-fi prototype LOOKS like: see slides 17-18 of Concepts/Prototypes lecture.
- For each design choice in your prototype, justify it based on (1) your user Research (Project Part 4) or (2) a particular principle/technique from the class or book (eg, a Gestalt principle, a design principle from HW1, one of Norman's gulfs, eye physiology, ...). Be specific.
Notes on constraints on your concepts/prototypes:
- Do not artificially constrain your interface ideas to the way a previous system worked or things that you know how to implement easily.
Instead, your design choices should have reasons. Should it be desktop/WIMP? Should it be Wearable? Should it be Tangible? Should it be in a Robot? Your concern is your users, not your developers! Remember to defend your decisions with Usability reasons. And remember to think about trade-offs: eg, the cost of a robot might not be feasible for on-line grocery shopping, so in this example you might eliminate a robot-based interface for reasons of the user's system purchase cost.
- Your UI does not have to be implemented in some other world/project/class. For example, if you're doing this on a senior design project, realize that this UI might be totally usable in that project, totally unviable for that project, or have influences on that project but not transfer directly. Do not constrain your UI to be a "deliverable" that another project could drop into place and use.
How you will be graded: This prototype will be graded based on how well you addressed the details I asked you for (see list in this subsection). This score will contribute a portion of your eventual grade for the prototype portion of the project.
Expected length: enough to fill up a poster, which will probably be 6-8 pieces of paper containing sketches, justifications, explanations, background. As usual, feel free to scan in sketches instead of drawing them on a computer.
Sample from a prior year (not exactly the same as this year's specs, but gives some reasonable ideas).
Give feedback to other teams here from Design Gallery #1
Tentative Grading Criteria for Design Gallery #1
Graded by: Doshna, Total of 100 points
- 5 points: Description of problem.
- 10 points: At least 3 significantly different concept ideas. (-5 for similar designs, -5 for each missing design)
- 10 points: Labels of what aspects each concept is exploring.
- 10 points: Included a transition diagram (-5 for arrows not coming from hotspots in the transition diagram)
- 24 points: Sketches include the majority of the interface. Depending on how much of the interface is covered, partial points are given.
- 24 points: Justifications using user research data and design principles, for major design decisions included. (points deducted if you only used general design principles to justify decisions. You need to justify
decisions based on YOUR users.) (-10 for not showing user research in atleast 2 justifications)(-10 for not justifying more than half of the screens)
- 10 points: Justification for why you chose to prototype that specific concept and not the other concepts.
- 7 points: No use of 'I' / 'We'-centered language (-1 for each occurrence)
Project Part 6: Empirical Evaluation Preparation (PRICPE):
Due date: See main class web page for the date. Use TEACH hand-in page
Plan (1) your Heuristic Evaluation and (2) your usability study, as follows:
- Heuristic Evaluation: you can use either the GenderMag Heuristics or the Nielson Heuristics. Your plan needs to say which you'll use.
- Usability Study: will use the think-aloud technique, in which one or two users perform one or more tasks you give them on a portion of your Mockups prototype. As in other usability studies, your goal is to find out what usability problems this part of your prototype has. The user's task(s) must involve a total of at least 6 user actions.
Heads-up: By the time of your actual evaluation (next assignment), your prototype will need to be in Mockups (even if it is just a scan in of sketches with widgets/transitions added).
The updated prototype will need to support some user in your target population doing some task that's fairly central for your prototype. For example, if you are working on EmpCenter, the task might be "enter my hours for this week". Or if you're working on a Bookstore Map system, the task might be "Find one of the Jim Chee Mysteries by Tony Hillerman".
What to turn in:
- Which Heuristic evaluation you decided to use (GenderMag HE or Nielsen's HE)
- Which two team members will do the Heuristic evaluation
- Usability study's user(s):
- Turn in a brief description of the user(s) you're going to watch performing the task, and how they fit your target population. (e.g., "A male college student (non-CS major) who likes to go to physical bookstores.")
- Usability study Materials/tasks/procedures:
- Turn in the think-aloud practice exercise you'll have the user do.
- Turn in the task instructions you'll give the user (e.g., "Find one of the Jim Chee mysteries by Tony Hillerman").
- If you intend to ask follow-up questions, use additional materials, etc., list them.
Samples: Sample #1. Sample #2. Note: the requirements for this assignment were different for these samples, but they will still give you some ideas.
Tentative Grading Criteria (these need to be updated to include the HE part, but this gives some idea):
Graded by: doshna, Total of 40 points
- 10 points: For Heuristic Evaluation
- 5 points: the exact task(s) you intend to evaluate, and why this is a useful task for your evaluation goals; Include task instructions you'll give the user.
- 5 points: the particular user you are evaluating in the empirical evaluation (but no name please), and why they is/are suitable user(s) within your target population. (-2.5 if no particular user specified but there is a mention of why the listed users are suitable users)
- 5 points: the data you will be collecting, and how that data relate to what you are trying to find out
- 3 point: talked about data collection method (audio/video recording/notes).
- 5 points: all the materials (training, follow-up questions, etc.) to be used.
- 6 points: Involved at least 6 user actions.
- 1 point: Included a think a loud practice exercise.
- -2 points: if the data you are collecting is vague or too generic. You should know exactly what data you are collecting - eg - voice/video recordings, notes about the interview/observations etc
- -1 point: if you plan to interview at the end of your evaluation and you do not have interview questions
Evaluation (PRICPE):
Due date: See main class web page for the date.
The evaluation will be evaluated on your adherence to the evaluation
plan (or explanations of why you deviated) and the richness and completeness of the analysis performed.
Turn in:
- A repeat of the list you first wrote for your evaluation plan, but updated to reflect the way they really happened in your evaluation.
(See evaluation plan: includes what you were trying to find out, what task you actually evaluated, etc.)
If any of these items changed after you turned in your evaluation plan, say that you changed
and how/why.
-
Your Heursitic Evaluation should be on at least 6 screens.
- Your prototype (so that we can make sense of your study and its results):
- Turn in your Mockups prototype.
- Also turn in drawings or a list of 6+ actions you were expecting/hoping the user to perform the task. (The users don't have to perform those exact same actions, but at least we can tell what you had in mind as a reasonable way to do the task(s).) You can print the screen sequence and annotate it with your pencil, or just write up a list of the actions like this: (1) Enter email address, (2) Click "ok" to give permission to share location data (3) Select "by author" on the next screen, (4) Type "Tony Hillerman", (5) Click "More criteria" on the next screen, (6) Click "By main character" on the next screen, (7) Enter "Jim Chee" on the next screen, (8) Follow the map to the first highlighted location it shows, (9) Click the location to show that they've arrived there, (10) Look at location on the shelf it highlights, (11) Grab the Tony Hillerman book and start browsing it.
- How you actually performed the study (eg, "We sat on the couch at the front of the bookstore to do the think-aloud exercise, and then we explained the task and handed our user a tablet computer with the prototype in the right state to begin the task. We all got up, and one of us walked with the participant around the bookstore as he performed the task using the prototype. That team member tried to take notes on paper at the same time, but that was hard since we were walking around. Since following the participant around made it hard to take notes, a second team member followed behind to take additional notes. This note taking was really difficult in these circumstances (walking around a bookstore, watching the user and the prototype, writing, and trying not to run into things), so we had rehearsed it first to work out who was responsible for gathering what kind of data."
- Data and results:
- A list of the problems you found in the usability study and the Heuristic Evaluation. Be specific about what feature(s)/screen(s) were involved, and at what portion of the user's task.
- Insights from this HE and study on fixes you need to make to your prototype.
- All the raw data you collected, and all the HE forms you filled out, for at least 6 of your screens: GenderMag HE forms or Nielsen HE forms.
Samples: Sample #1. Sample #2. Note: the requirements for this assignment were somewhat different in prior terms, but these will still give you good ideas.
Tentative Grading Criteria for Evaluation (100 pts possible)
Graded by: Doshna
- Heuristic Evaluation (50 Points)
- (15 points) Answered all Heuristics in every form
- (15 points) Done for all screens
- (5 points) Done by two people
- (5 points) No users
- (10 points) Insights
- User Evaluation (50 Points)
- (25 points) Process (Must have a think aloud practice -5)(For not providing Detailed Process -5)
- (10 points) Raw Data
- (5 points) Adherence to original plan or justification for changes made
- (10 points) Insights
- (-5 points)For not clearly distinguishing between Heuristic Evaluation and User Evaluation
Prototype #2 (Mockups) (PRICPE):
For brainstorming, feedback at our Design Gallery. See main class web page for the date.
Also turn in the pages from your poster electronically the same day.
Use the TEACH hand-in page.
(Although the runnable Mockups prototype needs to exist for your poster, you do not need to turn in the actual prototype.)
Present the following. All materials must be READABLE by human eyeballs. :-)
- A very brief summary of the problem your system is addressing and its users.
- A Mockups screen transition diagram of the complete prototype (printed out). (How to do this: "export to pdf" to get all your screens, draw arrows with your pen.)
Your users' "main" tasks should all be part of the prototype.
At least 1/3 of it should now be high-fidelity.
(Your low fidelity parts should also be in Mockups: to do them, just scan them in from your drawings, then add "shape" controls on top of the buttons, links, etc. that you have drawn on there, make them 25% opacity so that you can see the drawing beneath them, and make them "live" by linking the covering shapes to the right next screen.)
- A laptop with your Mockups prototype live.
- For the design decisions you have made since the last Design Gallery, include a justification based on your users, research and evaluation activities, and/or principles you've learned in class of each of your design decisions in the prototype.
Sample. (Somewhat different specs that year and different prototyping tool, but still gives an idea.)
Give Design Gallery #2 feedback to other teams here
Note: This hand-in will not be graded alone, but will be taken account in the final version's grade. This hand-in was to document the rate of progress from the first version to this version to the final version.
Final Prototype (Mockups) and Team Presentation:
Due date is on the main class web page. Turn in:
- Your "runnable" Mockups prototype
- An updated list of the prototype (brief summary, Mockups screen transition diagram, justifications of decisions -- but now the justifications combine your (updated) justifications from your early prototype (Design Gallery #1) with your (now-updated) justifications from your later prototype (Design Gallery #2), user studies, any other kind of feedback/material from class
- Your powerpoint file from your 10-minute team presentation (you can update it to improve it before you finally hand it in), which emphasizes the above items. Namely:
- (Brief, eg 1-2 minutes): Start with the brief summary of the problem you are trying to solve and generally what kind of users you're targeting.
- (Brief, eg 2-5 minutes): Familiarize us with your Mockups prototype (can be a live demo, or can be static pictures in your powerpoints).
- (Most of your presentation): Spend most of your time telling us about the design decisions you made, explaining why you made each one.
Tie the decisions to the various Research/Feedback/Evaluations you have done and to CS 352 principles. Be specific about things you saw (eg, in a user observation, in the Inclusive Design workshop, in a Design Gallery, in HCI principles taught in class, in Heuristics you've learned, etc.) that led to your decisions, and specific about which principles and feedback led to what decisions.
- At the end of the 10 minutes, there will be a couple of minutes for the rest of the class to get their questions/notes together to give you feedback, while we transition to the next team.
You can give your feedback on teams from their final presentations here
Your final prototype (by now all High Fidelity (i.e., completely done in Mocksups, no more scanned-in pencil drawings) will be graded on the strength of connection between usability design principles and your users with the decisions you made.
See the Presentation guidelines above for various acceptable things to use as justifications.
In summary, the more justification for each design decision based on HCI principles and the "Research" and "Evaluation" aspects of PRICPE you have experienced, the better your grade.
Tentative Grading Criteria for Final Prototype and Team Presentation
Graded by: Doshna, Total of 60 points
- 20 points: Updated List of Elements from Prototype #2 (includes brief summary, cogtool prototype and justifications for design decisions
- 20 points: Powerpoint file (includes brief summary of problem / users, familiarizes us with the prototype, justifications for design decisions)
- 20 points: Strength of connection between usability design principles and your users (-5 points: wrong justifications based on design principles, -5 points: wrong justifications based on analytical work, -10 points: wrong/missing justifications based on empirical work)(-3 points: missing justifications based on design principles)
- -5 points for not turning in screen transition diagram
Date of last update: Mar. 15, 2019