|
|
|
|
|
|
Intro/Overview |
|
Usability Testing |
|
Define |
|
Overview of steps |
|
Demo of a test |
|
Lessons |
|
Questions |
|
|
|
|
Analysis of web transaction logs |
|
Card sorting |
|
Focus Groups |
|
Structured Evaluation |
|
Survey/Questionnaire |
|
Usability Testing |
|
|
|
http://www.cs.umd.edu/~zzj/UsabilityHome.html
includes brief descriptions of methods |
|
|
|
|
One-on-one sessions (45 mins-1 hr. each) |
|
Give participants a set of tasks |
|
Observe (minimal intervention) as they complete
tasks |
|
Participants “think aloud” |
|
(1) Experimenter interfaces with the
participant; (1-2) Recorder writes |
|
4-6 participants |
|
|
|
|
|
|
|
What you can learn |
|
Detailed information about interface and
functionality problems |
|
Understand user’s mental model |
|
|
|
When to conduct |
|
Early on and throughout the iterative design
process |
|
|
|
|
|
|
Define your objectives |
|
“Walk-through” the site |
|
Create tasks |
|
Pre-test the tasks |
|
Conduct the test |
|
Debrief with participant |
|
Analyze results and make recommendations |
|
Test the changes |
|
|
|
|
|
|
|
List what you want to learn and prioritize |
|
What features you want users to use |
|
Information you want users to find |
|
|
|
Keep in mind the primary purpose and audience of
your site |
|
|
|
|
|
|
|
|
|
|
|
|
Objective: Can users find and use the search
feature |
|
Task: Find a book by Ernest Hemingway |
|
|
|
Objective: Can users find the library hours |
|
Task: You are coming to the library on Sunday
and want to know if it will be open |
|
|
|
|
|
|
Do’s |
|
Make first task easy |
|
Test realistic tasks |
|
User-centered tasks |
|
Simple tasks |
|
Use scenarios |
|
Pre-test tasks! |
|
|
|
Don’ts |
|
Don’t use words from the site |
|
Don’t include funny tasks |
|
Don’t include dependent tasks |
|
Don’t create too many tasks (8-10 is enough) |
|
|
|
|
|
|
|
|
|
“Walk-through” the site to create tasks |
|
Create logging
sheets (include tasks and
ideal path) |
|
|
|
Recorder’s Role - Write it ALL down |
|
Click path |
|
Participant’s impressions |
|
Time it took to complete each task |
|
Whether they completed the task or not (note
whether the participant thinks they completed the task) |
|
|
|
|
Make them feel welcome |
|
Introduce all the people in the room |
|
Read intro script |
|
Have them read the consent form (human subjects approval); Committee
for Protection of Human Subjects (http://cphs.berkeley.edu:7006/) |
|
Fill out the pre-test questionnaire |
|
Allow them to look around the site 1-2 mins. |
|
Give tasks (one-by-one, verbally or written) |
|
|
|
|
|
|
|
|
|
|
|
|
|
Do’s |
|
DO listen; be patient |
|
Prompt them to think aloud: |
|
What are you thinking now; What did you expect
to find; What label/link were you expecting to see |
|
If you give feedback, say “good” or “ok” for
everything, not just when they do what you want |
|
|
|
|
|
|
|
|
Don’ts |
|
Don’t lead the user (“That was difficult, wasn’t
it”) |
|
Don’t help or correct them (e.g., if a
participant asks “should I click here?” turn it around say “what do you
think you’ll get if you click there?” or “go ahead click and see what
happens”) |
|
Don’t direct questions at them (“Does this
button confuse you?”) |
|
Don’t defend the site; remain objective |
|
No smiles/nods between Recorder and Experimenter |
|
|
|
|
If several participants have already identified
a problem, no need to let them flounder |
|
You feel you have learned enough |
|
The participant is visibly frustrated |
|
|
|
White lies – “Others have had this problem,
let’s stop” |
|
|
|
|
|
|
|
|
Ask them what they thought; let them talk |
|
Go back and replay things that you have
questions about |
|
Have some debrief questions |
|
|
|
Give them their incentives (e.g., copy cards,
jamba juice gift certificates) |
|
|
|
|
|
Look for trends |
|
Rate problems |
|
Catastrophic - unable to complete the task or
worse, completed it incorrectly and didn’t know it |
|
Serious- user was significantly delayed but did
complete |
|
Minor - briefly delayed |
|
Make design recommendations |
|
Test it again! |
|
|
|
|
|
|
|
|
|
|
Users are not designers |
|
“I found it but others won’t”; “I can’t find it,
but others will” |
|
Watch what users DO, not what they SAY |
|
Take the time to create good/realistic tasks |
|
Debrief is a good time to learn a lot |
|
Document – make screen shots |
|
Make it a pleasant experience for the
participants |
|
|
|
|
|
|
|
|
|
|
Rubin, Jeffrey. Handbook of Usability Testing:
How to Plan, Design, and Conduct Effective Tests. New York: Wiley, 1994. |
|
Krug, Steve. Don’t Make Me Think! A Common Sense
Approach to Web Usability. Indianapolis: New Riders Publishing 2000. |
|
Instone, Keith.
Usableweb.com |
|
Nielsen, Jakob. Useit.com |
|
|
|
My PPT and Handouts -- Usability Testing Tool
Kit: http://www.cdlib.org/libstaff/education/evaluation/ |
|
Email: Rosalie.Lack@ucop.edu |
|
|
|
Notes
