Its important that you follow this tutorial in sequence - from start to finish - as each step builds on the work in the step before.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
In order to conduct quantitative analysis you need to have quantitative measures. Jakob Nielsen points out the following metrics that can be gathered from usability testing: learnability, efficiency, memorability and errors. But quantitative doesn't necessarily mean objective, and so Nielsen also includes a fifth: satisfaction.
ISO 9241 (PDF 608KB) "Ergonomics of Human System Interaction" uses a similar list of usability metrics: effectiveness, efficiency and satisfaction where "effectiveness" can be considered a combination of Nielsen's learnability, memorability and errors.
So these are the things that can be measured. How they are measured is by capturing task completion rate, time-on-task and satisfaction ratings (typically on the Likert scale). Regardless of your measure, you can't conduct statistical analysis on anecdote alone, you need to count something.
For the purposes of this tutorial, we'll be looking at the following task completion example:
"Using Travelocity.com, find the cheapest flight from St Louis to Chicago."
Now, imagine that out of 10 participants tested, 9 participants successfully complete the task and 1 participant does not. Is it reasonable to conclude that 90% of all end-users should be able to successfully complete this task? Strangely, no. Keep reading...