andrea l spray

Special Note

Remember that we're working on a task completion example (a.k.a. a Bernoulli distribution). If you have time-on-task or satisfaction ratings you would do an entirely different type of analysis.

Clarifying Statistical Usability

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Next >

Bring On the Statistics!


Remember our scenario:

"Using Travelocity.com, find the cheapest flight from St Louis to Chicago."

And our goal:

To be complete, statistical conclusions should always include:

  1. Success Probability (a.k.a. the % of users expected to pass the scenario)
  2. Confidence Level (a.k.a. how confident your are in your conclusion)
  3. Confidence Interval (a.k.a. worst-case to best-case)

Below are the nine steps to accomplishing this goal.

  1. Success Probability
  2. Confidence Level
  3. Standard Deviation
  4. Standard Error
  5. Degrees of Freedom
  6. t-Statistic
  7. Adjusted Success Probability
  8. Margin of Error
  9. Confidence Interval

Along the way we're going to talk about observed data (what we observed in the lab) and calculated data (what we've finagled with). Here's the observed data we've collected so far. Make sure that this makes sense to you because its the foundation for the rest of our analysis!

Observed Data Symbol Formula Value
Sample Size
Number of participants who attempted this task.
n   10
Task Successes
Number of participants who successfully completed the task.
x   9
Task Failures
Number of participants who failed to complete the task.
  n-x 1
Completion Rate p x/n .90
Failure Rate q (n-x)/n .10

Next >

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18