andrea l spray

Which Success Probability to Use?

Note that the Confidence Interval uses the Adjusted Success Probability (calculated in step 7), not the Success Probability (calculated in step 1).

Ideally you could use the Success Probability for both - it would make the calculation much easier and the result statement less awkward. However research conducted by Sauro and Lewis (PDF 98KB) concluded that two different adjustments are required to ensure the greatest degree of accuracy.

To avoid the awkward result statement, drop the Success Probability and simply state your results in terms of the Confidence Interval: "With 95% confidence it can be concluded that users will complete this task between 57.5% and 100% of the time."

Clarifying Statistical Usability

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Next >

Step 9 of 9: Confidence Interval


We finally have all the information we need to achieve our goal. Our Confidence Interval is merely the Adjusted Success Probability we calculated in step 7, plus and minus our Margin of Error.

Calculated Data Symbol Formula Value
Confidence Interval   (pam) to (pa + m) 57.5% to 100.5%

To be complete, statistical conclusions should always include:

  1. Success Probability (a.k.a. the % of users expected to pass the scenario): 83%
  2. Confidence Level (a.k.a. how confident your are in your conclusion): 95%
  3. Confidence Interval (a.k.a. worst-case to best-case): 57.5% to 100%

Congratulations! After all your hard work you can now conclude:

Users will successfully complete this task 83% of the time. With 95% confidence it can be concluded that users will complete this task between 57.5% and 100% of the time.

Here’s all the work it took to get us here: complete calculation table

But you might be saying to yourself “Wow! That’s a wide Confidence Interval…”

Next >

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18