1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Here's our now familiar observed data:
| Observed Data | Symbol | Formula | Value |
|---|---|---|---|
| Completion Rate | p | x/n | .90 |
The way we compute this first adjustment depends on both the expected and the actual distribution of successes across the task. If you have no idea how participants will perform on the task, the Success Probability will be calculated using the LaPlace method. However, if you expect the majority of participants to pass the task then deciding which method to use depends on the value of that observed Completion Rate. Here's the chart:
| p <= .5 | Wilson | This task performed lower than expected, so the effect of this adjustment is to improve the observed Completion Rate by bringing the Success Probability closer to .5. |
| .5 > p >.9 | Maximum Likelihood Estimate (MLE) | This task met expecations, so no adjustments are made. |
| .9>= p >1.0 | LaPlace or Jeffreys | This task performed better than expected, so the effect of this adjustment is to diminish the observed Completion Rate by bringing the Success Probability closer to .5. |
| p=1.0 | LaPlace | This task performed better than expected, so the effect of this adjustment is to diminish the observed Completion Rate by bringing the Success Probability closer to .5. |
In our example, most of the participants passed the scenario, as expected, so in order to decide which method to use we need to look at the value of the observed Completion Rate. The chart above indicates that for an observed Completion Rate of .90 we should use either LaPlace or Jeffreys method. We're going to use LaPlace.
Here are the formulas for each of the methods:
| Calculated Data | Formula | Value (given x=9; n=10) |
|---|---|---|
| Jeffrey's Method | (x+.5)/(n+1) | 9.5/11 =.86 |
| LaPlace Method | (x+1)/(n+2) | 10/12 =.83 |
| Maximum Likelihood Estimate (MLE) | x/n | 9/10 = .90 |
| Wilson Method | (x+2)/(n+4) | 11/14 =.79 |
You can see how each adjustment method gives you a very different Success Probability! In our example we need to use LaPlace. So instead of our observed Completion Rate of .90 (or 90%), we have a Success Probability of .83 (or 83%).
Note our progress:
To be complete, statistical conclusions should always include: