1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
This should be good news for anyone concerned about the Return On Investment (ROI) of usability. Ultimately what it all boils down to is that (1) in most cases streamlined usability should suffice and (2) statistical usability is more for the benefit of the practitioner than the client.
Sauro's Confidence Interval Calculator makes it cheap and easy to do the analysis we've worked through in this tutorial. Doing so, however, is unlikely to make the conclusion any more concrete and quoting such wide Confidence Intervals to a client may, in fact, do more harm than good from a change-management perspective. Obviously if the client wants them, provide them. But be prepared for a lengthy explanation. Here we find ourselves at the genesis of the old refrain, 'usability is not quantitative, it's qualitative'.
What it does mean from an ROI perspective is that you need not strive for scientific precision beyond the capacity of the statistics. Money and effort that might go into making a small-sample usability test more "scientific" could probably be better spent in additional rounds of testing. And clients who insist on statistical significance or a high degree of certitude need to be prepared to pay for large-scale studies.