Usability Myths Need Reality Checks

Two papers, published by Robert Virzi in 1992 and Jakob Nielsen and Thomas Landauer in 1993, made a pretty good case that five users would uncover 70% of major usability problems and the next three would get most of the rest - for certain types of software. The usability community took the idea and ran with it.

Arguments advanced in the articles justified five to eight users as a significant number. It was also an affordable number. All sides were satisfied. Let the testing begin!

The two papers gave five to eight users a scientific basis for the testing of small software applications. However, extending the "five user theory" to larger software applications and the world-wide web turned a useful rule into a myth with no scientific basis.

Good myths are plausible explanations that serve a purpose. Feel free to chuckle, but at one time more people believed that a big guy in the sky with a hammer caused thunder and lightning than ever believed five users was enough to test anything, even though praying to Thor never did stop the rain. The myth served its purpose, which was not to control the weather, but to keep the Nordic priesthood in power.