How many users do you need for a user test?
Five users per profile is enough to find most usability problems in a qualitative test. To measure or compare, you need a survey and far more answers.
- For a qualitative usability test, 5 users per user profile are usually enough to find most problems.
- If you have several very different profiles, test each one separately.
- Five users tell you what breaks and why. They do not tell you how many of your users are affected.
- To measure, use a survey: around 100 answers give a margin of error of about ±10 points, around 400 about ±5.
Qualitative tests: five users per profile
The well-known rule comes from Jakob Nielsen: testing with five users reveals most usability problems, and each extra user mostly repeats what you have already seen. It is better to run several small rounds than one large one.
In the Betclic Poker study, five regular high-stakes players tested the new desktop app. Five out of five were held back by the speed of play: you do not need fifty people to see a problem that strong.
When five is not enough
Five users work per homogeneous profile. If your product serves very different people, for example managers and employees, or beginners and experts, test each group: their problems are different.
Five is also not enough when you need a number to decide, to compare two designs, or to convince a sceptical stakeholder with a percentage. That is a measurement question, not a discovery question.
Surveys: count answers, not users
To size a problem, you need enough answers for the percentage to be stable. As a rule of thumb, the margin of error at 95% confidence is about ±10 points with 100 answers and about ±5 points with 400.
Split results by segment only when each segment has enough answers on its own. A segment of 20 answers tells you a direction, not a figure.
Combine both
The most useful setup is usually a small qualitative test to find the problems and understand them, then a survey to know how many users they affect. Qualitative tells you why, quantitative tells you how much.
Questions teams ask.
- Is 5 users really enough for a usability test?
- For finding usability problems with one homogeneous profile, yes in most cases. Run several rounds of five rather than one round of twenty, fixing problems in between.
- How many users for an unmoderated test?
- Unmoderated tests lose some depth, so plan a little more: 6 to 10 per profile, and watch the recordings rather than only reading completion rates.
- How many answers do I need for a survey?
- Around 100 answers for a margin of error of about ±10 points, around 400 for about ±5. More if you want to compare segments.
Source: Nielsen Norman Group, “Why You Only Need to Test with 5 Users”