Schedule Time & Gather Materials
- Time per Session: 5-30 Minutes, Seconds for a Guerrilla Test
- Number of Users: At Least 50 Users for Accurate Results Regarding a Given Element
Due to the open-ended nature of the questions and the variety of responses, more users are better.
- A testable version of your site. This could be anything from static page images to a live site – the user does not need to interact with it directly.
- Paper and pen to record results
Carry Out This Method
- Before testing, meet with stakeholders to determine the elements that need to be tested. This could be anything from the name of your product to a specific interactive element.
- Write a series of open-ended questions about these elements. To avoid bias, they must not be “yes” or “no” questions, and must not include leading details that the user could agree or disagree with.
- Recruit as many users as possible from your target user base.
- Present your test user with the testable version of your product, at whatever level of detail is absolutely necessary. Do not give them navigational control.
- If you are testing an interactive element that appears during a typical task flow, walk the user through that flow until they reach the element to be tested. This will provide them with the same context that users will have when they reach it naturally.
- Ask users what they expect from the element you are testing. Remember, these are open-ended questions. Resist any temptation as a moderator to include explanatory details.
- Continue the test as needed. Do not lead test users; at every step of the test, give them only the information they need.
Try These Tips
- Ask customers to explain the reasoning behind their answers. You will need to know if users expect something because it is common practice at similar sites, if it is their pie-in-the-sky dream of how a product should work, or if it mimics something from their daily life.
- If you have only one element that needs to be tested, you can perform this test very quickly as a Guerrilla Usability Test. This can often provide great insight at low cost because you can test many more users.
Explore More Resources
- Nielsen Norman Group: Don’t Prioritize Efficiency Over Expectations
- User Zoom: Are users getting what they expected out of the product?
- Spotless: Meeting user expectations throughout a site normally delivers good usability. Ways to make sure you meet expectations: user research, reviewing competitor sites and following usability guidelines.
- Fitzgerald Steele: Usability Testing and User Expectations