Woman stares at computer screen, frustrated

Error Analysis

Error Analysis is a lab testing method that can be used alongside, or simultaneous with, other lab tests. No matter how well we cover our bases when designing an app, users will encounter errors. A thorough Error Analysis allows you to identify the frequency and type of errors that occur so you can address their causes and prevent users from becoming frustrated with your product.

Description

Schedule Time & Gather Materials

Clock Icon

Schedule Time:

  • Time Per Session: ~ 1 Hour
  • Number of Sessions: 10 – 20
Gather Materials

Gather Materials:

  • Screen recording software that records click attempts
  • Video of a user test where the screen was recorded with such software
  • Notebook
  • Timing device (i.e. stopwatch)

Carry Out This Method

  1. Conduct a Think-Aloud Test while recording video of the user and their screen. Your screen capture software needs to record user clicks.
  2. After the standard test has concluded, watch the video and record errors. You can record errors during the test, but using video is easier and more accurate.
  3. Begin your stopwatch at the beginning of each task you record to calculate the length of time tasks take as it relates to error rates.
  4. Every time the user encounters an error, record it, even if the user is making the same error multiple times when trying to complete the same task.
  5. Note what sort of error it is. According to Jeff Sauro, you can classify errors into four basic groups:
    1. Slips: The user attempts the correct action but botches the execution (e.g. typos, accidental clicks slightly to the side of the correct radio button).
    2. Mistakes: The user attempts or executes an incorrect action because they are unaware of what they are supposed to do (e.g. leaving a required checkbox unchecked).
    3. Interface Problems: These are a subset of the “Mistakes” category caused by a lack of clarity in the interface design or copy.
    4. Scenario Errors: These errors are unlikely during everyday use, but arise during usability tests. Placeholder data confuses the user; they are unsure if an action was carried out correctly or not.
  6. Stop your stopwatch at the end of each task and record how long the user took to complete it.
  7. Compare error rates and task completion times between test participants. Bring the data to your product team, suggesting changes to the interface based on the most commonly encountered mistakes.

Try These Tips

For some errors, the root cause and the proximate cause may not be the same thing. In his book The Design of Everyday Things, Don Norman cites Toyota’s “Five Whys” method for finding the root cause of an error. You may not ask “Why” five times exactly, but you may need to ask “Why” multiple times. For example:

  1. “Why did the user not check the Terms & Conditions checkbox?” Because they ignored it.
  2. “Why did the user ignore the checkbox?” Because they could not read the text next to it.
  3. “Why was the user unable to read the text?” Because they can not read text that is too light a shade of gray.
  4. “Why can the user not read text that is too light?” Because they have diminished eyesight or their monitor is not calibrated properly for it.

Explore More Resources

Shipping & Delivery

Vestibulum curae torquent diam diam commodo parturient penatibus nunc dui adipiscing convallis bulum parturient suspendisse parturient a.Parturient in parturient scelerisque nibh lectus quam a natoque adipiscing a vestibulum hendrerit et pharetra fames.Consequat net

Vestibulum parturient suspendisse parturient a.Parturient in parturient scelerisque nibh lectus quam a natoque adipiscing a vestibulum hendrerit et pharetra fames.Consequat netus.

Scelerisque adipiscing bibendum sem vestibulum et in a a a purus lectus faucibus lobortis tincidunt purus lectus nisl class eros.Condimentum a et ullamcorper dictumst mus et tristique elementum nam inceptos hac vestibulum amet elit