Schedule Time & Gather Materials
- Preparation: 1 Day (depending on your organization)
- Testing Time: 1-3 Weeks
- Analysis: As quick as looking at a graph or table
- An editable version of your live site (this will depend on the platform on which your site is built and what testing tool you use)
- A/B testing software subscription (see below for links)
Carry Out This Method
- Determine your success metric. This may be time spent on your site, sales, conversions, e-mail signups, or number of support tickets submitted.
- Determine the page element that needs to be altered and tested. This may be headline wording, call-to-action visibility, images, or any page element that has been ambiguous to test users.
- Formulate a hypothesis regarding how to fix your problematic page element. Apply that fix to a new version (“version B”) of your site, leaving the original version (“version A”) unchanged.
- Use an A/B testing tool to serve version B to a randomly chosen subset of users.
- Run the test for approximately two weeks in order to reach a statistically significant number of users.
Try These Tips
- Serve the original version of the page to Google’s bots to avoid being penalized in search results.
- Avoid the temptation to stop the test as soon as you see a statistically significant difference between the two pages. Continue the test until its scheduled end. This is especially important if you plan to publish results or your research is funded with grant money.
- Serve both versions to users concurrently in order to account for seasonal variables.
Explore More Resources
- Wikipedia: A/B testing
- Visual Website Optimizer: The Complete Guide to A/B Testing
- Smashing Magazine: The Ultimate Guide To A/B Testing
- Oracle: A/B Testing Increases Conversion Rates
- Google Webmaster Central Blog: Website testing & Google search
- MeasuringU: 10 Things To Know About A/B Testing
- MeasuringU: A/B Test Calculator
- Mashable: 10 A/B testing tools to help improve conversions
- Optimizely: Experiment to uncover customer insights and create high-performing experiences
- Crazy Egg: How to Choose the Right Testing Software For Your Business
- Conversion Rate Experts: Split Testing Software
- Google: Google Analytics Solutions: Optimize