Schedule Time & Gather Materials
- Time for Preparation: 2-3 Days (depending on your organization & testing needs)
- Time for Testing: 2-3 Weeks
- Time for Analysis: 1 Day
- An editable version of your live site (will vary based on testing tool used)
- Multivariate testing tool (see “More” for links)
Carry Out This Method
Determine your success metric.
This may be time spent on your site, sales, conversions, e-mail signups, or number of support tickets submitted.
Determine the page elements that need to be tested.
This may include colors, link placement, wording, or any other design choices that influence user behavior.
Determine the treatments to be done.
This will depend on the nature of your test and the context of the page on which the elements to be treated are found.
Create version B of each page element.
Depending on how your multivariate tool works, you may need to create combinations of edits manually, or it may randomize the combination of them.
For example, if you have A and B versions of your headline, CTA color, and hero image, then you will be testing the following combinations:
- Headline A, CTA color A, Image A (Control)
- Headline A, CTA color B, Image A
- Headline A, CTA color B, Image B
- Headline A, CTA color A, Image B
- Headline B, CTA color A, Image A
- Headline B, CTA color B, Image A
- Headline B, CTA color B, Image B
Try These Tips
Run the test for as long as you need to get a statistically significant amount of data. This will vary, but a test time of approximately three weeks will suit most organizations. Compared to a traditional A/B test, this test will require more users for statistically significant results.
Explore More Resources
- 18F Guides: Multivariate Testing
- Adobe: Discover the perfect combination. Multivariate tests lead the way.
- Wikipedia: Multivariate testing in marketing
- Smashing Magazine: Multivariate Testing 101: A Scientific Method Of Optimizing Design
- Optimizely: What is Multivariate Testing?
- Google Analytics Solutions: Optimize