Two pizzas

Multivariate Testing

Multivariate Testing is a software-aided analytical test, similar to A/B Testing, wherein multiple changes to a site's design are tested in several combinations. Rather than two versions of your page being served to users, multiple versions of the page are served, each with its own combination of edited elements. This allows you to test several treatments in context with one another, since a combination of edits together may be more or less effective than the apparent sum of their parts.

For information on testing only one change, see A/B Testing.

Description

Schedule Time & Gather Materials

Clock Icon

Schedule Time:

  • Time for Preparation: 2-3 Days (depending on your organization & testing needs)
  • Time for Testing: 2-3 Weeks
  • Time for Analysis: 1 Day
Gather Materials

Gather Materials:

  • An editable version of your live site (will vary based on testing tool used)
  • Multivariate testing tool (see “More” for links)

Carry Out This Method

Team in a conference room is involved in discussion

Step One

Determine your success metric.

This may be time spent on your site, sales, conversions, e-mail signups, or number of support tickets submitted.

Step Two

Determine the page elements that need to be tested.

This may include colors, link placement, wording, or any other design choices that influence user behavior.

Step Three

Determine the treatments to be done.

This will depend on the nature of your test and the context of the page on which the elements to be treated are found.

Step Four

Create version B of each page element.

Depending on how your multivariate tool works, you may need to create combinations of edits manually, or it may randomize the combination of them.
For example, if you have A and B versions of your headline, CTA color, and hero image, then you will be testing the following combinations:

  • Headline A, CTA color A, Image A (Control)
  • Headline A, CTA color B, Image A
  • Headline A, CTA color B, Image B
  • Headline A, CTA color A, Image B
  • Headline B, CTA color A, Image A
  • Headline B, CTA color B, Image A
  • Headline B, CTA color B, Image B

 

 

Try These Tips

Run the test for as long as you need to get a statistically significant amount of data. This will vary, but a test time of approximately three weeks will suit most organizations. Compared to a traditional A/B test, this test will require more users for statistically significant results.

Explore More Resources

Shipping & Delivery

Vestibulum curae torquent diam diam commodo parturient penatibus nunc dui adipiscing convallis bulum parturient suspendisse parturient a.Parturient in parturient scelerisque nibh lectus quam a natoque adipiscing a vestibulum hendrerit et pharetra fames.Consequat net

Vestibulum parturient suspendisse parturient a.Parturient in parturient scelerisque nibh lectus quam a natoque adipiscing a vestibulum hendrerit et pharetra fames.Consequat netus.

Scelerisque adipiscing bibendum sem vestibulum et in a a a purus lectus faucibus lobortis tincidunt purus lectus nisl class eros.Condimentum a et ullamcorper dictumst mus et tristique elementum nam inceptos hac vestibulum amet elit