Page tree
Skip to end of metadata
Go to start of metadata

Your Rating: Results: 1 Star2 Star3 Star4 Star5 Star 116 rates

Draft for 5.1

Collection of thoughts about testing content variations

A/B & Multi-Variate Testing are two methods to test variations of content to determine which works best.

  • A/B will just test variations of one item, e.g. variations of a header (A/B does not mean it can only have two variations)
  • Multi-variate testing (MVT) means you test several things at once. In multi-variate, you define a number of options for a number of elements, and then they are automatically iterated through. Example: imagine the following paragraphs: a header (3 headlines), teaser (2 variants), an image (5 variants) and a call to action (3 variants) - there are 75 combinations of these!

Day CQ has a video showing A/B (they call it multi-variate but it is not) , the example starts at 1 min 25 s.

Their multivariate A/B paragraph has a statistics view in the paragraph itself. This is great for A/B but won't work for MVT, as this would be done across several paragraphs.

Testing in general could be on various levels, e.g. we could have a template ("test-folder") which allows to add variations of complete pages below, or we could test complete sites (fantastic to test alternative site structures!). It would also be interesting to test different "paths" - e.g. signup-funnels on a website. The most common form I believe is to test content on a single page, where one can define variants of content elements.

The task will be to find an intuitive way to manage these and to see the results. It could also be interesting to automate promoting the best variant, e.g. the testing starts out with all variants being equal, but once a clear pattern emerges, the best candidates are shown much more often to gain maximum benefit (until testing it finished).


  1. Why Split-Testing is Like Sex in High School. Everybody says they're doing it. Great article, especially the reasoning about statistical significance.

  2. To run A/B testing you need statistics data.

    If you would have In Page Statistics, Empower the Authors then you already are close to the statistics data of a page 'couple' to compare them.

    The only thing what is then missing is the random display of two or more pages sharing the same public URL. And maybe a side-by-side presentation of the statistics.