Tutorial 7: Split Tests

Split tests are a method for doing controlled experiments with content, and testing which variant performs the best in relation to a particular conversion goal defined by you.

There are two options for split testing content:

  • A content-based split test pitches two versions of a paragraph against each other
  • A page-based split test tests two versions of a page against each other

In this tutorial you will learn how to create and use content-based split tests – as the process is more or less identical with both types, we don’t need to do them both.

A content-based split test tests two variants of a paragraph against each other. This means that it’s useful for testing the conversion rate of e.g. design elements (like buttons) and texts.

If you’ve followed the T1 content tutorials, you will have both a Home page and a Store page – if not create them now, then create a paragraph on the Home page.

To create a content based split test:

  • Go to Content > Home and select the paragraph you want to split
  • Switch to the Marketing tab and click Setup split test (Figure 2.1)
Figure 2.1 Setup split test

This opens the Split Test wizard – follow it and:

  • Select Content based split test
  • Select the Choose another page as conversion page, then select your Store page – this means that the goal of the split test is to see which variant leads to most people viewing the Store page
  • Name the split test
  • Select register conversion in next step only
  • Select the percentage of visitors to include in the split test – I suggest 100%, which means that 50% will be shown the original and 50% the variant
  • Select Manually as the end split test criteria
  • Save

This creates the split test – now you need to create the content variation.

To create a variant of a paragraph:

  • Select any paragraph
  • Switch to the Marketing tab
  • Click the Create variant button (Figure 3.1)
Figure 3.1 Create variant

The paragraph will be copied in the list.

Open the variant and replace the text with a new text (e.g. a large GO TO STORE link or something).

Select the Marketing tab and choose Variation in the Variation dropdown. 

Figure 3.2 Select "Variation"

Now both the original and the variant will be marked with adornments so you can differentiate them (Figure 3.3).

Click the Preview button to switch between viewing your original and the variant – this will help you get the details right between the two versions.

Figure 3.3 Original and variant page

Once you’re happy with both your paragraphs you are ready to start the test – simply switch to the Marketing tab and click Start in the split test area (Figure 4.1).

Figure 4.1 Start split test

Once a split test has been started, the selected percentage of visitors will be shown either the original or the variant and the statistics collected.

You can pause or stop a split test at any point in time from this tab as well.

Of course, depending on your visitor trends it will take a while for you to get significant results on a split test – and on this tutorial you will get no visitors at all.

A page based split test pitches one page against another page on your solution. This means that you can create e.g. two front page layouts and test which one performs the best.

Creating a page bases split test is almost identical to the process outlined above, but there are a few key differences when working with page based split tests:

  • You create your variant pages before creating the split test, as the first step in the wizard is selecting the variant
  • You have access to slightly different goals, since not all goals make sense on both types of split test

Try it out if you have extra time – if not, then skip it.

Once you believe you have any kind of significant data:

  • Open the page with the split test (it’s marked with an adornment in the content tree)
  • Switch to the marketing tab
  • Click View Report

This opens a split test report (Figure 6.1).

Figure 6.1 Split test report

The split test report can help you determine whether a particular variant is significantly better at converting visitors than the alternative.

In this tutorial you’ve learned how to create a basic content based split test:

  • Creating the split test
  • Creating content variation
  • Starting and stopping the split test
  • How to view and use the split test report

This completes the T1 tutorials, and you’ve learned the basics of the CMS part of Dynamicweb.

Overall you’ve learned how to:

  • Create your own website
  • Use Items and Item types
  • Modify Template structures
  • Use Razor templates
  • Implement Email Marketing templates
  • Configure Email Marketing
  • Import / Export users
  • Use Personalization (short)
  • Set up Split Tests