Split Testing Follow-Up Messages Using Variants

Split testing is the process of sending two or more messages, as part of the same batch, to a small segment of your contacts. Each message has different content and each contact will only receive one message.

The aim of split testing is to maximize your ROI by discovering which email message performs better with a smaller segment of contacts. When you know which message has the better results (the winner) you send the winning message to the rest of your contacts.

You can split-test every follow-up message with any number of variants. This means you can test out different aspects of any message to compare their success.

How do you split-test?

First, you must create a follow-up message in your autoresponder (or broadcast message). This serves as the main or initial control message. You then create a variant of the main message which you will test against the control.

When a variant is first added to a follow-up message, it is an exact copy of the original message. It can then be tweaked to test every element of your message such as:

  • Subject line
  • Message format
  • Content
  • Layout
  • Attachments

“Some message variants will produce better results than others so you can adjust the rate at which each is sent at any time”

To create or manage the variants of a follow-up message, choose the Variants option on the Choose action… drop-down menu or click the number under the Variants column.

After doing so, the following screen appears:

To create a split-test variant, choose Add a variant to this message on the I want to… drop-down menu.

This creates a copy of the original message and opens up the message editor for the variant you have just created. (In the image below, we changed the Subject line so we can split test the variant against the original.)

Once you have made your changes, click the Save Variant button.

You will then be redirected back to the Message Variants list screen, which now shows the variant you just created.

In the image below, take a look at the Frequency column.

Simply use the slider to change the percentage of contacts that will be receiving the variant.

The system decides at random which message will be sent to each contact during the sending process.