Advanced Split Testing

Most split tests (A/B tests) pit two or more versions of a page against each other to see which has the highest conversion rate. Variations can include simple elements such as the headline or an image, or more complex changes such as form actions and form fields or page layout.

But what if the requirement is to split test between multiple versions of an entire funnel, with changes on each page?

An example would be a landing page where the default variation has a contact form, and a test variation has a single button as the call to action, perhaps to view a video on a separate page with a contact form on the video page. On form submission, the variation needs to be recorded (to be saved to the lead within out CRM). The forms will then redirect to a thank you page, which shows content based on the variation.

In addition, we want to change the phone number on every page of the site based on the variation a user is served.

I will show how we create such a test with Convert.com, the A/B testing platform we rely on here at Aqua Media.

We cannot really achieve this complex example simply by using the platform interface. When a user hits the landing page, the Convert.com script requests a variation from the Convert.com servers, and executes any of the variation Javascript and CSS set for the variation. The landing page can obviously be modified easily within the interface, but subsequent pages need to rely on some way to track which variation was served.

We will use a cookie to store the variation.

1 To actually make the changes to the page elements, we will typically create a function within a sitewide Javascript that takes a variation ID as the argument. In addition, any variation-specific CSS should be added to the individual pages or sitewide CSS. We typically add a unique variation-based class to the body tag to identify the variation selectors. Once you have set up variations within the Convert.com experiment, use the preview links to build and test each variation.

Our javascript function:

function abTest(variation) {
  //Let's use 1a and 1b as variations
  if ('1a' == variation) {
    $('body').addClass('variation-1a'); //Add class to body for CSS selector
    //Insert variation 1a code here 
  } else {   
    $('body').addClass('variation-1b'); //Add class to body for CSS selector
    //Insert variation 1b code here 
  }
}

 

2 Within the page variation editor in Convert.com, we will just add Javascript to trigger our code above.

To add Javascript, navigate to the ADD JS menu item for the variation you want to edit.

Convert.com uses a different naming convention for jQuery, so use REED_$ instead of $. All we're doing here is calling our on-site function and including the variation ID (1b in this case). All logic will be in our custom function.

 

3 We save the variation in a cookie, and retrieve it when we need to know which variation to display.

 

function abTest(variation) {
  //Let's use 1a and 1b as variations
  $('body').addClass('variation-' + variation); //Add class to body for CSS selector
  if ('1a' == variation) {
    //Insert variation 1a code here 
  } else {   
    //Insert variation 1b code here 
  }
  
  $.cookie('variation', variation, {
    expires : 30,
    path : '/',
    domain  : '.' + document.domain
 });

}

Now, it's all pretty straightforward to apply changes to the page if your function is called from the Convert.com subscription, but on subsequent pages you will need to check for the cookie on body load or some other event trigger:

 

...
var variation = $.cookie('variation');
if (variation!==undefined) {
  $('body').addClass('variation-' + variation);
  if ('1a' == variation) {
    //Insert variation 1a code here 
    $('#phone').html('555-1234'); //vary the phone number on all pages
  } else {   
    //Insert variation 1b code here 
    $('#phone').html('555-4321'); //vary the phone number on all pages
  }
}

 

Example code above also includes a couple of lines that change the phone number on the site based on variation. In some cases you may also need to write the variation value to a form that will have a hidden field just for storing the variation ID.

In essence, to carry variations through across multiple pages, whether simple or complex, the best way may be to set a cookie with the variation identifier, and based on its value, modify the page.

 

Comments

Cooper @ Convertify's picture

Great to see someone who knows what they're doing technically. Personally we tend to look at the window.location.href object to determine the URL then run the code based on that value, but this is just as good!

You guys should definitely get some case studies up, I'd love to read 'em!

Add new comment

Plain text

  • No HTML tags allowed.
  • Lines and paragraphs break automatically.
The 7 Critical Components of a Profitable Website


Download the guide by entering your details below.

Enter your best daytime phone number
About Aqua Media

Just as water is critical to sustaining life, leads and sales are critical to growing and sustaining your business. We help our clients build and maintain systems for the attraction, conversion, and retention of clients, customers and patients.

From the blog
December 04, 2015
Get in touch

Aqua Media
#104 - 15028 32 Avenue
Surrey
British Columbia
V4P 1A3

(604) 560-2632

hello@bravodelta.net