A/B testing or split testing is a technique of showing two variants of the same web page to different segments of website visitors at the same time and comparing which variant drives more conversions. Companies want visitors to take action (also called a conversion) on their website, and the rate at which a site can drive this is called its “conversion rate.” The more optimized your funnel, the higher is the conversion rate.

ABTesting

Here is a little scrap project helping gettings started on your own on small websites.

A solution based on cohorts.js Custom solution based on Sweaver [Draft] Multivariate This requires cache disabling.

A solution based on cohorts.js.

Cohorts is a simple, purely javascript, multivariate testing framework. Unfortunately is no longer maintained.

Advantages: 100% control over how the code works & any integrations, self-hosted, free, unlike payed alternatives like Optimizely.

Disadvantages: Need to perform your own analysis

Steps

Paste the piwik tracking code Include cohorts.js in the page Instantiate and configure the Cohorts. Test object.

Now the variations are expressed as html ids named ‘big’ and ‘small’:

<!DOCTYPE html>
<html>
<head>
    <title>ABTesting</title>
     <script type="text/javascript" src="js/jquery/dist/jquery.js"></script>
     <script type="text/javascript" src="js/cohorts/lib/cohorts.js"></script>
     <script type="text/javascript" src="js/firstTest.js"></script>
     <style type="text/css">
     #big {display: none;}
     #small {display: none;}
     </style>
 </head>
 <body>
 
<div id="big" class="header">
    <h1>Big</h1>
    <button id="signup-button" class="btn btn-large">Button value</button>
</div>
 
 
 
<div id="small" class="header">
    <small>Small</small>
    <button id="signup-button">Button value</button>
</div>

$(document).ready(function() {

var bigsmall = new Cohorts.Test({
        name: 'bigsmall',
        scope: 1, // Sets the scope for the test and custom variable: 1: Visitor, 2: Session, 3: Page
        cv_slot: 1, // Sets the custom variable slot used in the GoogleAnalyticsAdapter
        sample: 1, // we want to include all visitors in the test
        cohorts: {
            big: {
                onChosen: function() {
                    $('#big').show();
                    $('#signup-button').html('Sign Up!');
                }
            },
 
            small: {
                onChosen: function() {
                    $('#small').show();
                    $('#signup-button').html('Register');
               }
            }
        },
 
        storageAdapter: {
                 nameSpace: 'cohorts',
                 testName: 'bigsmall',

                 // The TrackEvent method is not run directly, it's triggered by onInitialize and onEvent
                 // onEvent appends some info to the label so that it can be distinguished as a conversion

                 trackEvent: function(category, action, opt_label, opt_value, int_hit, cv_slot, scope) {
                    _paq.push(['setCustomVariable', cv_slot , action ,opt_label]); // Piwik with a custom variable
                    _paq.push(['trackEvent', category , action ,opt_label, opt_value]); // Piwik with an event
                 },
                // onInitialize runs every time a test does
                // It tracks total users each variant was served to
                  onInitialize: function(inTest, testName, cohort, cv_slot, scope) {
                     if(inTest) {
                        this.trackEvent(this.nameSpace, testName, cohort, 0, true, cv_slot, scope);
                     }
                 },
                // onEvent runs when a conversion occurs
                 onEvent: function(testName, cohort, eventName) {
                        this.trackEvent(this.nameSpace, testName, cohort + ' | ' + eventName, 0, false);
                 }
        }
    });
 
    $('#big').click(function() {
        bigsmall.event('Clicked on Header');
    });
 
    $('#small').click(function() {
        bigsmall.event('Clicked on Header');
    });
});

The snippet below represents the minimum configuration needed to track a page asynchronously.

<script type="text/javascript">
  
 var _gaq = _gaq || [];
  _gaq.push(['_setAccount', 'UA-8040421-1']);
  _gaq.push(['_trackPageview']);
 
  (function() {
 
    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
 
  })();
  
</script>
 
 
 
<!-- Piwik -->
 
<script type="text/javascript">
 
  var _paq = _paq || [];
 
  _paq.push(['trackPageView']);
  _paq.push(['enableLinkTracking']);
 
 
  (function() {
 
    var u="//localhost/piwik/";
 
    _paq.push(['setTrackerUrl', u+'piwik.php']);
    _paq.push(['setSiteId', 2]);
 
    var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
 
    g.type='text/javascript'; g.async=true; g.defer=true; g.src=u+'piwik.js'; s.parentNode.insertBefore(g,s);
 
  })();
 
</script>
<noscript><p><img src="//localhost/piwik/piwik.php?idsite=2" style="border:0;" alt="" /></p></noscript>
<!-- End Piwik Code -->
</body>
</html>

You now can reach the test following: http://www.example.com/page.html#my_test=variation_a

Google Analytics is the default data storing.

If you want to change it specify an object.

myStorageAdapter = {
    // Called when the Cohort.Test is initialized, and the visitor is resolved into a cohort.
    //   inTest: whether the visitor is in the test
    //   testName: the name of the test
    //   cohort: the cohort chosen for the visitor. Will be null if the visitor isn't in the test
    onInitialize: function(inTest, testName, cohort) {
        // do initialization stuff
    },
     
    // Called when the event method is for the Cohort.Test instance is called.
    //   testName: the name of the test
    //   cohort: the cohort of the visitor
    //   eventName: the name of the event the visitor triggered
    onEvent: function(testName, cohort, eventName) {
        // do event stuff
    }
}

Integration with Drupal

Tracking anonymous users.

We need to avoid invalidating the Varnish cache.

We’ll be using jQuery.cookie provide by Drupal in core.

This allow us creating, reading and deleting cookies.

This library is not loaded unless specifically loaded:

function abtesting_init() {
  drupal_add_library('system', 'jquery.cookie');
}

You can achieve the same thing enabling it from hook_preprocess_page.

// Detecting the users is returning based on a javascript cookie
(function ($, Drupal)
  var basePath = Drupal.settings.basePath;
  
// generate a random between 0-10 and sort users out
 random = mt_rand(0, 10);
  
// sort users into one of the 2 groups
  if ($random < 6) {
    // User belong to group A
    userGroup= 'a';
  } else {
    userGroup = 'b';
  }
  
   // set a cookie to track where he belongs
   $.cookie('userGroup', userGroup);
  
 // append very up in the DOM the desired css file and avoid the caching
  $('<link>')
    .appendTo('head')
    .attr({type : 'text/css', rel : 'stylesheet'})
    .attr('href', basePath+'/css/abtesting-'+userGroup+'.css');
)(jQuery, Drupal);