How to do A/B Tests in WordPress?

There is a wide variety of plugins for A/B testing in WordPress, with a wide range of additional features and services.

Many providers offer a free version, with only a portion of the features available, or an unlimited but time-limited trial version.

However, if we have never done A/B tests before, the learning curve with some of these plugins can distract us from the important aspects of the whole process.

On the other hand, most also offer other types of tests, which can further complicate their understanding and configuration.

We have a better option to consolidate the concepts of the A/B tests and to know well their execution dynamics, through the Google Analytics Experiments, in the Behavior menu of the main Reports tab:

Since we are used to the Google Analytics environment and nomenclature, the learning curve is very smooth and we can focus our efforts on what really matters: understanding, running and analyzing the A/B tests.

In addition, as it is a Google tool, we make sure we are following the best practices for running A/B Tests.

Although there are many plugins for A/B Tests in WordPress, we can do them effectively with Google Analytics experiments.

1 – Create a new experiment

Assuming this is the first time we do an experiment, Analytics displays a blank list:

Then we click on the “Create experiment” button and Google shows us the four steps we must go through and fill in to configure our experiment:

Let’s see the fields to be filled in in the first step, “Select an experimental target”:

  • Name of this experiment

The name by which we want to identify and recognize this experiment in the corresponding list.

We must ensure that the name is meaningful and related to the purpose of the experiment, e.g. “Rebound rate on the registration page”.

  • Purpose of this experiment

Select the metric we have decided to analyze after the test run.

Analytics shows us a list of available metrics, but we can create our own targets.

  • Percentage of the traffic of the experiment

Be careful! Don’t confuse it with the distribution of visits at 50% of the A/B test: it has nothing to do with it.

It refers to what percentage of the total traffic will be devoted to participating in the experiment.

That is, if we set it at 60%, the A/B test will only be performed on 60% of the visits.

We usually leave it at 100%.

  • Mail notification of important changes

We enter our email address so that Analytics can notify us if an incident occurs.

Before continuing with the next step, we must click on the “Advanced options” link, as this is where we will set up an A/B test:

  • Distribute traffic equally among all variants

We must select “Yes”.

If you select No, we would no longer be doing a traditional A/B test, but an adaptive test according to the selected metric (in case you are interested, this type of test is called multi-armed bandits).

  • Advanced options

The other advanced options define how Analytics will determine when there is enough data to complete the experiment.

We can leave them at the default values.

Note that Analytics decides the duration of the experiment for us, through the confidence limit.

Using statistical analysis and the value of this parameter, Analytics determines when enough visits have been made so that the results of the experiment are sufficiently representative (both for good and for bad).

What a job it takes to get us off our backs!

2 – Set up the experiment

Click on the “Next step” button and enter the addresses of the control page (i.e. the original) and the address of the variant (i.e. the page with the change you have decided on) in the form provided:

Before entering the URL of the variant page, we must have created it in our WordPress, as Analytics verifies that it exists and loads correctly.

Note that we can add more variants of the page.

Analytics offers possibilities beyond the A/B tests, a sample of its versatility, but it can also complicate the analysis of the results and the test will take more time, as visits have to be spread over more versions of the original page.

Click on the “Next step” button.

3 – Configuration of the experiment code

This is perhaps the most delicate part of the whole Analytics configuration process to make our A/B test operational, since we must copy a JavaScript code into the original page of the experiment, inside the <head> tag of the HTML code:

You have a plugin (Google Content Experiments) that allows you to add this JavaScript code from the same screen where the WordPress pages are edited, only in the original page of the experiment:

4 – Review and begin

Once this JavaScript code has been copied, click on the “Next step” button and Analytics verifies that all the settings are correct

If everything went correctly and Analytics did not detect any errors, we can start the A/B test by pressing the “Start experiment” button.

5 – Analysis of the data of the experiment

Although the final results will not be available until Analytics considers that it has sufficient data, depending on the confidence limit that we have set, we can consult how the metric chosen for the original page and its variant varies, selecting the experiment from the list.

As it may be several weeks before Analytics shows some relevant information in the report, I’ll put the next report from one that has been running for several months so that we can interpret the information it shows:

As you can see, it is a more complex experiment than an A/B test, as it includes 3 variants of the original page, so it requires considerably more execution time.

Although it has not yet been completed, it is already seen as one of the variants is a clear loser (with 94% fewer conversions than the original).

Meanwhile, another variant has good prospects as a candidate, with 7% more conversions, although with a probability of surpassing the original still a little low (66.11%), hence Analytics has not yet completed the analysis.

In this case, given the length of time this experiment has been in progress (more than 5 months) and without any conclusive results, it would be necessary to consider whether it is worth continuing to be carried out.

One possibility, which would offer results in less time, would be to create a new experiment, but this time a real A/B test, with only the original page and the variant most likely to succeed.

Conclusions

A/B Tests offer a method to improve the metrics of a website, usually the Landing Page, that has, for example, few conversions, high bounce rate or short page dwell times, all without having to invest in SEO or SEM positioning campaigns.

With the A/B tests we can compare two versions of a website to find out which of them produces better results.

While there is no limit to how different these versions can be, changes should be implemented step by step, with successive tests identifying which ones are beneficial and discarding those that may harm us.

If you take a quick look at the article, you’ll notice that more than three-quarters of the article is about what A/B tests are and how they should be done, while only a quarter explains how to implement them in WordPress.

This division is not the result of chance: the actual implementation and execution of A/B tests is incredibly simple.

What really makes the difference is that we fully understand its dynamics, what we want to achieve, how we can achieve it and how to evaluate the results.

Never rush into running an A/B test without being sure of what you want to achieve.

Finally, a word of advice: do not expect spectacular results after a single A/B test.

Achieving such results requires doing and analyzing many A/B tests, until you find the changes that, accumulated, provide important improvements.

Examples of conversions that are multiplied by 10 due to a button color change are that: an example.

In such cases, act a little like devil’s advocate: if only the color of a button could achieve such wonderful results, why don’t all websites have high conversion rates?

With what we have seen in this article, how do you think you could have achieved better results?

And if you’ve never used them, can you think of a place where you could do A/B tests on your website?