A/B testing, also known as A/B tests or split tests, is a procedure used with digital medias to test two variants (A and B) of an element.
The variables tested can be complete marketing elements such as a web page (landing page), a visual communication or a newsletter, but also individual components such as a title, an image or a colour.
The aim of an A/B test is to analyse user behaviour in order to improve conversion.
One of the great advantages of digital media is the flexibility they offer for real-time adjustments or corrections. In traditional media, nothing can generally be changed after printing or distribution.
In the digital world, it's quite different. Whether it's on a website or in an advertisement, adjustments can be made at any time and in a matter of seconds.
It therefore makes sense for marketing professionals to constantly test and adjust different variations of their marketing assets. The variables to be tested can include a variety of elements, such as :
- Text variations
- Different images
- The colour or shade of a button or background colour
- Font type or size
The test can also cover less obvious elements such as:
- Different target audiences of an ad or newsletter
- The timing for sending a newsletter - time of day or day of the week
- The landing page where the user should be redirected to after clicking
When conducting an A/B test, it is important to test only two variations of a variable. One version A and one version B (in the case of several simultaneous tests, we speak of multivariate tests).
The two versions are tested simultaneously against each other, dividing the users (traffic/recipients) into two random groups.
The test continues until one of the two variations shows a statistically significant improvement.
In order to achieve this significance, there must be sufficient interactions with the respective variations.
In other words, you need enough users/traffic for each of the variations. A website with many users will achieve a result much more quickly than a small website with few visitors. For advertising, the success also depends on the reach.
The winner of an A/B test is the variation that generates a better conversion rate.
Keep in mind that conversions are not just sales. They can also be micro-conversions such as clicking on a link, subscribing to a newsletter or filling out a form.
Nowadays, it's relatively easy to set up A/B tests for digital advertising. Almost all digital advertising providers have included the option of A/B testing in their systems.
For example, you can enter two (or even more) variations of your ad on Facebook or Google Ads. This option should always be used to continually optimize the use of resources.
Testing elements on a website or mobile application is a little more complex and requires a technical solution that divides visitors equally and randomly between the two variations and ensures, using a cookie, that the same variation is always displayed to the same visitor.
However, there are now many suppliers offering both free and paid solutions. Google, for example, offers Google Optimize, a free tool linked to Google Analytics.
What if the test doesn't produce a statistically significant result? No problem! This can often happen. In this case, simply try other variations of the test or extend the test period to obtain more data.
A/B testing is an ongoing process that can be very profitable. Companies like Google and Facebook test several hundred variations on their different pages around the world every day.
A well-known example is Google's "50 shades of blue" test. The aim was to increase clicks on ads. To find the best solution, Google tested 50 variations of the colour blue. The result was a $200 million increase in advertising revenue.
So it may well be worth it!
This text was originally written in German by the author and published in the book "Marketing Buzzwords Debunked" available on Amazon.de.