What is A/B Testing?
A/B testing is a method for working out what the most successful online marketing and promotional strategy for your business. A/B testing can be used to test all kinds of things such as website copy, sales emails, CTA, search ads, and more. While running A/B tests can take more time, the additional time is typically outweighed by the benefits provided by the test results.
With well-designed and implemented A/B tests, well you can make an incredible difference to how effective your marketing becomes by increasing the ROI of your team’s spend in a cost-effective way. By finding the most effective promotion elements and combining these, you can ensure that your marketing skills can bring a lot more success and become more profitable in your campaigns.
How is an A/B test is planned.
The first step is to work out what you want to test. Is it a test on-site or off-site? An on-site test will require you to consider which sales related areas of your website that you’re planning to test. Whereas an off-site test will either be to test a sales email or an ad. For instance, by testing ad copy you can discover which ad is attracting the most converting visitors, which will then lead you to figure out where you need to focus your advertising. Once the best converting ad is determined and is generating as many conversions from customers as possible, you will no doubt feel that the money spent on it is justified.
The same can be said in testing emails. For example, if two different emails are sent out randomly to your list and you capture the conversion rates for each, it will be clear to see which email has created more conversions, therefore you will just send out that email rather than the two.
Getting started is as simple as having a hypothesis about what to test first. When you’ve established what it is you want to test, whether it’s a CTA button on an email or images used in an ad, then create a list of the different variables.You will probably include; the call to action location, which exact text is being used, the colour button and surrounding space, and so forth.
A/B testing is a process and often involves making numerous tests to distinguish the best way forward as well as finalising your decisions and choices of what to change. One of the most important aspects of A/B testing is setting up a scalable, reliable technical infrastructure to ensure that you’re always confident about the test results. Once this infrastructure is built, it requires minimal maintenance, thus empowering marketing teams to run multiple tests relatively autonomously.
Validating your initial hypothesis is the objective of each A/B or multi-variant test. It is important to make yourself aware of what results you are looking for when you begin testing. It is key to document your current baseline results, so that you can easily compare them to the new results once the A/B test results are available. Or, alternatively you may want to set your current baseline results as A to compare with those of B.
It is also important to carry out these tests simultaneously to ensure that the results are fairly representative at the same point in time, because if one test is done one day and another test carried out on another day, there may be different variables that will affect the results. To stop this from being the case, testing is carried out by splitting the traffic at the same time, and usually over a period of time deemed long enough to achieve statistical significance in the result. For instance, if you have low traffic to a landing page and only run an A/B test on a change to a banner image for a few days, the results may not be reliable because you don’t have enough data to confirm the results.
Bottom line improvements – A/B testing
A/B testing can make a positive impact on your bottom line sales. By using these controlled tests and analysing the results, your business can dramatically improve results by focusing on the key marketing areas and in turn creating more sales. Planning out each test properly and carrying out A/B testing consistently, having the facts and figures to prove it is working well, and reviewing the results carefully to ensure that you are increasing sales, may all add to the value of your company. It is also important to bear in mind that many things can change over time so regular testing needs to be carried out – embedding A/B testing into your team’s process for driving conversion as a recurring tactic is recommended.
What Can You Test?
Almost everything can be tested in the marketing material you are using such as: headlines, images, body copy, call to action copy or design, text size, and many other things. A good rule of thumb to keep in mind is that if it can be changed then it can be tested.
However, that doesn’t mean that testing absolutely everything is the right approach. Instead, you just need to choose to test changes you predict are more likely to achieve a better result, including; the headline, your calls to action, sales graphics, and the sales copy or product descriptions. These are also similar to the things you would look at changing within your emails. Generally, there are less items to change within an ad and even less in a text ad which is similar to a search ad. These ads usually involve just the main headline being tested or the actual offer itself.
If you do test different offers, it is important to ensure that the same promotion is offered to each person receiving it. This is explained in the following example; if a free gift is offered to group A, and a discount is being offered to group B, then you need to ensure that group A always contains the same visitors, as does group B.
You may also find testing things in conjunction with each other is very informative, for example, if you tested newsletter A with landing page A, and newsletter B with landing page B, then, later, you tested newsletter A with landing page B, and vice versa. This can give you a more accurate result especially if you were getting mixed results, or close results previously.
Commitment Of Time To Testing
A/B testing can be a time consuming process, from the initial setup to the time required to design and plan each new test. The former may may require a few days or can even take a number of weeks. The length of time you should run the test will depend on the amount of traffic you have. Remember that running one test at a time is the best method for obtaining the most accurate results.
Not giving enough time to a test can lead to inaccurate results, due to not having enough visitors to reveal statistically significant, or realistic results. On the other hand, running a test for too long can also give inaccurate results due to there being more variables you are unable to control over a longer time period (for instance, an A/B test that runs over a holiday period could skew the results of your tests). It is important that you are looking out for anything that could affect your results, so that you can account for any statistical anomalies you may have picked up on when reviewing your results. If you’re unsure of results or have any doubts about certain things, it’s preferable to run the test again.
The impact A/B testing can make on your bottom line certainly makes it worthwhile to make sure they are properly carried out over a number of weeks, testing one thing at a time.
Can More Than One Thing Be Tested At A Time?
This question has two approaches.
Say for example, it was your headline you just wanted to test, but with three possible variations. In this case, running a single test and splitting your visitors into three groups is fine (or recipients of an email), it is still likely still be considered as an A/B test. It is more efficient than running three separate tests like A vs. B, B vs. C, and A vs. C.It may also be beneficial to give your test an extra couple of days to run, so that you have enough results on which to base your conclusions on.
Typically, it is not recommended to run more than one test at a time on the same group of customers, or on the same page. That is, don’t more than one change to an element in an email template at once, since you won’t be able to confirm which change actually made an impact. Similarly, don’t include a given user (like User A) in two tests, since if they do end up converting, you know know which test was what led to that conversion.
In Conclusion
In conclusion, A/B testing can be one of the most impactful ways to drive conversions and increase sales in your business, provided that you approach the design, planning, and analysis of the tests carefully. Investing the time to build smart tests will give you detailed confirmation about your hypothesis and lead to increased value to many teams in the organization.
There are lots of tools out there that can help you with A/B testing emails. While you will need to rely on your email service provider for A/B testing functionality when sending emails, you can use tools like Dyspatch, an email template builder that gives you the ability to easily create templates that you can A/B test. Check out the Dyspatch visual builder for building high quality emails.
