As a business owner or a brand manager, an important part of your role is to make the most of every opportunity that you have to capitalize upon your website traffic and to maximize conversions.
So what can we do to cut down on wastage and make our businesses more efficient? That’s where multi-armed bandit (MAB) testing comes in. Let’s take a closer look at how it can help you and your brand.
What is multi-armed bandit testing?
Essentially, multi-armed bandit testing is a more advanced version of A/B testing, which is also known as split testing or multivariate testing. It uses new technologies, such as artificial intelligence (AI) and machine learning to facilitate a more dynamic type of testing.
It can facilitate and iterate tests at a faster rate by removing the need to wait for the results of a single test. Think of it as testing all of the elements of a landing page at the same time instead of having to wait to do them one at a time.
Multi-armed bandit testing can work well for content-based websites, too. In fact, it’s how Netflix makes its recommendations. News sites, review sites, aggregation sites and social networking sites can all benefit from multi-armed bandit testing, and indeed the only real requirement is that you have the large amounts of traffic that’s required to make the approach viable.
The casino example
Still struggling to understand what this concept means? Imagine walking into a casino and looking at the one-armed bandit machines. If you’ve never been to the casino before, you have no information to guide you on which machines to use. The chances are that you’d pick a machine and stick to it. But imagine if you could go from machine to machine, perfectly playing the odds to maximize your winnings.
This brings us on to the multi-armed bandit problem. The idea underpinning the problem is that we need to find the optimum or most profitable outcome in a given situation based on a series of different choices. In a casino, this means figuring out which machine to use and how many times to use it. On a landing page, this means identifying the right elements to display people to encourage them to make a purchase.
What are its benefits?
As a growth marketing agency, we’re able to carry out multi-armed bandit testing for companies of all different shapes and sizes, and we’ve noticed a few different key benefits that companies can tap into. Just a few of the main benefits include:
1. Maximizes total number of conversions
This is really where multi-armed bandit testing shines, but it’s also where much of the confusion comes from. As marketers, we have a tendency to focus almost exclusively on conversions, because conversions automatically mean short-term sales.
Your number of conversions isn’t the only metric you should focus on, but it is one of the most important. Still, while boosting conversions is all well and good, if that’s all you’re using MAB testing for, you’re missing out.
2. Short-term optimization
A/B testing is very much a long haul approach in which slow and steady wins the race, relying on gradual improvements over time. This has its merits, such as on the main contact form of your website, which isn’t likely to go anywhere, it’s just that it might not work so great on a short-term microsite.
MAB testing really shines in situations where you don’t have the time for long-term testing. For example, let’s say you’re running a short-term summer campaign over a six-week period. With A/B testing, the campaign would be over before you had a chance to act on what you’d learned. With MAB testing, you can do your best to ensure the maximum number of conversions while the site is still live and online.
3. Reduce regret and lost opportunities
If you ask growth marketing experts why they’ve invested their time and energy into MAB testing, most of them will either cite the need for increased conversions or they’ll nod their head towards the lost opportunity cost.
When we talk about lost opportunities, we’re normally talking about the difference between the results that you actually achieved and the results that you could have achieved if everything went perfectly. Sometimes, when you get your approach right, there’s very little difference between the two. On other occasions, it’s the opposite, and you end up finishing a campaign and shaking your head and wondering why a promising campaign failed to deliver.
4. Give discarded options another chance
Another great advantage of multi-armed bandit testing is that it allows you to automatically re-test variants that might have been unsuccessful in the past but which might be more relevant today. The business environment is constantly changing and just because something didn’t work in the past, it doesn’t mean that it won’t work in the future.
There’s a perpetual element of randomness to MAB testing which is arguably the approach’s biggest strength. Sometimes it’s the most unlikely approaches that get the biggest results. It will also work the other way around, helping you to switch out high-performing pages once they start to deliver diminishing returns.
5. Cut down on traffic spend
When it comes to growth marketing strategies to boost your digital advertising revenue, there are really only two options. You can either spend more money to reach more people or you can focus on improving your conversion rate.
Increasing your ad spend can work as a short-term fix, but it doesn’t always lead to an improved ROI. And if your conversion rate remains the same, the only way to increase the number of conversions is to focus on driving more traffic.
MAB testing takes us along the other route, allowing us to improve the performance of our existing traffic. This can lead to a greater amount of revenue with the same ad spend, ultimately meaning that you can expect to make more profit and at a greater ROI. And of course, you can imagine what happens when you combine MAB testing with an increased ad spend to get the best of both worlds.
Does this mean A/B testing is obsolete?
Do you want the short answer or the long answer? The short answer is not completely.
The longer answer is that it really depends upon your own unique situation. If you’re trying to deliver a more personalized experience at scale while driving increased conversions through your website, you’ll want to stick with multi-armed bandit testing.
If, on the other hand, your objective is to collect data to make a critical business decision, A/B testing can help you to more tightly define your messaging and growth marketing strategies.
Think of multi-armed bandit testing as being more of a scattergun approach, while A/B testing gives you a laser focus. It’s the better option when you need statistical confidence or when you need to scientifically test two different things against each other. It’s much more like focusing your attention on achieving one goal, rather than many.
There’s a time and a place for both types of testing, and as with most things, the best thing is usually to aim for a balance. You can use multi-armed bandit testing to attempt to guess the best course of action for visitors at scale while simultaneously using A/B testing to carry out more targeted, specific experiments.
Examples of companies that use MAB testing
When we talked about news and content companies using multi-armed bandit testing to guess what kind of content visitors might enjoy, we weren’t talking about some sci-fi vision of the future. In fact, Yahoo has been doing exactly this since as far back as January 2014, if not before.
eBay, meanwhile, uses multi-armed bandit models to power their Marktplaats offshoot in the Netherlands. Functioning as part of eBay Classifieds, Marktplaats uses a recommendation system and tapped into multi-armed bandit testing as a solution to a very particular problem.
When you’re making huge amounts of money from your commissions, testing changes can end up costing you cash if you get it wrong. Instead of just abandoning it as a bad job and telling themselves it’s not worth the effort, switching to multi-armed bandit testing allowed the Marktplaats team to mitigate much of the risk.
Marktplaats used a technique called the epsilon-greedy method which is essentially designed to find the most successful strategy as quickly as possible. During the first phase, users were shown a random recommender, and the data from that quickly allowed them to move on to exploitation phase. Here, 90% were shown the best performer from the first phase, while the remaining 10% were exposed to another random variant.
There are many other brands that are pioneers of MAB testing (including other big names like Netflix), and we would encourage you to start thinking about it for the future of your business too.
Attracting new customers is crucial to your brand growth. You need to create high-quality leads through your online marketing channels and nurture them into paying customers. The strategies highlighted above will help you attract new leads regularly without spending too much.
Setting all this up will take you time, but Growth Marketing Genie can help you move your business towards success. As a growth marketing agency, we have the tools and talent to turn your website into a lead magnet.
Need more info? Let’s chat today about how we can make the most of your marketing opportunities.
Book in a Free Consultation