Advanced A/B Test Training Series: Intro [1/5]

Alon Even, on August 2, 2020

[1/5] Advanced A/B Test Training Series: IntroSay you injured your knee while out on a run and went to one doctor who recommends surgery and rehabilitation.

Before following this doctor’s advice, you first go to another doctor who recommends physical therapy without surgery.

What you’ve done, of course, is what we call getting a second opinion.

By going to two doctors, you can now weigh each of their recommendations and decide which one works best for you – a decision that could come down to money, time, and the likely success rate of each approach.

The concept of a second opinion is not too different from A/B testing – also called split testing.

It allows you to compare multiple versions of your ad layout to determine which one will be more effective at generating increased scale and/or viewability, while keeping the UX intact.

And of course, as a publisher, this is your goal. Improving your ad layout will result in higher CPMs that will drive higher RPMs.

The ABCs of A/B testing

A/B Testing


So, what exactly is A/B testing?

In simplest terms, an A/B test is an experiment.

And like with any experiment, you have a ‘control’ (which would be your ad layout) and a challenger or variant (which would be your ad layout but with a change in one or more of the elements).

What makes A/B testing so powerful is that you can test two granular elements of your inventory to determine which one best meets your KPIs.

For example, you can A/B test two sizes of the same ad to see which one generates increased viewability to boost your CPMs.

In the simplest terms, the process takes two versions of a single variable and tests them against each other until you find a winner.

This winner will be chosen based on KPIs you’d like to improve and set for your A/B test.

What the publisher revenue equation has to do with that 

A publisher’s revenue equation is made up of three factors:

publisher’s revenue equation


What this means is that driving the traffic from a high-quality audience, optimizing ad inventory, and improving demand will lead to higher revenue.

A/B testing your ad layout before an ad call goes out is an important first step to getting the best performance out of this equation, essentially providing a quick win for publishers.

This optimization will help you increase viewability and scale, while maintaining a superior user experience, which ultimately leads to an ad layout that performs better.

So what do you get out of A/B testing your ad layout?

Without A/B testing your ad inventory, you’re essentially making assumptions about what your audience will respond to without testing your assumptions to see if they’re right.

With that in mind, here are three benefits to A/B testing your ad layout:

  • Increased revenue

Avoid leaving money on the table.

Having the wrong size, placement, or format for an ad may generate lower impressions or decrease your viewability score, among others, leading to decrease in your ad revenue.

Without A/B testing an ad layout, you are making assumptions about user behavior, and merely guessing at how your ad layout will perform.

It also allows you the opportunity to make changes before an ad call goes out  — after which point, you’ve missed your opportunity to address the necessary adjustments that should’ve been made.

By running an A/B test on the various elements of your ad layouts, you can maximize the performance of your ad inventory.

  • Greater impact  

This is a real place where you can make an impact on your bottom line, and not just through the content you post.

As a publisher, A/B testing means that you are looking at all aspects of your website’s layout, both ads and content, which gives you a more holistic view of your site.

By analyzing all aspects of your design, including headlines, content, images and ads, you can better optimize the performance of your ad inventory.

By seeing the big picture of your ad inventory, you’ll be running more effective A/B tests with a greater impact on the KPIs you’ve set.

  • Low cost, high in reward

Whatever it might cost you to conduct the test, you’ll make up for it by optimizing your ad inventory.

All you need to run a successful A/B test is the right platform, such as Optimizely provides, and create a variant, making it a low-cost endeavor.

Don’t forget.

You can go up and running with the A/B test in a relatively short time, and analyze the results once you reach the right sample size for each variation in your test, ensuring their validity.

And then by determining what elements maximize the performance of your ad inventory, making the required changes to your ad layouts will probably generate higher ad revenue.

Why you should go beyond manual A/B testing  

The benefits of A/B testing are clear.

But, one of the real challenges you’re probably facing as publishers is that the process can be very labor-intensive.

Keep in mind that in order for you to maximize the value of your A/B testing, you should be running multiple tests.

In other words, running your control up against several different variations to determine that you’ve got the most effective layout.

Attempting this utilizing manual optimization is extremely time-consuming and not scalable.

Bear in mind that content is consumed in real time, and each user engages with your content differently.

So, even if you reach a point where you are running multiple A/B tests, due to the dynamic nature of your ad inventory, it’s nearly impossible to create an effective layout.

Jump on the real-time, automated optimization train

One ad layout doesn’t fit all.

This is why real-time, automated optimization is the best approach to create the best performing ad layouts.

By processing multiple data points in real-time driven by AI-powered platform, you can automatically personalize ad layouts per user and page.

This will produce the highest combination of viewability, scale and UX.

And then you’ll be able to unlock the true potential of your ad revenue.

What’s next?

A/B testing can be a good start for optimizing your ad layout.

However, I’m sure you don’t want to settle for less than the optimal way.

The most efficient and effective way to optimize your layout is through an AI-powered platform, such as that provided by Browsi Revenue OS, which will continually improve your ad inventory automatically and in real-time.

Want to give Browsi a try? Talk to our product specialist today. 

Related Posts