Skip navigation

Real-Time A/B Testing vs. Traditional A/B Testing in Email Marketing

Share This Post

Real-Time A/B Test

In the world of email marketing, an A/B test is one of the most important tactics for delivering the best possible email to your audience. A tiny difference in a subject line or a slight change in copy can make or break an entire email campaign.

But whenever you A/B test in email marketing, there’s a problem – part of your list gets the losing creative. If you find out that one image or set of copy really outperforms another, that means you miss out on potential sales and never realize the full power of the winning email.

Real-time emails can fix this. By allowing email marketers to change the content of an email after send, real-time emails can fundamentally change how A/B testing is done. At Movable Ink, an element of our contextual marketing platform can perform real-time A/B testing that we call “creative optimizer.”

Here’s how it works:

The Creative Optimizer

The first steps for using the creative optimizer are similar to a traditional A/B test. You create two sets of creative, Option A and Option B, and divide your list in half. Send Option A to one half of the list and Option B to the other half.

This is where real-time A/B testing comes into play. By using the creative optimizer tool, email marketers can set a threshold for performance. So if Option B starts outperforming Option A and the difference in click-throughs reaches 25%, you can set a rule so that, automatically, all the emails showing Option A start showing Option B, instead.

Even if customers have opened the email already, they’ll see Option B if they open it again. Along with optimization, real-time A/B testing can be highly useful for deploying new product lines and seeing which resonates best.

Real-Time A/B Testing in Action

Allen Edmonds, a retailer known for finely crafted footwear, wanted to push a product line of apparel this past February. In order to see which message resonated the most, the company sent two different sets of creative.

Option A put the sale front-and-center. Option B emphasized the line of apparel instead:

Real-Time A/B Testing for Email

After it was clear that Option A was dramatically outperforming Option B, Allen Edmonds automatically swapped all the creative so that both emails showed Option A instead, resulting in a 114% increase in click-through rates (check out our full Q&A with Allen Edmonds about contextual marketing here)

In this way, Allen Edmonds used real-time A/B testing not just to ensure that every customer got the best possible creative, but to see what kinds of imagery and copy resonated best.

If this had been a traditional A/B test, the losing creative would have been sent to half of the email list, which would have resulted in far fewer clicks. With real-time A/B testing, the opportunities for making the best possible email grow exponentially.

Contextual Marketing eBookReal-time A/B testing is all about context. Want to learn everything there is to know about contextual marketing? Download our eBook, “Everything You Ever Wanted to Know About Contextual Marketing.”

Marketing News that Matters

Sign up for Movable Ink's newsletter to receive the latest news, research, and omnichannel personalization resources.

Sign Up Now