Failing Forward With Your A/B Testing Plan

As a digital marketing agency focused on cultivating a Challenger Mindset every day, we have a love (obsession) with A/B testing. Whether it’s subject lines or high traffic webpages, we’re always ready for a test.

One ecommerce client came to us with a common problem: they had a product that wasn’t converting as well as they wanted. They’d tried several tactics to improve results, but they hadn’t found lasting improvements.

While they started with adjustments that made sense, only the data could decide what was right.

A/B testing is crucial in determining what triggers your audience to convert. What may work in one instance may not work for another, and we constantly drive strategic testing plans to see what’s performing now.

For this client, we quickly saw the impact of video when A/B testing their main landing page. Our experiment showed that using video in the header decreased the cost-per-acquisition (CPA) by 45%. That was a huge early win.

From there, we added CTA buttons in a high contrast color where appropriate. These buttons decreased CPA again, this time by 10%. While not as in-your-face impressive as the first test, these results were still significant since the KPI in question was so meaningful.

After these tests, we tried changing the header background color, again creating more contrast. But CPA actually increased by 6%. While the intention followed the logic of the previous test, and the updated version was arguably more aesthetically pleasing, the data showed that conversions didn’t follow.

Failing Forward

When testing, it’s easy to either obsess over these missteps and blow them out of proportion or do the opposite and sweep them under the rug while focusing on the wins. But neither of these approaches helps you to move forward and detach from the test. To find optimizations, we have to accept that there will inevitably be losers. Whether that’s the new change you made or the current site you have, one will be the loser.

Failing forward is this process of accepting and moving on.

We like to think that every A/B test comes back with results that show us the path to the “ideal” landing page/banner ad/lightbox. But if you’re testing enough, you will sometimes come up results that don’t improve your asset. And that’s okay. The Challenger Mindset doesn’t demand that you always win, but rather it requires that you fail forward and let data guide the way.

We help turn Challengers Into Champions every day and email marketing A/B testing is just one way we do it! Contact us to get started today.