The Power of A/B Testing

A/B Testing is a powerful tool. Every now and then, a good A/B test reminds me of the power that A/B testing has. It’s the power to weed through customer interactions and decipher user preferences. It’s a power that I don’t want to take for granted.

A Step-Back Situation

This power to identify user tendencies led me to utilize A/B testing on a recent project for a client. The client was looking to create a signup form to capture user data. While it may be a pretty common task and goal, the client desired to create their own design. While we encourage designers to break out of their mold and delve into user interactions and web design, we do so with the hope that they’ll seek the guidance of seasoned web design and development professionals.

In this case, the client’s designer hailed from a print design background, and was largely unfamiliar with web design. Out of respect for the client’s wishes, we moved forward with their design, because we knew we had A/B testing in our back-pocket. Sometimes it’s best to step back and consider that some of our clients require a little more of an education to help them understand the challenges that face web designers and developers, and this was one of those “step-back” situations.

The Initial Design

After receiving the initial design from the client, we took a couple of hours to analyze the design, the purpose of the design, and develop a strategy. Our analysis led us to determine a few things about the design: it broke web conventions, branding was deemphasized, the primary action of the page was featured in the sidebar, and the secondary content dominated.

When we talk about web conventions, there are a few things that users have come to expect. A new user to a website or webpage expects to see a logo or some brand identifier at the top, or most-likely top-left, of the page. Unfortunately, the initial design didn’t recognize this convention. In fact, the branding for the product was in the bottom of the sidebar. Additionally, the primary content and secondary content were flip-flopped in the page’s hierarchy. All of the primary content was in the sidebar, while the secondary content was in the main-content area. Needless-to-say, I was a bit concerned about this design, but I knew I had my secret weapon.

Aiming the Secret Weapon for Maximum Results

As we know by now, this secret weapon is A/B testing. With A/B testing, I knew that I’d aim create new design options to offer up to the consumers. My analysis of the initial design led me to a few key deductions: the main content and call-to-action needed to be the primary focus for the user, the brand needed to be top-left for recognition, and the extraneous content could be reduced. With this knowledge, I created option B and option C to test against the default design. Option B featured the brand’s logo in the top-left, reduced content, and the primary content in the main-content area. Meanwhile, Option C featured the same brand awareness, even less extra content and a similar hierarchal structure.

My gut told me that Option C would create a conversion lift, mainly because other tests have shown that users tend not to read what they don’t have to and they typically like to know what they are signing up for.

In the end, my gut was right and Option C showed a 13% conversion increase over the default design. We were able to present the results back to the client and educate them using our findings. This education opportunity would have been diminished were it not for the power of A/B Testing (and a solid analytics team).