If A/B Testing Can Help Win Elections, What Can It Do for Your Business?

By Dan Siroker , marketingprofs.com, 28-04-2014

Online retailers are always looking to differentiate themselves in more meaningful ways. Compelling shopping experiences, product recommendations, and overall superior customer service are key ways retailers set themselves apart. Today's data-driven marketing tools can help them unlock those experiences by using the data they have about their customers.

Prior to founding my own company, I served as the director of analytics on the 2008 Obama campaign. By experimenting with changes to elements of the campaign splash page, we were able to help raise an additional $57 million in campaign donations.

The guiding principles that made the campaign successful are no different from approaches online retailers and other marketers can adopt to make a real business impact.

First, you need to know your constituent. The behavior of visitors who come to your website is very much indicative of the kind of messaging that would work on them. What you show a returning visitor is different from what you'd show to a new visitor, or a mobile visitor vs. a desktop visitor.

The urgency for businesses to use data to show the right thing, to the right person, at the right time is stronger than ever. Targeted messaging is the most to effective way to get consumers to convert.

Second, you must know the facts. One of the greatest challenges (and areas for error) for businesses is getting the data right. Online retailers should take prudent effort to make sure the infrastructure and process they've implemented is sound. For example, the "novelty effect" suggests that just because a change has an impact initially doesn't mean it can be sustained over time. Will your customers grow tired of cyber deals every 10 minutes for a week?

Finally, it's imperative to ask the right questions. The challenge for online retailers is not about prescribing the right answers but about asking the right questions. For instance, are you trying to see whether a visitor will respond better to more product selections on a page, or fewer? Consider first what you want the answers to be, and those hypotheses will then help you decide what to measure.

Common examples of A/B testing for online retail include homepage bounce rates, category-page views, product-page views, shopping cart ads, and all stages in a checkout flow all the way to the Thank You page.

In general, to get more effective and relevant results, rather than asking "What are the variations we are testing?" consider asking "What question are we trying to answer?" To ensure the best return on your effort, first look at your Web analytics to see which pages have the most room for improvement. You'll want to attack those areas first.

The following are some actionable insights to help retailers optimize their sales by running website experiments that help them deliver a better experience to their visitors.

Homepage. The homepage is notoriously the most over-scrutinized page, yet it is also likely to be the most under optimized. For instance, imagine you're a consumer shopping on a retailer's website for a new coat. You arrive on the homepage and see a banner for a sale. Just as you're about to click it, the experience changes to pants. The rotating carousel of images means you have to continually reorient yourself, and it diffuses the focus of your original purpose. And when consumers are distracted, they're probably not purchasing.

Category pages. Unfortunately, most retailers tend to overlook their category pages; luckily, there are some easy and effective tests to evaluate them. One simple experiment centers on the performance of tiled vs. list views. For example, in our experience, the list views perform better and lift sales for scenarios in which consumers are making a complex purchase decision. The list format enables consumers to scan information easily and compare between categories; it also gives the retailer space to display the best sellers above the fold.

Product detail pages. Where does the consumer ultimately decide whether to buy or bounce? The product detail page is where the final persuasion happens. Therefore, it is one of the most important areas of your website.

Often, retailers are looking for a solution to a distinct challenge: solid brand awareness, but poor conversion. In that scenario, it's difficult for the retailer to determine what to improve. To increase conversion, I recommend carefully examining the following key conversion factors in the product detail pages:

• Value proposition: Is it strong or weak?
• Relevance: Is the content pertinent to the target audience and their needs?
• Clarity: How clear is the imagery, eye flow, copywriting, and call to action?
• Distraction: Are you redirecting attention from the primary message with too many product options? Are upsell and cross-sell options provided prematurely? Are design elements overwhelming the message?
• Urgency: Are you giving the consumer a reason to act now?

Primary calls to action. At the sitewide checkout entry point, make your calls to action loud and clear. Test phrases like "add to cart" or "sign up for emails" to clarify what specifically you want your website to achieve and whether you are effectively directing shoppers to accomplish those goals.

For example, knowing that users become more invested as they click through the signup funnel, the 1-800-DENTIST team hypothesized that making the first step as simple as possible would decrease drop off rate and lead to more successful signups further down the funnel. To test the hypothesis, the team considered how to best simplify the first step without losing valuable data collection. Since all dentist matches depend on location, ZIP code was the most logical input to lead off with. Then, the team moved the two other fields—insurance and dental need—to pages later in the funnel, ensuring they would still be able to collect each piece of information. In less than a week, the team found that shortening the first step of the checkout funnel increased conversions 23.3%.

* * *

The greatest opportunity for online retailers today lies in facilitating the process of experimentation to help their teams move from the era of Mad Men into the era of Math Men. The role of creativity is still as important as it's always been, but now it's not centered on the most intuitively creative person but, rather, the most data-driven creative person.

We live in a time where you can let the data help you get to the right answers. You'll quickly find that the tests take the guesswork out of website optimization and enable data-backed decisions that shift business conversations from "we think" to "we know."

And knowing your website's weaknesses means that you can turn them into strengths and sales.

Read more: