Over 10 years ago in a college marketing class, I learned something surprising about those omnipresent ninety-nine cent endings on price tags: Most businesses actually had no evidence they increased revenue vis-à-vis whole dollar prices. If a shirt was listed at $19.99 at a department store, it was often because a marketing executive simply believed that was the right approach (or read generic academic research).
That seemed foolish to me back then. It should seem ludicrous to you today…especially if you’re running an online business.
In your company’s DNA
Compared to our “offline” brethren, an online business is able to conduct tests on an impressive amount of business decisions. The question is: Does your company take advantage of the tools that enable such tests?
I heard this mindset named a “culture of testing” by Netta Marshall, who’s working to implement such a culture as head of design at non-profit Watsi. “We want data behind almost every decision we make, from logos to landing pages to new features,” Marshall said.
Philosophically, this means pointing to concrete data instead of making subjective arguments. Of course, no organization will have data behind every single decision. So in practice, how do you tell if you have a culture of testing vs. a “culture of gut feel”?
A company with a culture of testing does not pick a landing page based on which one the CEO likes, or which one the designer thinks is the most attractive. Rather, it runs A/B tests on different variants, using a tool like Optimizely or SiteSpect. (Protip: If you have low traffic, be sure to test drastically different variants – you won’t reach the sample size needed to see if the blue button beats the red button.)
Such a company also does not assume their product works well just because it feels right to them. Instead, it constantly watches strangers use their product; many entrepreneurs recommend UserTesting.com for this service. A company with a culture of testing also compares different conversion funnels by always diving into their web analytics (KISSmetrics, Google Analytics, and Heap are popular options).
A culture of testing naturally leads to lots of survey work, too. My company Survata lets companies poll their target audience for any business decision, from gathering feedback on new product packaging to picking a name for a new app. If you want to ask your existing customers why they use your website, you can plug in Qualaroo.
Every month, new marketing services are being launched, so never hesitate to update your testing arsenal.
Know the limits
Of course, all tools have limitations, so businesses still need to understand when a “gut feel” decision is needed. Sometimes you won’t collect a statistically significant amount of visitors for an A/B test, and sometimes the margin of error from your survey is so high that it blurs the takeaways. Testing will often provide a directionally correct answer, which you need to augment with your own observations.
Just as importantly, be careful which metric you’re optimizing. If you’re testing multiple landing pages, one may have the highest click-through rate, another may lead to the most sales conversions, and yet another may lead to the highest customer lifetime value.
Finally, ask yourself if your organization is actually ready to commit to this culture. “A true culture of testing requires a very high degree of intellectual honesty,” said Robert Moore, the CEO of RJMetrics, which sells business intelligence software that enables data-driven decisions. “Everyone at the table needs to trust the data and allow it to change their opinion. If you're not going to change your behavior no matter what the data says, you shouldn't run the test in the first place.”
Of course, don’t let these risks scare you away. Go forth and test!
