Facts are simple and facts are straight
Facts are lazy and facts are late
Facts all come with points of view
Facts don’t do what I want them to
—Crosseyed and Painless, Talking Heads (written by Brian Eno and David Byrne)
Business growth at established companies tends to fall relentlessly over time in the absence of inspired innovation, an impact I affectionately refer to as “gravity”. If a CEO wants to fight this gravity and improve the long-term growth trajectory of his or her business, he or she needs to take proactive, concrete steps to make it happen.
As CEO, I was always trying to develop and test a portfolio of potential new initiatives to support business growth, targeted at both optimizing the core business as well as adding new layers of growth (see my last post on this topic about Adding Layers to the Cake). I wanted to identify completely new initiatives that would boost business growth—things we weren’t already doing. Things you are already doing are largely yesterday’s news, and their impact on future growth tends to wane over time. Implementing completely new innovations can help your business fight gravity.
At OpenTable, one of the most highly leveraged examples of an innovation to optimize our core business was developing a rigorous methodology to pursue and assess potential site improvements. A while after I became CEO of OpenTable, my predecessor Thomas Layton (who did a spectacular job positioning OpenTable for long-term success) sent me a fascinating video of a guy named Ron Kohavi talking about Amazon’s approach to something he called data-driven product development. Here’s an old link to one of his presentations (FYI, it only works sporadically): http://videolectures.net/kdd07_kohavi_pctce/.
The video details how Amazon rigorously deployed A/B testing to optimize website efficiency. Kohavi starts the presentation by showing a number of different executions of the same feature that they had tested over time, and he asks viewers to vote on which they thought had performed better. The results suggest that the folks in the audience—all website geeks—were not able to consistently pick the winning execution. That was mildly surprising. But what was astonishing was the delta between the results driven by different executions of the same feature: what often appeared to be subtle changes could drive huge improvements in performance!
In the video, Kohavi also talks about the impact of the “HiPPO” (Highest Paid Person’s Opinion) in the product development process. Not surprisingly, most HiPPOs believe that they know intuitively what will work best (spoken by the former HiPPO at OpenTable). But pretty much none of us have the product instincts of a Steve Jobs, and Kohavi makes a very compelling case for letting the data and not the HiPPO make the decision.
So we resolved to test data-driven product development as one of OpenTable’s potential core business optimization initiatives. We tasked a talented product manager, Julie Hall, to lead the effort and we procured tools to inform the effort. For the art side of the effort, we found the low-cost and highly efficacious usertesting.com service that enabled us to get qualitative user feedback literally overnight to inform what we planned to test. And for the science side, we bought the overpriced but also highly efficacious “Test & Target” system from Omniture (developed by Offermatica, now owned by Adobe) that enabled robust quantitative measurement of a number of simultaneous A/B tests.
These two tools worked together marvelously. One time, we literally stumbled upon a big “improvement opportunity” (a.k.a. a nasty usability problem on the site). We used usertesting.com to task a handful of unregistered users with making a reservation. But we watched in horror as a significant minority of the test users got trapped in our “Sign-In” functionality and couldn’t complete their online reservation. In the real world, they would probably get pissed off and simply pick up the phone (OpenTable’s biggest competitor for consumers) and call the restaurant, a disaster for our user acquisition, brand affinity and OpenTable economics.
Here is the offending page:
The problem we encountered was that non-registered users would set the radio button that said “I am a new OpenTable customer” and then fill in their email address and select a password….which sat under the “I am an OpenTable member” section. This combination caused the page to return an error message.
In response, we developed alternative treatments of our Sign-In functionality and tested them via Test & Target. The winning executions presented non-cookied users with a form tailored directly to new users, with clear visibility of a link for existing members to “Sign In”:
After the user hit the “Complete Free Registration” button, we then presented them with a popup that prompted them to register to become an OpenTable member:
These simple changes boosted our reservation success rate by 10% over the prior implementation. And a 10% improvement in the revenue stream that comprised well over half of the company’s total business due to one simple change was a monster win for the business.
Over time, the data-driven product development methodology at OpenTable matured into a highly disciplined testing regimen. Hundreds of tests have been run in the past few years. Not all were homeruns like the change above, but lots of singles and doubles supplemented the occasional home run to have a highly material impact on the business. I can’t recommend a rigorous data-driven product development process enough to managers of website businesses—it’s extremely low-hanging fruit in the pursuit of growth.
Other examples of core business optimization initiatives that helped OpenTable boost growth included:
1. Developing a new version of our enterprise software used by restaurants that improved search-to-reservation conversion by having the software better mimic the decision-making process of the person at the host stand
2. Redesigning key pages of the site to improve their efficiency, such as a complete overhaul of our search results pages (tested thoroughly through A/B testing)
3. Focusing dedicated resources on optimizing the percentage of OpenTable restaurant customers who had “make an online reservation” links on their own websites, as well as improving the visibility of those links
4. Applying concerted product and engineering efforts to boost our ranking in search engine results
5. Hiring a lot more sales people to accelerate the acquisition of restaurants using OpenTable (not product, but highly effective)
The key takeaway here is that all of the above were new, concrete initiatives that were not yet part of the company’s arsenal. Their successful deployment helped fight off the impact of gravity and led to accelerating growth for the company.
