Time is money, so I went and bought a Rolex
—Wiz Khalifa, Phone Numbers
https://www.youtube.com/watch?v=l85JNOGorwc
When I ran Opsware, we had the non-linear quarter problem also known affectionately as the hockey stick. The hockey stick refers to the shape of the revenue graph over the course of a quarter. Our hockey stick was so bad that one quarter, we booked 90% of our new bookings on the last day of the quarter. Sales patterns like this make it difficult to plan the business and are particularly harrowing when you are, as we were, a public company.
Naturally, I was determined to straighten out the hockey stick and bring some sanity to the business. I designed an incentive for sales people to close deals in the first two months of the quarter by issuing bonuses for deals in those months. As a result, the next quarter became slightly more linear, and slightly smaller than anticipated – deals just moved from the 3rd month to the first two of the following quarter.
When I ran a large engineering group at Netscape, I measured one of our engineering products on schedule, quality and features. The team shipped a product with all the required features, on time and with very few bugs. Unfortunately, the product was mediocre, because none of the features were that great.
When I was at HP, we ran all the businesses by the numbers with extremely strict revenue and margin targets. Some divisions made their numbers, but did so by underfunding R&D. They dramatically weakened their long-term competitive position and set themselves up for future disaster.
In all three cases, managers got what we asked for, but not what we wanted. How does this happen? Let’s take a look.
Flattening out the hockey stick—the wrong goal
In retrospect, I should never have asked the team to flatten the quarters. If that is what I wanted, I had to be willing to—at least temporarily—accept smaller quarters. We had a fixed number of sales people who were maximizing the size of each quarter. In order to deliver linear quarters, they had to modify their behavior and adjust their priorities. Unfortunately, I liked the old priority of maximizing revenue better.
Given the situation, I was actually pretty lucky. Sun Tzu in his classic work The Art of War warns that giving the team a task that it cannot possibly perform is called crippling the army. In my case, I did not cripple the team, but I screwed up my priorities. The right thing to do would have been to make the hard decision up front: what was more important a) maximizing each quarter or b) increasing predictability. The instruction only made sense if the answer was (b).
Over Focusing on the Numbers
In the second example, I managed the team to a set of numbers that did not fully capture what I wanted. I wanted a great product that customers would love with high quality and on time—in that order.
Unfortunately, the metrics that I set did not capture those priorities. At a basic level, metrics are incentives. By measuring quality, features, and schedule and discussing them every staff meeting, my people intensely focused on those metrics to the exclusion of other goals. The metrics did not describe the real goals and I distracted the team as a result.
Interestingly, I see this same problem play out in many consumer Internet startups. I often see teams who maniacally focus on their metrics around customer acquisition and retention. This usually works well for customer acquisition, but not so well for retention. Why?
For many products, metrics often describe the customer acquisition goal in enough detail to provide sufficient management guidance. In contrast, the metrics for customer retention do not provide enough color to be a complete management tool. As a result, many young companies over focus their teams on their retention metrics, but do not spend enough time getting into intense depth on the actual user experience. This generally results in a frantic numbers chase that does not end in a great product.
It’s important to supplement a great product vision with a strong discipline around the metrics, but if you substitute metrics for product vision, you will not get what you want.
Managing strictly by numbers is like painting by numbers
At HP, we were highly focused on results. As with the situation at Netscape, some things that you want to encourage will be quantifiable and some will not. If you report on the quantitative goals and ignore the qualitative ones, you won’t get the qualitative goals, which may be the most important goals. Management purely by numbers is sort of like painting by numbers—it’s strictly for amateurs.
At HP, the company wanted high earnings now and in the future. By focusing entirely on the numbers, HP got them now by sacrificing the future.
Note that there were many numbers as well as more qualitative goals that would have helped:
Was our competitive win rate increasing or declining?
Was customer satisfaction rising or falling?
What did our own engineers think of the products?
By managing the organization as though it were a black box, some divisions at HP optimized the present at the expense of their downstream competitiveness. The company rewarded managers for achieving short-term objectives in a manner that was bad for the company. It would have been better to take into account the white box qualitative and quantitative issues and reward only those managers that hit their numbers while readying the company for a strong future.
Closing thought
In the examples above, it is easy to see that there are many ways to be misinterpreted. To get things right, you must recognize that anything that you measure automatically creates a set of employee behaviors. Once you determine the result you want, you need to test the description of the result against the employee behaviors that the description will likely create. Otherwise, the side-effect behaviors may be worse than the original situation.
