Shortly after joining Andreessen Horowitz, I started hearing excitement around one of our seed portfolio companies, SigOpt. Much of it was coming from our Executive Briefing Center, a resource we have that helps our portfolio companies gain relationships with larger businesses. While many startups tend to focus on one vertical in their early days, SigOpt — as I understood it — already had a customer base that spanned dozens of verticals and use cases that included everything from high frequency trading, to fraud detection, to oil and gas exploration, and even to the beer brewing industry!
Deeply interested in learning more, I set up a meeting with the founders Scott Clark, a PhD from Cornell who was considered a world expert in mathematical optimization, and Patrick Hayes, a hardcore distributed systems and machine learning developer with experience working on production systems in some of the largest operations in the world including Foursquare and Facebook. I was curious to understand what it was they were doing that generated so much excitement across such a broad space.
It didn’t take long to get it. The way that Scott explained it was simple: Many business endeavors can be framed as an optimization problem. That is, as a model or a process that takes a set of inputs and produces some output with business value. And for complicated problems there is no simple correlation between the inputs and the outputs, which tends to be all of the interesting ones.
To be a bit more concrete, many of the most salient problems we tackle in business today require tuning parameters to find a desired outcome. This ranges from running machine learning models (fraud, security, customer retention, etc.) to high frequency trading models, to risk simulations, to physical simulation for oils and gas, to tuning recipes for manufacturing and development. Generally, there is a very complicated relationship between the inputs and the output so it really isn’t possible to determine what the optimal inputs are to produce a desired output. As a result, these systems are often run by domain experts who spend the majority of their time laboriously adjusting inputs in order to find the right output. As you can imagine, this can take anywhere from days for some problems to months for others.
SigOpt provides optimization as a service. Exposed as just a few function calls to their API or web interface, the service runs a number of open and proprietary algorithms on the back end to provide results exponentially faster than traditional methods.
Further, existing methods often return suboptimal results because they return the best result of the limited space they explore, rather than the true optimal value. Therefore — and this is important — not only is SigOpt orders of magnitude faster, but it can bring models to previously unattainable results in mature industries where every incremental gain has enormous impact!
I have personal experience with this. When working at Livermore National Laboratory on simulation codes in the early 2000’s, the vast majority of computation time was spent running simulations, getting inaccurate results, tweaking the inputs and simulation parameters—and then repeating the entire process. Due to the number of tunable parameters, and their non-obvious relationship to the output, there was no simple way to know how to set them. Instead we wasted years of man and compute hours using trial and error to tune the system.
Given what an exhaustive and costly process it can be, it’s little wonder SigOpt is getting the excitement and traction they are. They get to better business results faster with fewer resources for some of the most salient problems in industry. And Scott certainly has the credibility to make these claims. Not only did he do his PhD in Cornell in exactly this space, but he’s applied it in practice. This includes founding and leading the team at Yelp that created one of the most popular open source optimization packages.
We are exceptionally thrilled to continue the partnership and deepen our investment in SigOpt. When we first were introduced to the company, they were a top-notch team at Y Combinator with a big idea. Since then, they’ve honed their system and won some big deals with big brands such as Johnson & Johnson and MillerCoors, as well as a number of major players in the banking and financial industry including Prudential. By all telltales, this is only the beginning. As Scott said, most problems can be formulated as an optimization problem, in which case, SigOpt is there to solve them.
