Last week’s excellent Times article on the high salaries being paid to statisticians is just one sign of the market value of number crunching. As I wrote in the afterword to the paperback version of Super Crunchers:
In 2007, there were three major acquisitions of “business intelligence” firms: in April, Oracle purchased Hyperion Solutions Corp. for about $3.3 billion; in October, SAP purchase Business Objects for $4.8 billion; and, in November, IBM announced its $4.9 billion purchase of Cognos. These multi-billion dollar acquisitions of firms whose sole product is number crunching is powerful evidence that data-driven decision making has market value.
To this list of billion-dollar purchases we can now add IBM’s acquisition of stat software provider SPSS for a mere $1.2 billion.
But in some ways, an even more eye-opening market test is for a much smaller amount. Microsoft shelled out $115 million to buy Farecast.com, a company that crunches numbers to tell you whether you should buy an airline ticket now or wait until closer to when your flight leaves. I’ve long been a fan of the Farecast predictions (which wonderfully include estimates of their own precision):
It was a wonderful decision of Microsoft to use Farecast predictions as a central element of Bing Travel. Still, $115 million surprises me. Farecast has terabytes of data but in some sense it is just running something close to a very simple multivariate regression. It goes to show that thinking up the right regression to run can be worth millions.
and 
. You can also watch it on 
I agree that the number of staticians and their salaries are growing, at leat by what I have read in your blog and in Tim Harford’s blog. But I differ in a way… Those databases have other purposes too, such as run the regular business operations. Their main goal is not to crunch numbers and give out statistics, such as the example of Farecast.com.
I think the main idea is correct, but the examples are not.
From a quick Google search I found that Farecast had raised $8.5 million in venture capital funding through 2005. It is likely that further amounts were invested. So the returns are not that out of line with any other kind of startup.
Even a successful restaurant (valued at say 5x earnings) could hope for a 10x return over initial capital investment. The work of a few smart people make it possible.
Given the scalability of Farecast I think Microsoft may have gotten Farecast at a large discount.
http://www.broadcastthoughts.net/2009/08/expected-value-of-information.html
Maybe Ananova.com could have made more money if they went with ANOVA?
Nurses are worth much more to society. The problem is just that no one wants to be a statistician so the saleries are high to get someone do it.
I can’t seem to figure out how to make it display prices in Canadian dollars.
I tend to argee with Miguel @ 1.
I’m a BusinessObjects developer, but I would hardly consider myself a statistician or a number cruncher, per se.; instead I tend to view myself more as an IT professional. These business intelligence tools are used to aggregate, analyze or present data that are not necessarily easy to evaluate directly from the database. You essentially get reports on the specific information you already have.
Farecast, however, is a more traditional statistical model that is used to determine the probability of a certain event occuring (ie. a fare increasing) based on a deep review of prior results.
Guess I just differ on your definitions. But I would bet you that many other in the business intelligence field agree with me.
Andrew,
That is exactly why the business intelligence field, in general, is not intelligent. It is simply reporting tools. The missing part of most BI is the intelligent analysis that should be occurring one step after what you are describing as your job. (I work in BI as well and think BI really is just IT at this point, when the promise is for it to be so much more).