Big data analytics, Moneyball, MIT and corporate treasure

While sport pages remain obsessed with Moneyball and statistical analysis, 2012 seems to be the year that the corporate world really came alive to Big Data Analytics.

In our increasingly networked digital world, ‘Big data’ refers to data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process within a reasonable time.

Hybrid data analytics specialists Saffron Technology blogged this week on how corporates can unearth treasure by combining their usual structured data with the invaluable insights available in unstructured data… e.g. from social media, call logs, customer records, meeting notes, maintenance reports, news articles etc.

Diverse Data Analytics finds corporate treasure

Through 2015, organizations integrating high-value, diverse, new information types and sources into a coherent information management infrastructure will outperform their industry peers financially by more than 20%. Rita Sallam, Gartner April 2012.

Gartner have identified this as a new sort of business intelligence – the analysis of hybrid data.

The idea is not exactly new – particularly in the national security space where the concept of Total Information awareness has been around for a decade.

Football experts frequently write that statistics cannot properly explain the sport because it is too fast, too rich, too complex.

Yet in business intelligence, national intelligence and risk intelligence we are already moving away from the idea of simple rule based analysis towards new tools which can combine structured and unstructured data, and statistics and semantics.

Similarly we want to be able to handle real time data – not take 3 weeks to analyse what the opportunities and risks are.

Following Zach Slaton’s coverage of the MIT Sloan Sports Analytics Conference it appeared that football is not really using this big fuzzy data at all.

Indeed the sport seems hung up on the idea transplanting the idea of MoneyBall to Football.

Yet to me Moneyball is not a lesson in statistics but a lesson in Business Intelligence.

He who has the better data analytics will profit.

Could Big Data analytics be Moneyball 2.0?

Here’s a video we made with then VP of Research at SAP Labs, Dr Paul Hofmann, on how hybrid data analytics trumps traditional statistics.

 

wistiaEmbed = Wistia.embed(“c0dcbabe22”, {
version: “v1”,
videoWidth: 600,
videoHeight: 338,
volumeControl: true,
playerColor: “cccccc”
});