Big Data and Social Intersect in 2012: Implications for Business
Big Data and Social has become the latest buzzword off late, and I can personally vouch for it by the number of people (on the client side) we see nowadays who are eager to attend the meeting. Give the "definitional" confusion around Big Data; we thought we will put together our thoughts.
First, Big Data is very much unlike traditional BI tools and processes. It requires a paradigm shift in thinking about how an organization deals with data and a mindset shift of its technology team and analytics team. We sense that less than 5-10% of the traditional IT have the right mindset and skillset to embrace and exploit this shift: in most organizations it is probably a handful of people who are capable of embracing this change
Second, "good-enough" real time analytics has starting to influence, sometimes at the expense of exact accuracy. For example, unlike 5 years back, today a retailer is more likely to open to rather getting real-time "signals" on fashion trends that they could act on quickly. This is where social and big data are starting to impact the traditional orgs in a disproportionate way: leading retailers (think Amazon, and increasingly, WalMart) are starting to crunch big data in real time and embedding social signals in their merchandizing and sales process and reap disproportionate viz-a-viz more traditional retailers.
Third, Big Data is not about big volume - factors such as velocity (how fast the data is coming and how quickly one can crunch them - think of 200 million+ tweets a day), variety (text, image, video, audio), and variability are critical dimensions of big data and make the use of it more interesting. In fact, if one had just one variable, say, high volume, then probably the existing BI tools may have sufficed. It's when several characteristics come together big data becomes very interesting. Example: when a retailer combines customer CRM data with social graph data, it may be able to identify the "key influencers" (people who are evangelists or considered experts on certain products, like a camera) among its customers and use them for creating awareness and buzz about a new product being launched.
Big Data is here, and will provide companies that can leverage it significant competitive advantages. By most estimates, an average organization today only uses 5-10% of the available data. Even the leading ones are only crunching 3-5 times data than others, yet the impacts are obvious - their EPS's are growing 2-3X their competitors.
While the term "Big data" is hyped and a lot of practitioners seem to be appearing from the woods, the companies that see beyond the hype to focus what's fundamentally changed and embrace it are going to be significantly rewarded.