Yes, the phenomenon of so-called “big data” has clearly made it to the buzzword level. Sometimes truly important technology trends, in fact, do become buzzwords.
To begin to qualify this, we can look at the projected numbers from the analysts—which, of course, are themselves computed by using big data sets. Big data storage (and the analytics processing that uses all that raw data) is a burgeoning market that IDC, for one, has forecast to surge at a 40 percent compound annual growth rate, from $3.2 billion in 2010 to $17 billion in 2015. The hockey-stick curve will certainly continue to zoom up-and-to-the-right after that.
Microsoft recently conducted a big data survey of its enterprise customers which found that almost two-thirds (62 percent) of respondents said they currently store at least 100 terabytes of data. Take a look at how much data that you, as a single consumer, store; it’s a safe bet that it is way more than you might have imagined. Then think of how much data enterprises have to manage on a daily basis or even an annual basis. Big data presents a formidable new frontier in which data sets can grow so large that they become awkward to work with using traditional database management tools. The need for new tools, frameworks, hardware, software and services to handle this emerging issue represents a huge market opportunity.