You may or may not have heard of it, but Big Data has been the phrase for internet and marketing people for some time now. Like most technologies promising to revolutionise the world as we know it, as a concept it’s both simple, and yet complex and nebulous all at once.
It’s simple because at its core it simply means processing vast amounts of data collected every fraction of a second, and turning it into valuable insight, most usually about you and I. Ever wondered why once you’ve visited a site on holidays in Tuscany, you’re then bombarded with Tuscany ads wherever you go afterwards? That’s big data in its tiniest form.
It gets much more sophisticated. eBay.com uses two data warehouses at 7.5 petabytes and 40PB as well as a 40PB Hadoop cluster for search, consumer recommendations and merchandising. Walmart handles more than 1 million customer transactions every hour, which are imported into databases estimated to contain more than 2.5 petabytes (2,560 terabytes) of data, the equivalent of 167 times the information contained in all the books in the US Library of Congress. I could go on … (thanks to Wikipedia).
For consumers it means you are constantly being profiled, your every choice online and increasingly offline; being absorbed and analysed until the business that is doing the analysing gets to really know you. Google, for example, knows where I live, and when I ask directions, it starts and ends my journey not from the random ‘Palm Jumeirah’ location I enter, but from my exact building. It could probably follow me upstairs.