According to market intelligence company Transparency Market Research the global Hadoop market is growing at almost 55 per cent a year and is expected to be worth $20.9 bn in 2018.
No-one professes himself more surprised by this meteoric rise than Hadoop’s co-creator Doug Cutting. After all, the platform that is emerging as the de facto means of pulling together, storing and processing diverse types of data started life in 2005 as a side project at Yahoo!, a way to enable two open-source search projects that Cutting was working on at home to run across lots of machines.
“I wasn’t expecting it,” he told Computing. “In a way it was a case of being in the right place at the right time – although of course you have to be able to take advantage of being in the right place at the right time.”
Another surprise, at least in the early days, was that none of the major software vendors – the Oracles, IBMs and Microsofts – picked up on the growing demand for managing and processing data on an ever-increasing scale by developing their own distributed platforms. This he puts down to the strength of the open source model, or rather the weakness of the proprietary one.