We are now in the era of big data 2.0, as defined by Foster Provost and Tom Fawcett in Data Science for Business. There’s growing interest in predictive analytics solutions powered by machine learning. As InsightsOne’s CEO Waqar Hasan puts it: “Predictive is the ‘killer app’ for big data.” Quite interestingly, McKinsey & Company predicted a shortage of machine learning talent in the coming years, and at the same time, we started to see services that made machine learning and predictive analytics accessible to the masses. We’re seeing more and more of these services: Apigee launched one last April, just a couple of months after buying InsightsOne.
One of the first things I learned when I took computer science classes at university was that it was our job to “put ourselves out of business.” There are things that we do by hand, and our job as computer scientists is to make programs that do the same things, then other programs that replace them and that are quicker, more reliable, require less maintenance, and so on. The same applies to data science.