One of the knocks against Apache Hadoop has been that it was built as a batch processing system and hence is no good for real-time data analytics. Hadoop 2.0 promises a lot of improvement in this area, however. Its YARN resource management layer, for instance, offers better support for stream-processing platforms such as Storm, which recently became an Apache open-source project. Hadoop’s shortcomings have also created an opportunity for stream-processing technology providers, which have been busy partnering up with Hadoop vendors.
A growing number of companies are entering the real-time, stream-processing space, including Vitria, a 20-year-old Silicon Valley firm. According to Vitria co-founder and chief technical officer Dr. Dale Skeen, the market for continuous, real-time analysis is quickly evolving from “visionary” early adopters to more mainstream use.