Code submitted this week for inclusion in the Hadoop stack will help speed the spread of the distributed big-data platform, according to Hortonworks co-founder Arun Murthy.
The submission of the Slider framework to the Apache Software Foundation Incubator will result in existing applications, such as NoSQL databases, running unmodified on Hadoop and its YARN resource-management layer.
“It’s looking ahead to the future of Hadoop and YARN. This is a really important step forward because it allows us to expand the spectrum of applications and use cases that you can actually service with Hadoop and YARN,” Murthy said.
“A NoSQL database is an example; an analytic service is an example. We expect these things to use Slider to bridge the gap between a silo that they live in today to run natively in Hadoop.”
Work on Apache Slider has already been going on for the past eight or nine months and the framework is expected to be available to the wider market by the second half of this year.
YARN was released last October in Hadoop 2.0 and separates MapReduce’s resource management and processing components, allowing other processing algorithms to be used.
Murthy, who has worked on Hadoop since day one in 2006, described Slider as broadening Hadoop beyond just processing data.