What is SMACK (Spark, Mesos, Akka, and Kafka)?

This blog introduces the convergence of complementary technologies – Spark, Mesos, Akka, Cassandra and Kafka (SMACK) stack. And we will see how Apache Kafka can help us to get data under control and what is it role in our data pipeline, how Spark & Akka help us to process the data, and how Cassandra to store data. Also we will look what is Mesos a cluster manager.

Apache Spark

Apache Spark is a powerful open-source processing engine built around speed, ease of use, and sophisticated analytics. It was originally developed at UC Berkeley in 2009.

Apache Spark

Speed, Ease of use, and a unified engine are three core benefits of Apache Spark. Keep Reading…

Apache Mesos: It is a distributed systems kernel and Mesos is built using the same principles as the Linux kernel, only at a different level of abstraction. The Mesos kernel runs on every machine and provides applications (e.g., Hadoop, Spark, Kafka, Elastic Search) with APIs for resource management and scheduling across entire datacenter and cloud environments.

Apache Mesos

It has rich features like,

  • Scalability to 10,000s of nodes
  • Fault-tolerant replicated master and slaves using ZooKeeper
  • Support for Docker containers
  • Native isolation between tasks with Linux Containers
  • Multi-resource scheduling (memory, CPU, disk, and ports)
  • Java, Python and C++ APIs for developing new parallel application
  • Web UI for viewing cluster state. Keep Reading…


Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM. Akka was designed to enable developers to easily build reactive applications using a high level of abstraction. It does so in a very natural and simple way, without having to deal with low-level concepts like thread pools, mutexes, and deadlocks.

It does so by leveraging the Actor Model of concurrency and fault-tolerance. This is a powerful model that allows the behavior and state of the application to be encapsulated and modeled as an actor. The key principle behind an actor is that the application only interacts with it through messages and never talks with it directly. This isolation allows Akka to manage the currency of the actor.

Apache Akka

It has rich set of features like,

Simple Concurrency & Distribution (Asynchronous and Distributed by Design. High-level abstractions like Actors, Streams and Futures).

Resilient by Design (Write systems that self-heal. Remote and local supervisor hierarchies).

High Performance (50 million msg/sec on a single machine. Small memory footprint; ~2.5 million actors per GB of heap).

Elastic & Decentralized (Adaptive cluster management, load balancing, routing, partitioning and sharding).

Extensible(Use Akka Extensions to adapt Akka to fit your needs).

Apache Cassandra

It is a top-level Apache project born at Facebook and built on Amazons Dynamo and Googles BigTable, is a distributed database for managing large amounts of structured data across many commodity servers, while providing highly available service and no single point of failure.  Cassandra offers capabilities that relational databases and other NoSQL databases simply cannot match such as continuous availability, linear scale performance, operational simplicity and easy data distribution across multiple data centers and cloud availability zones. Cassandra’s architecture is responsible for its ability to scale, perform, and offer continuous uptime. Rather than using a legacy master-slave or a manual and difficult-to-maintain sharded architecture, Cassandra has a masterless “ring” design that is elegant, easy to set up, and easy to maintain.

Apache Cassandra

In Cassandra, all nodes play an identical role; there is no concept of a master node, with all nodes communicating with each other equally. Cassandra’s built-for-scale architecture means that it is capable of handling large amounts of data and thousands of concurrent users or operations per second— even across multiple data centers— as easily as it can manage much smaller amounts of data and user traffic. Cassandra’s architecture also means that, unlike other master-slave or sharded systems, it has no single point of failure and therefore is capable of offering true continuous availability and uptime — simply add new nodes to an existing cluster without having to take it down. Keep Reading…

Apache Kafka

Kafka is one of those systems that is very simple to describe at a high level, but has an incredible depth of technical detail when you dig deeper. Kafka is a distributed publish-subscribe messaging system that is designed to be fast, scalable, and durable.

Like many publish-subscribe messaging systems, Kafka maintains feeds of messages in topics. Producers write data to topics and consumers read from topics. Since Kafka is a distributed system, topics are partitioned and replicated across multiple nodes.

Messages are simply byte arrays and the developers can use them to store any object in any format – with String, JSON, and Avro the most common. It is possible to attach a key to each message, in which case the producer guarantees that all messages with the same key will arrive to the same partition. When consuming from a topic, it is possible to configure a consumer group with multiple consumers. Each consumer in a consumer group will read messages from a unique subset of partitions in each topic they subscribe to, so each message is delivered to one consumer in the group, and all messages with the same key arrive at the same consumer.

Zoo Keeper

What makes Kafka unique is that Kafka treats each topic partition as a log (an ordered set of messages). Each message in a partition is assigned a unique offset. Kafka does not attempt to track which messages were read by each consumer and only retain unread messages; rather, Kafka retains all messages for a set amount of time, and consumers are responsible to track their location in each log. Consequently, Kafka can support a large number of consumers and retain large amounts of data with very little overhead.

Reference – Big Data Analytics Communities, and DataStax.com.

This article originally appeared here. Republished with permission. Submit your copyright complaints here.

  1. Miguelvulky 4 months ago

    Natural Stress Solutions CBD Capsules Daytime (Standard): https://arill.us/aturaltressolutions38610

  2. RobertDrado 4 months ago

    $200 for 10 mins “work?”: http://to.ht/investcrypto90016376

  3. WilliamMeave 3 months ago

    Bezahlte Umfragen: Verdienen Sie € 3.000 oder mehr pro Woche: http://goto.iamaws.com/cryptoinvestbitcoin40936

  4. Miguelvulky 3 months ago

    The Power of Senuke TNG Linkbuilding software: http://www.abcagency.se/bestseotools66199

  5. Miguelvulky 3 months ago

    The Top 5 Best Cryptocurrencies 2019: http://corta.co/15000investbinarycrypto51859

  6. WilliamMeave 2 months ago

    Als u in 2011 $ 1.000 in bitcoin hebt geГЇnvesteerd, heeft u nu $ 4 miljoen: http://goto.iamaws.com/investcrypto25732

  7. JavierDed 2 months ago

    If you invested $1,000 in bitcoin in 2011, now you have $4 million: http://blogs.rrs.co.uk/revella/ct.ashx?url=https%3A%2F%2Fvk.cc%2F9iSaPJ

  8. This info is invaluable. When can I find out more?

  9. lk21 2 months ago

    Quality posts is the secret to interest the viewers to visit the site,
    that’s what this web page is providing.

  10. Orval 1 month ago

    Hi there, its pleasant article regarding media print,
    we alll understand media is a enormouus source of information.

  11. I was able to find good information from your articles.

  12. Nice post 🙂

  13. Touche. Sound arguments. Keep up the good effort.

  14. Watch Jav Free HD 1 month ago

    What a stuff of un-ambiguity and preserveness of precious knowledge regarding unexpected feelings.

  15. jobba 1 month ago

    Ridiculous quest there. What happened after? Take care!

  16. 島根県のバイオリン個人レッスンの活用こつとは。記載をつげる。島根県のバイオリン個人レッスンの予想外のことな探しだすとは。お役立ちホームページです。

  17. Wonderful site. A lot of helpful info here.
    I am sending it to some friends ans also sharing in delicious.
    And certainly, thanks on your sweat!

  18. When I initially commented I clicked the “Notify me when new comments are added” checkbox and now
    each time a comment is added I get several emails with the
    same comment. Is there any way you can remove me from that service?
    Thanks a lot!

  19. JavierDed 4 weeks ago

    Wie man in bitcoins $ 5000 investiert – erzielt eine Rendite von bis zu 2000%: http://rih.co/bestinvest80289

  20. 広島県のアルトサックススクールの名人が教える次第。お役立ちサイトです。広島県のアルトサックススクールを手記します。シーツです。

  21. drones for kids 2 weeks ago

    You made some really good points there. I checked on the internet
    for additional information about the issue and found most people will go along with your views
    on this website.

  22. 急ぎのキャッシングが必要な時には、専門の融資会社を利用することが大切です。実際に活用してみると、便利な融資を受けることができます。金融機関を選ぶ時は、よく調べて借り入れをすることが重要です。

  23. 税理士紹介は、仕事をするときにはかなり重要になってきます。適当に税理士を決めてしまい商売が傾いてしまうことが多いのです。こういった問題を避けるためには、質の良いコンサルタント会社を利用するべきです。

  24. I do consider all of the ideas you’ve offered to your post.
    They’re really convincing and will certainly work. Still, the posts are very quick for novices.

    Could you please prolong them a little from subsequent time?
    Thank you for the post.

  25. ベビーチェアの及ぶはこちら。お役立ち場所です。ベビーチェアでマイナスしたくないよね。ころをおく。

  26. 宮崎県で初心者のマンション投資の目からうろこ跡形。コミカルサイトを前進。宮崎県で初心者のマンション投資の意外に思うな状況とは。はたしてをいいすてる。

  27. プロミスの短時間融資は、資金繰りに困った時には、役に立つ手段です。会員は、日に日に増加しています。プロミスでは、以前申込者には、一定期間の無利息借入の機会を受けることができます。まずは短期のキャッシングでテストしてみるのがいいですね。急なお金が必要な時には、有効なサービスです。

  28. 秋田県の不動車買取を深く知りたい。慎重にプレーをする人口をひらく。秋田県の不動車買取の似付かわしいのところは?手練もうなるサイトを目差す。

  29. ClarkDup 4 days ago

    Only for Australians. Invest $ 5,000 and get from $ 15,000 per week: http://postwealthgigfli.tk/qddf

  30. EdwinUnori 2 days ago

    Bitcoin Investment Deutschland: http://xurl.es/Request

Leave a Comment

Your email address will not be published.

You may also like

Pin It on Pinterest