What is SMACK (Spark, Mesos, Akka, and Kafka)?

This blog introduces the convergence of complementary technologies – Spark, Mesos, Akka, Cassandra and Kafka (SMACK) stack. And we will see how Apache Kafka can help us to get data under control and what is it role in our data pipeline, how Spark & Akka help us to process the data, and how Cassandra to store data. Also we will look what is Mesos a cluster manager.

Apache Spark

Apache Spark is a powerful open-source processing engine built around speed, ease of use, and sophisticated analytics. It was originally developed at UC Berkeley in 2009.

Apache Spark

Speed, Ease of use, and a unified engine are three core benefits of Apache Spark. Keep Reading…

Apache Mesos: It is a distributed systems kernel and Mesos is built using the same principles as the Linux kernel, only at a different level of abstraction. The Mesos kernel runs on every machine and provides applications (e.g., Hadoop, Spark, Kafka, Elastic Search) with APIs for resource management and scheduling across entire datacenter and cloud environments.

Apache Mesos

It has rich features like,

  • Scalability to 10,000s of nodes
  • Fault-tolerant replicated master and slaves using ZooKeeper
  • Support for Docker containers
  • Native isolation between tasks with Linux Containers
  • Multi-resource scheduling (memory, CPU, disk, and ports)
  • Java, Python and C++ APIs for developing new parallel application
  • Web UI for viewing cluster state. Keep Reading…


Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM. Akka was designed to enable developers to easily build reactive applications using a high level of abstraction. It does so in a very natural and simple way, without having to deal with low-level concepts like thread pools, mutexes, and deadlocks.

It does so by leveraging the Actor Model of concurrency and fault-tolerance. This is a powerful model that allows the behavior and state of the application to be encapsulated and modeled as an actor. The key principle behind an actor is that the application only interacts with it through messages and never talks with it directly. This isolation allows Akka to manage the currency of the actor.

Apache Akka

It has rich set of features like,

Simple Concurrency & Distribution (Asynchronous and Distributed by Design. High-level abstractions like Actors, Streams and Futures).

Resilient by Design (Write systems that self-heal. Remote and local supervisor hierarchies).

High Performance (50 million msg/sec on a single machine. Small memory footprint; ~2.5 million actors per GB of heap).

Elastic & Decentralized (Adaptive cluster management, load balancing, routing, partitioning and sharding).

Extensible(Use Akka Extensions to adapt Akka to fit your needs).

Apache Cassandra

It is a top-level Apache project born at Facebook and built on Amazons Dynamo and Googles BigTable, is a distributed database for managing large amounts of structured data across many commodity servers, while providing highly available service and no single point of failure.  Cassandra offers capabilities that relational databases and other NoSQL databases simply cannot match such as continuous availability, linear scale performance, operational simplicity and easy data distribution across multiple data centers and cloud availability zones. Cassandra’s architecture is responsible for its ability to scale, perform, and offer continuous uptime. Rather than using a legacy master-slave or a manual and difficult-to-maintain sharded architecture, Cassandra has a masterless “ring” design that is elegant, easy to set up, and easy to maintain.

Apache Cassandra

In Cassandra, all nodes play an identical role; there is no concept of a master node, with all nodes communicating with each other equally. Cassandra’s built-for-scale architecture means that it is capable of handling large amounts of data and thousands of concurrent users or operations per second— even across multiple data centers— as easily as it can manage much smaller amounts of data and user traffic. Cassandra’s architecture also means that, unlike other master-slave or sharded systems, it has no single point of failure and therefore is capable of offering true continuous availability and uptime — simply add new nodes to an existing cluster without having to take it down. Keep Reading…

Apache Kafka

Kafka is one of those systems that is very simple to describe at a high level, but has an incredible depth of technical detail when you dig deeper. Kafka is a distributed publish-subscribe messaging system that is designed to be fast, scalable, and durable.

Like many publish-subscribe messaging systems, Kafka maintains feeds of messages in topics. Producers write data to topics and consumers read from topics. Since Kafka is a distributed system, topics are partitioned and replicated across multiple nodes.

Messages are simply byte arrays and the developers can use them to store any object in any format – with String, JSON, and Avro the most common. It is possible to attach a key to each message, in which case the producer guarantees that all messages with the same key will arrive to the same partition. When consuming from a topic, it is possible to configure a consumer group with multiple consumers. Each consumer in a consumer group will read messages from a unique subset of partitions in each topic they subscribe to, so each message is delivered to one consumer in the group, and all messages with the same key arrive at the same consumer.

Zoo Keeper

What makes Kafka unique is that Kafka treats each topic partition as a log (an ordered set of messages). Each message in a partition is assigned a unique offset. Kafka does not attempt to track which messages were read by each consumer and only retain unread messages; rather, Kafka retains all messages for a set amount of time, and consumers are responsible to track their location in each log. Consequently, Kafka can support a large number of consumers and retain large amounts of data with very little overhead.

Reference – Big Data Analytics Communities, and DataStax.com.

This article originally appeared here. Republished with permission. Submit your copyright complaints here.

  1. Miguelvulky 7 months ago

    Natural Stress Solutions CBD Capsules Daytime (Standard): https://arill.us/aturaltressolutions38610

  2. RobertDrado 7 months ago

    $200 for 10 mins “work?”: http://to.ht/investcrypto90016376

  3. WilliamMeave 7 months ago

    Bezahlte Umfragen: Verdienen Sie € 3.000 oder mehr pro Woche: http://goto.iamaws.com/cryptoinvestbitcoin40936

  4. Miguelvulky 6 months ago

    The Power of Senuke TNG Linkbuilding software: http://www.abcagency.se/bestseotools66199

  5. Miguelvulky 6 months ago

    The Top 5 Best Cryptocurrencies 2019: http://corta.co/15000investbinarycrypto51859

  6. WilliamMeave 6 months ago

    Als u in 2011 $ 1.000 in bitcoin hebt geГЇnvesteerd, heeft u nu $ 4 miljoen: http://goto.iamaws.com/investcrypto25732

  7. JavierDed 5 months ago

    If you invested $1,000 in bitcoin in 2011, now you have $4 million: http://blogs.rrs.co.uk/revella/ct.ashx?url=https%3A%2F%2Fvk.cc%2F9iSaPJ

  8. This info is invaluable. When can I find out more?

  9. lk21 5 months ago

    Quality posts is the secret to interest the viewers to visit the site,
    that’s what this web page is providing.

  10. Orval 5 months ago

    Hi there, its pleasant article regarding media print,
    we alll understand media is a enormouus source of information.

  11. juridiska tjänster 4 months ago

    I was able to find good information from your articles.

  12. Nice post 🙂

  13. Touche. Sound arguments. Keep up the good effort.

  14. Watch Jav Free HD 4 months ago

    What a stuff of un-ambiguity and preserveness of precious knowledge regarding unexpected feelings.

  15. jobba 4 months ago

    Ridiculous quest there. What happened after? Take care!

  16. 島根県のバイオリン個人レッスンの活用こつとは。記載をつげる。島根県のバイオリン個人レッスンの予想外のことな探しだすとは。お役立ちホームページです。

  17. Wonderful site. A lot of helpful info here.
    I am sending it to some friends ans also sharing in delicious.
    And certainly, thanks on your sweat!

  18. When I initially commented I clicked the “Notify me when new comments are added” checkbox and now
    each time a comment is added I get several emails with the
    same comment. Is there any way you can remove me from that service?
    Thanks a lot!

  19. JavierDed 4 months ago

    Wie man in bitcoins $ 5000 investiert – erzielt eine Rendite von bis zu 2000%: http://rih.co/bestinvest80289

  20. 広島県のアルトサックススクールの名人が教える次第。お役立ちサイトです。広島県のアルトサックススクールを手記します。シーツです。

  21. drones for kids 4 months ago

    You made some really good points there. I checked on the internet
    for additional information about the issue and found most people will go along with your views
    on this website.

  22. 急ぎのキャッシングが必要な時には、専門の融資会社を利用することが大切です。実際に活用してみると、便利な融資を受けることができます。金融機関を選ぶ時は、よく調べて借り入れをすることが重要です。

  23. 税理士紹介は、仕事をするときにはかなり重要になってきます。適当に税理士を決めてしまい商売が傾いてしまうことが多いのです。こういった問題を避けるためには、質の良いコンサルタント会社を利用するべきです。

  24. gamefly free trial 4 months ago

    I do consider all of the ideas you’ve offered to your post.
    They’re really convincing and will certainly work. Still, the posts are very quick for novices.

    Could you please prolong them a little from subsequent time?
    Thank you for the post.

  25. ベビーチェアの及ぶはこちら。お役立ち場所です。ベビーチェアでマイナスしたくないよね。ころをおく。

  26. 宮崎県で初心者のマンション投資の目からうろこ跡形。コミカルサイトを前進。宮崎県で初心者のマンション投資の意外に思うな状況とは。はたしてをいいすてる。

  27. プロミス即日 3 months ago


  28. 秋田県の不動車買取を深く知りたい。慎重にプレーをする人口をひらく。秋田県の不動車買取の似付かわしいのところは?手練もうなるサイトを目差す。

  29. ClarkDup 3 months ago

    Only for Australians. Invest $ 5,000 and get from $ 15,000 per week: http://postwealthgigfli.tk/qddf

  30. EdwinUnori 3 months ago

    Bitcoin Investment Deutschland: http://xurl.es/Request

  31. Thanks for this… I’ve been googling for this
    info for some time now but haven’t been able to get anything useful.

  32. ClarkDup 3 months ago

    PhenoPen CBD Vape Pen: http://jnl.io/qwe29939

  33. I couldn’t resist commenting. Well written!

  34. roofing leads 3 months ago

    I’d like to be a member of group where I can get advice
    from other experienced vets like you. If you
    have any recommendations, please let me know. Cheers.

  35. WayneItept 3 months ago

    Trouvez-vous une fille pour la nuit dans votre ville: http://xurl.es/ifwrc

  36. ClintonDam 3 months ago

    Investeer in cannabisvoorraden van $ 5.000 en ontvang $ 1.350.000 per jaar: http://v.ht/m4fAimN

  37. LonnieWhido 3 months ago

    Miten investoida kannabis-teollisuuteen: http://v.ht/v7GJ57

  38. ClintonDam 3 months ago


  39. roofing contractor 3 months ago

    Hi there, someone in my Facebook group shared
    your with all of us so I came to look it over. I’m definitely loving the info, thank you.

  40. MatthewLox 3 months ago

    Find yourself a girl for the night in your city: https://hec.su/jurR

  41. MatthewLox 3 months ago

    Find yourself a girl for the night in your city: https://hec.su/jurR

  42. camera app 3 months ago

    It’s actually a nice and helpful piece of info. I’m glad that you just shared this helpful info with us.

    Please stay us informed like this. Thank you for sharing.

  43. 新潟県の合鍵屋さんを心地よいして~を斡旋するしたい。才気調合薬します。新潟県の合鍵屋さんを心底知りたい。ねじこむサイトです。

  44. Isso mesmo, as vezes é necessário expor muito com termo simples de distinguir, como vi aqui nesse escrito.

    Prosseguirei ler outro item desse weblog.

  45. Você escreve e também pensa parecido comigo. Eu palato bastante de falar o que penso e também
    ajudar alguém se eu souber muito do asunto.

  46. Michaelnam 2 months ago

    Заберите свои 104845 честно заработанных рублей: http://merky.de/prrctl?eReIESmGx5X8

  47. Michaelnam 2 months ago

    Заберите свои 104845 честно заработанных рублей: http://merky.de/prrctl?eReIESmGx5X8

  48. Tyronevoinc 2 months ago

    Заберите свои 109429 рублей: https://hideuri.com/qPjV7W?NJoQkGpYUsB8aE

  49. WayneItept 2 months ago

    Get $1500 – $6000 per DAY: http://v.ht/tiGHYW?&hycpe=G7TGrYy8qbzSW

  50. WayneItept 2 months ago

    Get $1500 – $6000 per DAY: http://v.ht/tiGHYW?&hycpe=G7TGrYy8qbzSW

  51. Managed Services 2 months ago

    This is some great things about Managed Services and Managed Network Services https://itexamall.com

  52. garotas de programa 2 months ago

    Olha só, como esse diário virtual seu está rico em
    teor de qualidade. Gostei bastante do que vi por cá e em breve volto para
    ver novidade.

  53. บอลไทย 2 months ago

    Thanks for finally writing about >What is SMACK (Spark, Mesos, Akka, and Kafka)?
    <Loved it!

  54. painting contractor 2 months ago

    Are you on Instagram business account? Can I follow you?

    Hey, can you tell which internet service you are utilizing?

    What is the price? Do you recommend them?

  55. Eu cuido de blog e também sei como é difícil fazer uma postagem
    boa e com palavras que ficam fácil de escutar. Gostei da sua forma de redigir
    e também é correto que vou regressar cá.

  56. torrent-filmi 2 months ago

    If you would like to grow your familiarity only keep visiting this site and be
    updated with the newest news update posted

  57. uploadedtrend 2 months ago

    I really like your blog.. very nice colors & theme. Did you create this website yourself or did you
    hire someone to do it for you? Plz reply as I’m looking to construct my own blog and would like to know where u got this from.
    many thanks

  58. 먹튀검증업체 2 months ago

    When I originally commented I clicked the “Notify me when new comments are added” checkbox and now each
    time a comment is added I get several emails with the same comment.
    Is there any way you can remove people from that
    service? Cheers!

  59. hedp acid 2 months ago

    Hurrah, that’s what I was seeking for, what a material! existing here at this blog, thanks admin of this web site.

  60. ClintonDam 2 months ago

    What’s the easiest way to earn $30000 a month: http://monnlinsipoc.tk/youf

  61. ClarkDup 2 months ago

    Get $1500 – $6000 per DAY: http://merky.de/42xvid?&cpseb=SczOkaR6

  62. Você usou palavras tão simples de ler, escrito aqui. Eu gostei mais o
    blog desse jeito. Desta forma as pessoas compreendem rápido o que o autor deseja dizer.

  63. ClintonDam 2 months ago

    Get $1500 – $6000 per DAY: http://chaifuldedy.tk/vacs6

  64. If some one desires expert view about blogging after that i advise
    him/her to pay a visit this web site, Keep up the fastidious work.

  65. Garota de Programa 2 months ago

    Paladar bastante do seu blog e também também leio muito artigo escrito por
    você. Sou leitor de toda semana e também sempre que lembro volto aqui.

  66. Jamesniz 2 months ago
  67. Jamesniz 2 months ago

    The Best Cryptocurrency to Invest in 2019: http://emgreenofov.tk/erxr?uyRQKO

  68. ClarkDup 1 month ago

    LAZY way for $200 in 20 mins: http://siolecsoundbrin.tk/3lim2?rCu91fo

  69. ClintonDam 1 month ago

    Earnings on the Bitcoin course from $ 2500 per day: http://roundtimvoco.tk/jxyy6?&druil=16HXFPS6V9sjh

  70. LonnieWhido 1 month ago

    Earnings on the Bitcoin course from $ 2500 per day: http://senmargsandtep.tk/32lxt?&baiki=9Am2i

  71. Hi there to every , since I am truly keen of reading this blog’s post to
    be updated daily. It consists of fastidious information.

  72. ClintonDam 1 month ago

    Find yourself a girl for the night in your city: http://davertelshy.tk/kq1t?&ilgss=EYJOR61

  73. Jamesniz 1 month ago

    Make Money 10000$ Per Day With Bitcoin: http://criccaltenest.tk/l405n?WpE9AKIXV8q

  74. Ninfas Gyn 1 month ago

    Enquanto vejo um ponto que paladar de forma, leio o artigo todo
    e também quero mas. Espero que você continua a
    atualizar seu blogue desta forma. possui muito plumitivo bom e você é um deles.
    Agradeço a web por me dar a possibilidade de
    conhecer seu site e ser capaz revir.

  75. This is some great things about Managed Services and Managed Network Services http://craigtroy.com/__media__/js/netsoltrademark.php?d=itexamall.com

  76. CharlesSnili 1 month ago

    Earnings on the Bitcoin course from $ 2500 per day: http://outcondori.ga/gnmr?JZMQ9rB0Rx

  77. Raymondror 1 month ago

    Invest $ 5000 and get $ 55000 every month: http://frutmidossu.cf/9w2bg?SZRoquVRRC0

  78. Carlossow 4 weeks ago

    Find yourself a girl for the night in your city: http://murringdebo.cf/azsv?&rssxs=SNffWIXgG

  79. CharlesSnili 3 weeks ago

    Invest in Bitcoin and earn from $ 3000 per day: http://v.ht/udRCK7n?&ggozk=vhTCQ

  80. Raymondror 3 weeks ago

    Make Money 10000$ Per Day With Bitcoin: http://v.ht/acKaXy?2k6qr

  81. EdwinUnori 3 weeks ago

    What’s the easiest way to earn $30000 a month: http://go-4.net/f5RZ?skTni

  82. Carlossow 3 weeks ago

    Make Money 10000$ Per Day With Bitcoin: http://v.ht/buE41?9DgHvf7

  83. WilliamMeave 2 weeks ago

    Top 5 des sites de rencontre dГ©contractГ©s ZA 2019: https://hec.su/ld5W?iB1P4BkqNmw

  84. minecraft games 2 weeks ago

    It’s genuinely very complicated in this active life to listen news on TV, so I
    just use world wide web for that reason, and obtain the newest news.

  85. Piece of writing writing is also a fun, if you know afterward you can write or else it is difficult to write.

  86. Carlossow 1 week ago

    $15,000 a month (30mins “work” lol): http://gmy.su/:5r8Q?T4dpyq

  87. rättshjälp 1 week ago

    There is certainly a lot to learn about this subject.
    I really like all the points you made.

  88. WilliamMeave 1 week ago

    Forex + cryptocurrency = $ 9000 per week: http://go-4.net/f9xo?z5kOE0JNclCs5

  89. EdwinUnori 1 week ago

    Make Money 10000$ Per Day With Bitcoin: https://hec.su/lYJU?TtLlF0YJF

  90. GabrielFuevy 5 days ago

    Find yourself a girl for the night in your city: http://v.ht/P16Jk?hpkPVBRI

  91. stemple, 4 days ago

    Awesome article.

  92. RonaldVed 12 hours ago

    Make Money 10000$ Per Day With Bitcoin: http://gmy.su/:opYT?&ueswx=2PqAaY

  93. Joshuastazy 10 hours ago

    Earn Free Bitcoin 0.2 BTC Per day: https://hec.su/nsCs?&jkfbd=pV7ddxH7n1

  94. Antoniosix 6 hours ago

    Get $1000 – $6000 A Day: https://hec.su/l2su?&xglno=qIlga6ZtBiK

  95. Antoniosix 2 hours ago

    Become a bitcoin millionaire. Get from $ 2500 per day: http://gmy.su/:zLXT?&eljzr=IsWk7eAmqeaFfO

Leave a Comment

Your email address will not be published.

You may also like

Pin It on Pinterest