Select Page

Apache Kafka as a distributed streaming platform

Apache Kafka and BigData

Apache Kafka is a distributed, heavily scaled communication queue which is capable of processing huge amounts of messages and simultaneous client connections. Kafka makes it possible to accept the challenges posed by Big Data, where broker technologies, based on JMS or AMQP standards, have failed. The technology lends itself perfectly to be used on the dynamically changing service market. It addresses huge volumes of information generated by mobile environments or IoT (Internet of Things).

Apache Kafka, BigData

The solution implements the publish-subscribe communication model with particular emphasis put on maximum throughput and minimum delay in delivering messages. Kafka guarantees messages to be delivered to the subscriber’s system due to a message log mechanism. Permanent storage of data in Kafka nodes allows them to be processed also in “batch” mode, analogically to ETL tools.
Kafka has made its way to many organizations where it is necessary to process streams of terabytes of client information with maximum reliability. Examples include: Spotify, Uber or PayPal.

Some potential technical applications of Kafka include:

Gathering data for the purposes of real-time analysis (-> Apache Spark)
Data stream processing (-> Apache Storm)
Assuming the role of Message Oriented Middleware (-> MOM)
Aggregating application logs

BlueSoft has extensive experience in Apache Kafka, dating back as early as the incubation stage of this technology. We have successfully implemented Kafka-based production solutions at a number of banking sector clients.

BlueSoft has successfully implemented many projects in this area. We will happily present our portfolio directly as well as answer more questions about technology itself and benefits to be brought by its implementation.

Technology website: Apache Kafka

See other technologies, which we use in this area

Integration technologies