Apache Kafka and BigData
The solution implements the publish-subscribe communication model with particular emphasis put on maximum throughput and minimum delay in delivering messages. Kafka guarantees messages to be delivered to the subscriber’s system due to a message log mechanism. Permanent storage of data in Kafka nodes allows them to be processed also in “batch” mode, analogically to ETL tools.
Kafka has made its way to many organizations where it is necessary to process streams of terabytes of client information with maximum reliability. Examples include: Spotify, Uber or PayPal.
Some potential technical applications of Kafka include:
- Gathering data for the purposes of real-time analysis (-> Apache Spark)
- Data stream processing (-> Apache Storm)
- Assuming the role of Message Oriented Middleware (-> MOM)
- Aggregating application logs
BlueSoft has extensive experience in Apache Kafka, dating back as early as the incubation stage of this technology. We have successfully implemented Kafka-based production solutions at a number of banking sector clients.
BlueSoft has successfully implemented many projects in this area. We will happily present our portfolio directly as well as answer more questions about technology itself and benefits to be brought by its implementation.