Why we built a new Kafka client for Node.js The Node.js world needs better tools. Apache Kafka is vital for real-time data. It powers many businesses, especially in Fintech and Media. These fields see heavy data usage and need reliable solutions.

Node.js developers faced a tough choice. They could pick KafkaJS or node-rdkafka: 1) KafkaJS is a pure JS library but is no longer maintained 2) Node-rdkafka, while fast, it struggles with compatibility and worker threads This gap left teams searching for a modern Kafka client.

Many of our Platformatic customers asked for a solution. We knew we had to act. Today, we introduce @platformatic /kafka. This new library is built for modern Node.js. It focuses on performance, ease of use, and TypeScript support.

In our latest release, we compare @platformatic /kafka with KafkaJS and node-rdkafka. The three libraries offer different experiences. This comes from their age and the structure of node-rdkafka's C++ base. We look closely at how they handle data and their Admin API experience.

We run benchmark analysis on all three libraries We tested all three libraries in two common scenarios: consuming and producing data. Our target cluster had three brokers running on Docker. All data was produced and consumed for one topic with three partitions.

In the end, the @platformatic/kafka library stands out. Our benchmarks show that it offers better performance with a better developer experience than alternatives. Thanks to the built-in serialization support, TypeScript integration, and a simple API, @platformatic/kafka is the ideal choice. It helps devs build strong and maintainable Kafka applications in Node.js.

Read the full release here:

This is very clean! Thank you.

@matteocollina I’ve been meaning to ask you for a while. How would you parse large JSON without blocking the event loop in Node? Use a stream parser?

It really depends what you need to do with it. One option is to just send the whole payload to a worker thread and process it there. Another option is to use simdjson and only extract the fields you need.

The API is really nice! Awesome work!

Thanks!