@sensejs/kafkajs-standalone

This package provides easy to use kafka consumer and producer based on [kafkajs], without core parts of sense.js as dependencies.

Usage no npm install needed!

<script type="module">
  import sensejsKafkajsStandalone from 'https://cdn.skypack.dev/@sensejs/kafkajs-standalone';
</script>

README

Standalone Wrapper for KafkaJS

This package provides easy to use kafka consumer and producer based on kafkajs, without core parts of sense.js as dependencies.

Install

npm install @sensejs/kafkajs-standalone kafkajs

Highlights:

  • Consumer:

    1. Automatically set partitionConsumedConcurrently to number of partitions of a topic, to maximum the throughput.

    2. Detect rebalance as quickly as possible.

    3. Save offset when consumer group is going to rebalance, to avoid any message be re-consumed as possible as it can (while still you should not expect it won't happen).

  • Producer:

    1. Default maxInFlightRequests to 1 and idempotent to true, to fit requirements of transactional producer. Can be overridden if transaction is not required.

    2. Ability to provide messageKeyProvider to assign a key to message if not specified.