Skip to main content

General

Export to Apache Kafka

High-throughput Kafka export with partitioning, replication, and resilient delivery.

Enterprise-grade export to Apache Kafka with support for partitions, replication, and high-throughput streaming. Ideal for building streaming data pipelines and real-time analytics.

open_in_new

Official Documentation

kafka.apache.org

Configuration Parameters

ParameterRequiredTypeDefaultDescription
BootstrapServersNostringlocalhost:9092Comma-separated list of Kafka broker addresses.
TopicNostringproxus_device_dataTarget Kafka topic for data export.
ClientIdNostring(empty)Client identifier for metric reporting.
SecurityProtocolNostringPlaintextSecurity protocol: Plaintext, Ssl, SaslSsl, SaslPlaintext.
SaslMechanismNostringPlainSASL mechanism: Plain, ScramSha256, ScramSha512, etc.
SaslUsernameNostring(empty)SASL authentication username.
SaslPasswordNostring(empty)SASL authentication password.

Internal Behavior

Delivery Guarantees

This connector configures the Kafka producer for safe delivery:

  • EnableIdempotence: Enabled to reduce duplicate writes on retries
  • Retries: 3 retries with 1s backoff
  • Request Timeout: 30s

Performance Features

  • Async Publishing: Each payload is published asynchronously.
  • Message Headers: Adds DeviceName, DeviceId, and Timestamp headers.
  • Message Key: Uses the payload key as Kafka message key.