Protocols
Saved & recent

Kafka wire protocol

Also called: Apache Kafka protocol

Kafka clients communicate with brokers using a versioned binary protocol over TCP. Requests cover operations such as discovering cluster metadata, producing records, and fetching records. Clients use metadata to find the brokers responsible for the topic partitions they need.

For example, a consumer requests records from a partition starting at an offset, while a producer sends records to its partition leader. Ordering is defined within a partition, not across an entire multi-partition topic. The wire protocol also carries coordination operations; Kafka is more than a generic queue with one global sequence.

Related
Message broker, Delivery semantics, Correlation ID, Consumer group, Schema registry
Further reading
Apache Kafka — Protocol guide
client broker metadata request partition metadata Produce to partition 0 leader Fetch from partition 0 offset 6 records from offset 6 partition 0 leader 4 5 6 7 partition 1 17 18 19 20 offset order within each partition
Illustrative Kafka exchange: metadata identifies a partition leader; the client produces records and fetches from a specified offset. Offsets 0, 1 and 2 belong independently to each partition. The basic Fetch path is requested by the client, not broker push.

View in the full Protocols collection