Protocols
Saved & recent

Consumer group

Also called: Kafka consumer group, partition consumer group

A consumer group coordinates workers that share message consumption, but the precise rules depend on the messaging system. In Kafka’s partition-based consumer groups, each assigned partition belongs to one member at a time. Separate groups can independently read the same topic. This differs from a generic queue that distributes individual messages among available workers.

For example, a Kafka topic has four partitions and an indexing group has two consumers. Each consumer can own two partitions; a separate analytics group reads the topic independently. Membership changes can trigger reassignment, and committed offsets determine where processing resumes. Group membership alone does not make side effects exactly once. Other systems’ queue groups and Kafka’s share groups use different coordination models.

Read first
Kafka wire protocol
Related
Kafka wire protocol, Consumer acknowledgment, Delivery semantics
Further reading
Apache Kafka — KafkaConsumer API
Kafka topic partitions P0 P1 P2 P3 indexer A · P0 / P1 indexer B · P2 / P3 indexing group offsets P0 P1 P2 P3 analyst A · P0 / P1 analyst B · P2 / P3 independent analytics group offsets same topic · separate group assignments
A partition belongs to one member within each Kafka partition-based group. The indexing group shares four partitions between A and B; the separate analytics group assigns all four to C. Their illustrative P0 offsets differ.

View in the full Protocols collection