Kafka Offset

Kafka offset represents the index of a Kafka Events (Message, Record) inside a Kafka Partition.

For example:

kafka_offset_0 = value_of_kafka_message
kafka_offset_1 = value_of_another_kafka_message

kafka_partition1 = [kafka_offset_0, kafka_offset_1]
kafka_topic = [kafka_partition0, kafka_partition1]

The same kafka offset can has different message in different partition.

Use case

For Kafka Consumer to iterate and consume the messages. And also provide a mechanism for the consumers to continue where it left off.

auto.offset.reset

A setting to determine the action a consumer should take if there is no offset that has been committed to the current consumer group in which the consumer can choose to read from:

• earliest: from the earliest of the topic
• latest: from the last offsets