Kafka Producers and Consumers in Spring Boot

Kafka is a distributed commit log: producers append records to topics and consumers read them at their own pace. Because the log is durable, consumers can replay history — which makes Kafka a great backbone for event-driven systems.

Configure the connection

spring-kafka reads most settings from properties. Set a consumer group so partitions are distributed across instances.

spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=orders-service
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.producer.acks=all

Produce records

KafkaTemplate handles serialization and partitioning. Sending returns a future so you can react to delivery success or failure.

@Service
public class OrderProducer {
 
    private final KafkaTemplate<String, OrderEvent> template;
 
    public void publish(OrderEvent event) {
        template.send("orders", event.orderId(), event)
            .whenComplete((result, ex) -> {
                if (ex != null) log.error("publish failed", ex);
            });
    }
}

Consume records

@KafkaListener binds a method to a topic. Spring manages the polling loop, offset commits, and concurrency for you.

@Component
public class OrderConsumer {
 
    @KafkaListener(topics = "orders", groupId = "orders-service")
    public void handle(OrderEvent event) {
        inventory.reserve(event.orderId());
    }
}

Takeaways

• Use acks=all for durability on writes you cannot lose.
• The consumer group id controls how partitions are shared across instances.
• Keep listener methods idempotent — Kafka guarantees at-least-once delivery.

For failures, configure a DefaultErrorHandler with retries and a dead-letter topic so poison messages don't block the partition.