Stream Processing with Kafka Streams

Consuming records one by one is fine for simple work, but aggregations, joins, and windowing get messy fast. Kafka Streams is a library that models these as a topology running inside your service — no extra cluster required.

Define a topology

A StreamsBuilder describes the data flow. Here we count events per key and write the running totals to an output topic.

@Configuration
public class StreamTopology {
 
    @Bean
    public KStream<String, OrderEvent> pipeline(StreamsBuilder builder) {
        KStream<String, OrderEvent> orders =
            builder.stream("orders");
 
        orders
            .groupByKey()
            .count()
            .toStream()
            .to("order-counts");
 
        return orders;
    }
}

Windowed aggregations

Real-time analytics usually care about time buckets. Tumbling windows group records into fixed, non-overlapping intervals.

orders
    .groupByKey()
    .windowedBy(TimeWindows.ofSizeWithNoGrace(Duration.ofMinutes(5)))
    .count()
    .toStream()
    .to("orders-per-5min");

Enable it in Spring

@EnableKafkaStreams bootstraps the topology using your configured application id, which also names the internal state store topics.

spring.kafka.streams.application-id=orders-analytics
spring.kafka.streams.bootstrap-servers=localhost:9092

Takeaways

• Kafka Streams runs inside your app — no separate processing cluster.
• Use windowing for time-bucketed aggregations like per-minute counts.
• The application id names internal topics, so keep it stable across deploys.

State is backed by local RocksDB stores and replicated to Kafka, so instances can recover their computation after a restart.