Kafkastreams

In this post I will explore the ksqlDb API for creating streams. This is the second post about using ksqlDb with Spring Boot. Read the first post here. A stream in ksqlDb is analogous to a stream in Kafka Streams. One difference is how you create the stream. As described in the first post, a stream in Kafka Streams is created programmatically with an API. StreamsBuilder streamsBuilder = new StreamsBuilder(); KStream<Integer, Order> ordersStream = streamsBuilder.stream("orders"); ordersStream .filter((key, value) -> value.orderType == Electronics) .groupByKey() .aggregate(() -> 0.0f, (key, value, sum) -> sum += value.orderAmount) .toStream() .to("output"); Topology topology = streamsBuilder.build(); KafkaStreams kafkaStreams = new KafkaStreams(topology, new Properties()); kafkaStreams.start(); ksqlDb uplevels this to a more familiar SQL syntax. Here’s an example from the ksqlDb quickstart. ...

In this post I will show how to use the ksqlDb Java REST client to interact with the ksqlDb server using a Spring Boot application as a foundation. The Boot app will also use Spring Webflux to help expand the use of reactive programming. I will also show the use of the Testcontainers framework for integrating deployment of Docker containers as part of integration testing, in this case with JUnit 5. What this will not cover is details of Apache Kafka or Spring Boot, or even ksqlDb and streaming systems, except to provide background to understand the code. There are plenty of other examples and posts that cover those details. ...