Kafka Streams is a deployment-agnostic stream processing library written in Java. Even though Kafka has a great test coverage, there is no easy way to write unit-tests for processing topologies, until now. I released Mocked Streams for Scala, which simply allows you to unit-test multi-input and output stateful stream topologies (since Apache Kafka >= 0.10.1) without Zookeeper and Kafka Brokers in your favorite Scala testing framework e.g. ScalaTest and Specs2.
import com.madewithtea.mockedstreams.MockedStreams val input = Seq(("x", "v1"), ("y", "v2"))
val expe = Seq(("x", "V1"), ("y", "V2"))
val strings = Serdes.String() MockedStreams()
.topology { builder => builder.stream(...) [...] }
.input("topic-in", strings, strings, input)
.output("topic-out", strings, strings, expe.size) shouldEqual expe
Mocked Streams
Mocked Streams 1.0 is a library which allows you to do the latter without much boilerplate code and in your favourite Scala testing framework. It wraps the org.apache.kafka.test.ProcessorTopologyTestDriver class, but adds more syntactic sugar to keep your test code simple. The features of the 1.0 release include:
- Multiple input and output streams
- Specify your topologies as a function: (KStreamBuilder => Unit)
- Use .config to pass your Kafka Streams configuration (e.g. Timestamp extractor)
- Use .outputTable to match against the compacted table output as Map[K,V]
- Support for Scala 2.11.8
- Support for Apache Kafka 0.10.1.0
Multiple Inputs and Outputs
val ms = MockedStreams()
.topology { builder => builder.stream(...) [...] }
.input("in-a", strings, ints, inputA)
.input("in-b", strings, ints, inputB)
.stores(Seq("store-name")) ms.output("out-a", strings, ints, expA.size) shouldEqual(expectedA)
ms.output("out-b", strings, ints, expB.size) shouldEqual(expectedB)
Getting Started
See the README for using Mocked Streams. For examples check out the test code of Mocked Streams itself. Mocked Streams is also located in the Maven Central Repository, hence you just need to add:
libraryDependencies += "com.madewithtea" %% "mockedstreams" % "1.0.0" % "test"
If you have any questions or issues regarding Mocked Streams, get in touch via Github. Alternatively, you can get in contact via mail.