Stream Reactor released for Kafka Connect 0.11 and 1.0

Stream Reactor, the largest open-source collection of Apache Kafka connectors, has released today many new features, bug fixes and new connectors for Apache Pulsar!

• Version 0.4.0 for Kafka Connect 0.11

• Version 1.0.0 for Kafka Connect 1.0

A big thank you goes to the community who contributed to this release, not only with raising issues, but also with talking with us and raising multiple pull requests (PRs). Lets have a closer look at some of the changes:

About StreamReactor

Stream Reactor is an Apache License, Version 2.0 open source collection of components built on top of Kafka and provides Kafka Connect compatible connectors to move data between Kafka and popular data stores. Stream Reactor provides source connectors to publish data into Kafka and sink connectors to bring data from Kafka into other systems. The connectors support KCQL (Kafka Connect Query Language), an open source component of the Lenses SQL Engine that provides an elegant and simple SQL like syntax for selecting fields and routing from sources or topics to Kafka or the target system (topic to target entity mapping, field selection, auto creation, auto evolution, error policies).

We hope you find Stream Reactor useful and want to give it a try! Stream Reactor has over 25 connectors available, tested and documented, supporting both Kafka 0.11 and Kafka 1.0 and you can give it a try by downloading Lenses Development Environment or find the jars on GitHub or even build the code locally and help us improve and add even more connectors.

New Connector: Apache Pulsar source and sink

Apache Pulsar is an open-source distributed pub/sub messaging system originally created at Yahoo and now part of the Apache Software Foundation.

In our effort to provide you with the best tooling and experience in building and operating meaningful data pipelines, we are continuously working with streaming technologies. As part of that, we are now releasing Kafka Connectors for Apache Pulsar that allow you to source or sink data in and out of both Kafka and Pulsar with ease.

Fixes and Features

Fix in kafka-connect-common to handle primary keys with doc strings thanks, @medvekoma!

Cassandra connector

• Write to the same table from more than 1 source topic #284

• Upgrade to Cassandra driver 3.3.0 and refactor Cassandra tests

• Fix on Cassandra source, configurable timespan queries. You can now control the timespan the Connector will query for

• Add DELETE functionality to support delete CDC functionality, thanks!

• Allow start poll timestamp in Cassandra source

• Allow setting initial query timestamp on Cassandra source

FTP connector

• Add FTPS support. You can set up now ftp.protocol to be ftp (default) or ftps.

MQTT connector

• Fix for MQTT source High CPU Thanks!

• Add support for setting the Kafka Message Key value to be one of the incoming the MQTT payload fields.

JMS connector

• Fix on JMS source transacted queues #285 thanks!

Redis connector

• Allow multiple primary keys on the redis sink

InfluxDB connector

• Fixes to handle dates and big decimal in Influx sink

Learn more

Follow us on Twitter @lensesio, chat with in the Slack community channel to talk with the community. If you find problems or have ideas on how we can improve Stream Reactor, please let us know or log an issue.