Stream Reactor released for Kafka Connect 0.11 and 1.0
Stream Reactor, the largest open-source collection of Apache Kafka connectors, has released today many new features, bug fixes and new connectors for Apache Pulsar!
- Version 0.4.0 for Kafka Connect 0.11
- Version 1.0.0 for Kafka Connect 1.0
A big thank you goes to the community who contributed to this release, not only with raising issues, but also with talking with us and raising multiple pull requests (PRs). Lets have a closer look at some of the changes:
Stream Reactor is an
Apache License, Version 2.0 open source collection of components built on top of Kafka and
provides Kafka Connect compatible connectors to move data between Kafka and popular data stores. Stream Reactor
source connectors to publish data into Kafka and
sink connectors to bring data from Kafka into other
systems. The connectors support KCQL (Kafka Connect Query Language),
an open source component of the Lenses SQL Engine
that provides an elegant and simple SQL like syntax for selecting fields and routing
from sources or topics to Kafka or the target system (topic to target entity mapping,
field selection, auto creation, auto evolution, error policies).
We hope you find Stream Reactor useful and want to give it a try! Stream Reactor has over 25 connectors available, tested and documented, supporting both Kafka 0.11 and Kafka 1.0 and you can give it a try by downloading Lenses Development Environment or find the jars on GitHub or even build the code locally and help us improve and add even more connectors.
New Connector: Apache Pulsar source and sink
Apache Pulsar is an open-source distributed pub/sub messaging system originally created at Yahoo and now part of the Apache Software Foundation.
In our effort to provide you with the best tooling and experience in building and operating meaningful data pipelines,
we are continuously working with streaming technologies. As part of that, we are now releasing Kafka Connectors
for Apache Pulsar that allow you to
sink data in and out of both Kafka and Pulsar with ease.
Fixes and Features
Fix in kafka-connect-common to handle primary keys with doc strings thanks, @medvekoma!
- Write to the same table from more than 1 source topic #284
- Upgrade to Cassandra driver 3.3.0 and refactor Cassandra tests
- Fix on Cassandra source, configurable timespan queries. You can now control the timespan the Connector will query for
- Add DELETE functionality to support delete CDC functionality, thanks @sandonjacobs!
- Allow start poll timestamp in Cassandra source
- Allow setting initial query timestamp on Cassandra source
- Add FTPS support. You can set up now
- Fix for MQTT source High CPU Thanks @masahirom!
- Add support for setting the Kafka Message Key value to be one of the incoming the MQTT payload fields.
- Fix on JMS source transacted queues #285 thanks @matthedude!
- Allow multiple primary keys on the redis sink
- Fixes to handle dates and big decimal in Influx sink
Follow us on Twitter @lensesio, chat with in the Slack community channel to talk with the community. If you find problems or have ideas on how we can improve Stream Reactor, please let us know or log an issue.