Fast Data CSD 3.2 release

Fast Data 3.2 is officially out! This release cycle took longer than usual but it brings many changes that will provide you with a more streamlined Kafka experience and let us build, test and enhance future releases quicker and with more confidence. The release has been available for our clients since last month.

Fast Data is our solution for installing and managing a modern Kafka stack through Cloudera Manager. Check here for an overview and request a trial today!

If you already use our CSD, read our documentation for instructions on how to upgrade without downtime. We are always available to help and can arrange for an engineer to walk you through.

The 3.2 release will support the 3.2.x Kafka release from Confluent. We already provide the Confluent OSS 3.2.2 package as a Cloudera parcel.

The most significant change is that we altered the configuration layout. Up until now we imitated the way Cloudera setups their Kafka which can be counterintuitive to Kafka users. The main example for this are the broker listeners. Kafka brokers support 4 types of listeners (plaintext, sasl_plaintext, sasl_ssl, ssl). In typical Kafka fashion you configure any of these listeners and then add them to the broker listeners’ string. Cloudera’s way is to provide options such as enable kerberos and enable ssl and then deduct one type of listener to use. Depending on your setup, you may need more than one type of listeners, for example SSL has a significant performance impact on Kafka, so you may want to restrict SSL to some clients. With the new CSD you can get the exact setup you want and transfer knowledge from almost any Kafka resource.

Other enhancements include support for all authentication scenarios by all roles, so for example Schema Registry or Kafka Connect can use Kerberos, SSL or a combination to connect to the brokers and all Kafka options are now available, with their descriptions, in the web interface.

As Kafka REST 3.2.x provides the new v2 API, our Fast Data Tools CSD also got upgraded with the new and shiny Kafka Topics UI 0.9.x. Now you can browse through your messages using offsets or selecting partitions!

Development though, isn’t the only factor in the length of a release cycle. The other significant factor is testing. This is a personal opinion; as Kafka becomes mainstream, its development has accelerated and new features are added on every release cycle. While this is very nice indeed, Kafka is rolling release software for now. You can’t get a stable version of Kafka that has only bug and security fixes. In order to get bug fixes you have to get a new version of Kafka, which, due to the new features, introduces new bugs.

At Landoop we try to hit any bugs or inconsistencies before our clients, so we can either prevent them or provide quick support when needed. Fast Data makes it easy to restart one broker, one connect worker or your whole Kafka cluster in a few seconds with a couple mouse clicks. Hopefully you will have to restart your cluster rarely, ideally only on upgrades. Us on the other hand, we may restart our clusters thousands of times during CSD development and testing, trying to weed out any issues. When we do encounter an issue, we debug and try to understand the cause, find a solution and if possible a way to avoid it altogether. This is integration testing.

You can find the new CSD in your client area. Our documentation has also been updated for the new release.

Stay tuned for the future, we plan on improving constantly the whole range of our CSDs.