Lenses.io has joined forces with Celonis | Learn more >
Hero Lense

Last Updated: August 2021

The Managed Kafka comparison guide

Which Apache Kafka Managed service should you choose?

Apache Kafka Cloud Compass

Optimizing your Kafka in 2021 and beyond

Some traditional businesses are being pushed to the cloud for the first time. Digital natives are assessing or refining their single or multi-cloud strategy.

We now know that cloud migrations aren’t a one-time- 2-month project, but an ongoing process. The Kafka Cloud comparison 2021 is an unbiased resource to help you assess which managed service may work best for you now and down the line.

If you haven’t already, get started by checking these key considerations when moving your Kafka to the cloud.

3 questions about your cloud requirements:

Use the filters to compare the most relevant managed Kafka services for you.

Which cloud do you want to host your Kafka on? (choose max 3)

Which features do you need?

Which Apache Kafka could work for you?

The relevant services are highlighted based on your selections.

Aiven

Amazon MSK

Azure HDInsight

Confluent Cloud

Instaclustr

Overview

ServiceGreat for...SuperpowersPricing ModelGet Started
AivenThe managed Kafka in any cloud you want.Great for those who want an all-in-one Kafka with fast, reliable set up, no vendor lock in, and real human support.
  • Open
  • Mature
  • Reliable
Compute/storage utilization (all inclusive price, including network costs)More info on Aiven for Kafka >

(with developer tooling)

AWS MSKThe one with open tooling that plays well with AWS.Great for those who have set up an AWS shop and want granular control over their environment.
  • Native to AWS
  • Highly customizable
  • Supporting Services
Resource consumptionMore info on AWS MSK >

(with developer tooling)

Azure HDInsightThe Microsoft-backed Enterprise Kafka with a powerful big data ecosystem.Great for a broad range of advanced messaging scenarios with Azure analytics and DevOps components.
  • Microsoft backed OSS distribution
  • Enterprise Grade Security
  • Lowest Price to Performance Ratio
Cluster billed per minuteMore info on Azure HDInsight >

(with developer tooling)

Confluent cloudThe one with fully-managed event streaming.Great for developers and enterprises who want to go ‘ops free’.
  • Serverless
  • Complete
  • Flexible
Resource consumptionDocs Confluent with Lenses >

Download ConfluentCloud >
InstaclusterThe truly open-source one.Great for those who want to scale applications without license fees or lock-in.
  • Dedicated support
  • Best-of-breed services
  • Consulting available
  • All inclusive pricing for infra & support
  • Run in your own cloud with per-node support fee
Docs Instaclustr with Lenses >

Hosted by...

ServiceAWSGoogle Cloud PlatformAzureIBMUp cloudDigital OceanCloud
Aiven
AWS MSK
Azure HDInsight
Confluent cloud
Instacluster✔*✔*✔*
*Via ‘run in customer managed infrastructure’

Deployment

ServiceKafka versionSingle/Multi TenantTerraform support
Aiven2.8Single by default
AWS MSK2.7 (2.2.1 recommended)Single
Azure HDInsight2.4.1 (early April)Single
Confluent cloud2.7Either
Instacluster2.5.1Single by default

Supporting Services

ServiceManaged MirrorMakerSchema RegistryKafka ConnectKafka Rest Proxy
Aiven (free)(free)
AWS MSK(with glue)
Azure HDInsight
Confluent cloudConfluent Replicator
Instacluster

Core Service Features

ServiceACL managementNo dealing with Zookeepers & costAccess to Infrastructure Performance MetricsBonus features
Aiven
  • Real Human Support
  • Single and Multi-AZ clusters
  • Multi-Cloud replication and migrations
  • Multi-Region, Global deployments
  • Connect Connectors (30+ fully-managed)
  • Stream processing with Lenses (Lenses SQL)
  • VPC Peering (free of charge)
  • AWS Transit Gateway
  • AWS PrivateLink
  • AWS Outposts deployments (on-premises)
  • Audit logs
  • RBAC
  • Encryption at-rest and in-transit
  • Bug Bounty programs
  • Penetration test reports
  • Karapace.io
AWS MSK (Cloudwatch)
  • VPC
  • KMS at rest
  • ACM for authentication
  • IAM for control-plan
  • Lambda
  • Kinesis Data Analytics
  • Granular pricing for capacity planning
  • Auto-scaling cluster performance
Azure HDInsight
  • No SaaS tiers to deal with, performance limited only by infrastructure
  • Single tenant architecture
  • Ranger based fine grained access control
  • AD based authentication
  • Always Encrypted
  • Growing ISV ecosystem
Confluent cloud
  • Stream processing with ksqlDB
  • 120+ self and fully-managed connectors
  • Single and Multi-AZ clusters
  • Schema registry
  • Private Networking options (VPC Peering, AWS Transit Gateway)
  • Audit logs
  • Role-based access controls
  • Encryption at-rest and in-transit
  • AWS Private Link & BYOK
  • Cloud Data flow & Metrics API
Instacluster
  • 24x7 Support included by default
  • Support includes Kafka application issues and bug fixes
  • Availability and Latency SLAs
  • Multi availability zones by default
  • VPC Peering
  • Private Network Clusters
  • Encryption at-rest and in-transit
  • Provisioning via Console, API or Terraform Provider
  • Bundled and supported Kafka Connect Connectors
  • Upload and run your own Kafka Connect Connectors
  • Lenses compatible
  • Audit log shipping
  • Customer Initiated Resize for vertical scaling
  • Horizontal scaling on request
  • Cluster configuration backups

Compliance

ServiceComplianceAccess to Infrastructure AuditsAuthentication methods
AivenISO/IEC 27001:2013, SOC2, GDPR, HIPAA, CCPA,PCI/DSSOut-of-the-box integrations includedSSL, SASL/PLAIN, SASL/SCRAM
AWS MSKHIPAA eligible, PCI, ISO, SOC 1,2,3(Cloudtrail)SSL, SASL/PLAIN, SASL/SCRAM
Azure HDInsightSOC2, PCI/DSS, HIPAAKerberos
Confluent cloud SOC 1/2/3, ISO 27001, PCI, CSA (level 1), EBACan get our SOC 1/2/3 and ISO 27001 reportsSASL/SCRAM/SAML/SSO
InstaclusterSOC2, PCI/DSS,HIPAASOC2 and PCI available on requestSASL/SCRAM

Managed service selected?
See what further tooling you might need:

Calculate Kafka productivity