Vortexa is an energy analytics provider. They offer the most complete dataset for global refined products and crude oil movements which supports traders to formulate forward trading decisions on the market. Their advanced real-time analytics service is built around a stream data platform based on Apache Kafka.
“While Kafka and Kafka Streams are extremely powerful technologies, their complexity and “black-box” nature led to slower than desired deployments of our applications. Lenses created unprecedented visibility into the real-time data flows in Vortexa’s Kafka infrastructure, helping to reduce development cycle times and build confidence in R&D”
Maksym Schipka, CTO, Vortexa
As Vortexa scaled up their Kafka environment they encountered several challenges. Administrative tasks were burdensome and required a lot of manual effort. Investigation of incidents involved a series of in-house developed scripts and open source tools.
Today, new data pipelines that integrate data across their different technologies, including Elastic Search and Redis, can now be built and deployed in production in minutes.
Everyday tasks, such as inspecting messages on a stream, understanding profile of a topic, or verifying the replication factor are now easy and are performed in under ten seconds.
They see a real-time topological view of their entire data landscape: their flows to Redis and Elastic Search as well as inside their Kafka Streams applications. Troubleshooting a problem on a streaming application can now be done within one minute compared to hours using previous tools.