What If Media Group stands at the forefront of AdTech innovation, where real-time data processing has become the cornerstone of success. As consumer expectations evolve, the ability to deliver hyper-personalized messages at the exact right moment has become essential for performance marketing.

Their platform ARIA builds universal user profiles by analyzing behavior across web, mobile, and voice, all in real-time. Acting as a “hivemind of models,” it predicts the best ad to show, on the right channel, at the perfect moment. This powers dynamic bidding driven by AI and streaming data - where milliseconds and conversion confidence determine success.

Real-time data processing for hyperpersonalized customer acquisition, at scale, is a great example of what Apache Kafka was built for. What If Media Group recognized this early, standardizing on Kafka as the foundation of data movement across their business and different services, allowing them to build cutting-edge applications.

Running multiple Kafka clusters is important for What If Media. They needed to host different applications on different clusters. This was to isolate workloads but also to give more autonomy to different teams, so they could run their own infrastructure.

Their growth has been remarkable: starting with a single Kafka cluster for processing leads for their insurance customers, they've scaled to 15 clusters accessed by more than 40 software, data, and AI engineers. Like many businesses, What If Media Group found themselves adopting different Kafka vendors and deployments over time, through choice and acquisitions - from open-source Kafka deployed on ECS and AWS MSK, to Strimzi on Kubernetes and Confluent.

Their leadership philosophy centers on engineering autonomy as the engine of innovation. By giving developers and data scientists direct access to real-time data streams, they can create an environment where products can be created and improved without reliance on a centralized team.

Despite their success with Kafka, they faced significant challenges managing their expanding infrastructure and maintaining developer productivity. With 15 Kafka clusters running across different vendors and platforms, maintaining unified governance and developer experience became increasingly difficult. Having data siloed in different clusters was good architecturally, but made it particularly difficult to foster a culture of innovation and data sharing across teams:

Troubleshooting streaming applications across different clusters was particularly painful. Engineers had to dig through command line interfaces, looking up multiple commands just to find specific messages in a topic. What should have been quick fixes often turned into hours-long investigations.

The company faced a critical question: How could they empower all their engineers and data scientists to work effectively with real-time data while maintaining governance across different clusters?

What If Media Group implemented Lenses Developer Experience to create a single interface across their 15 Kafka clusters, dramatically changing how their teams interact with data:

This approach transformed Kafka from a tool used by a handful of specialists to a source of innovation accessible to their entire engineering organization. Ryan Anguiano describes Lenses as “the stairs” that help developers climb the Kafka learning curve. “It's not just 5 highly specialized engineers that can work with data in Kafka anymore – it's 40+ engineers and data scientists”, Anguiano explains. “This fundamentally changes our business.”

Lenses Global SQL Studio was particularly useful; developers could use familiar syntax to find messages across any cluster. The governance capabilities underpinning this decentralized approach allow teams to collaborate effectively while maintaining appropriate access controls and visibility across their technology stack.

“Using Lenses SQL over our multiple Kafka clusters is a way to drive product innovation and create new applications that wouldn’t otherwise be possible. Engineers can now write a query to search across Kafka environments to find and work with data they wouldn’t otherwise have had access to. We can open up our imagination and our opportunities with the data we have”, Anguiano adds.

What If Media Group prides themselves on empowering their engineers to be autonomous and creative with data. In today's rapidly evolving tech landscape, their leadership recognizes that equipping engineers with cutting-edge tools is essential for staying competitive:

The impact on What If Media Group's operations has been transformative. Across the organization, five different teams have adopted Kafka with Lenses for their specific needs:

• The voice & leads team processes web leads and routes phone calls to insurance agents
• The website team manages consumer-facing sites that generate leads
• The messaging team reads events to trigger direct marketing across various channels, using AI models to generate personalized content
• The delivery team handles email processing
• The data engineering team supports all teams with data movement and transforms

Troubleshooting that once took hours now takes minutes. “You might spend hours looking up commands, typing them in, fixing spelling errors... Lenses just gives you the ability to visually see everything at the same time and pick up on cues”, says Ryan Anguiano.

Looking ahead, What If Media Group is consolidating their infrastructure by migrating their clusters to Strimzi on Kubernetes. This transition offers the ideal balance between control and ease of management without disrupting developer workflows, as Lenses works consistently across any Kafka deployment.

“Strimzi gives you the same level of control as running Kafka on your own EC2 instances but with minimal config. It replaces what was an entire repo with a single configuration file”, notes Anguiano.

With their Kafka clusters now unified under a single pane of glass, What If Media Group is ready to tackle more ambitious data processing goals. They plan to use Lenses SQL Processors to move ETL operations into their real-time streaming pipeline, shifting data processing left so it is performed closer to the source.

This strategic shift will provide consistent, timely, reliable access to important business data where and when it is needed – powering the next generation of AI-driven marketing solutions and cementing What If Media Group's position as a leader in performance marketing innovation.