New Lenses Multi-Kafka Developer Experience

Download

4 data streaming trends for 2025

By Guillaume Aymé
Dec 20, 2024

Buckle up, we’re past the AI hype. Now, it’s about making intelligent systems that act on our behalf.

In 2025, AI isn’t just a tool– it’s becoming our core way of operating, powered by real-time data. How we stream, manage and monetize that data will define the next generation of business.

Here, we zoom into four examples of what autonomous real-time intelligence could look like in the coming year. 

1. Building systems that build systems 

Engineers will increasingly become architects tasked to build the systems (AI) that build systems. By the end of 2025, the majority of software and data management pull requests will be triggered by AI agents.

Other engineers will become the managers of a team of AI Agents, with each Agent being a tool and given different responsibilities and set of capabilities. It will be down to the manager to instruct the agents. The human will very much still be in the loop. 

2. AI is connected by streams

To maximize AI and automation, businesses will expand their "streaming data fabric," interconnecting their different domains, clouds, and applications – whether on-premises, in the cloud, or at the edge. This streaming fabric will span multiple vendors and technologies, likely extending beyond a single organization as data is shared.

The management and orchestration of this data will be handled by both humans and AI on an "operating fabric" that remains decoupled and independent from infrastructure vendors.

This operating fabric will respond instantly to rapidly changing business requirements: scaling up, scaling down, classifying data, as well as moving, processing and sharing data across different clouds and parts of the business.

3. Real-time data is monetized 

With the AI boom, the monetary value of data – and more importantly, the ability to easily share it both internally and externally – will explode. Those who can share data as a real-time stream will benefit the most. Businesses will increasingly need to provide real-time data streams to their customers, suppliers, and partners.

4. The X factor: Developer Experience

Open formats like Apache Iceberg and Apache Kafka APIs have opened up the data infrastructure market. Giants like AWS, Azure, Databricks, and Snowflake – along with streaming players like Confluent and Redpanda – are all competing to host and process your data and run your AI and software applications.

So Developer Experience will become a crucial differentiator for these vendors as they vie to attract a new generation of engineers. And since the lines between software, data, and AI workloads continue to blur, we'll see tools across these domains converge into a unified experience.

Companies will pay a premium for tools that make AI integration easy, fast, and accessible to everyone in their organization. The best platforms won't just offer features – they'll offer simplicity and be a joy to use.