Confluent launches Streaming Agents to scale real-time AI

Confluent has announced Streaming Agents, a new capability that enables enterprises to build and scale AI agents that can monitor, analyse, and act upon real-time data.

The new feature is now available within Confluent Cloud for Apache Flink, providing organisations with tools to more easily create AI-powered agents that leverage live business data for decision-making and automated action. The company said Streaming Agents helps overcome barriers that many enterprises face when moving artificial intelligence (AI) initiatives from experimental phases to production-ready systems.

Addressing prototype challenges

Research by IDC indicates that between 2023 and 2024, organisations piloted an average of 23 generative AI proofs of concept, yet only three typically reached production. This often leaves companies caught between promising prototypes and real-world deployments that meet strategic business objectives.

"Agentic AI is on every organisation's roadmap. But most companies are stuck in prototype purgatory, falling behind as others race toward measurable outcomes," said Shaun Clowes, Chief Product Officer at Confluent. "Even your smartest AI agents are flying blind if they don't have fresh business context. Streaming Agents simplifies the messy work of integrating the tools and data that create real intelligence, giving organisations a solid foundation to deploy AI agents that drive meaningful change across the business."

The complexities and costs associated with integrating real-time data into agentic AI are significant, leading to inconsistencies and unreliable outcomes. According to IDC, among generative AI projects that do make it to production, only 62% meet expectations, with the remaining falling short due to challenges including data lag and insufficient context.

Stewart Bond, Vice President of Data Intelligence and Integration Software at IDC, commented on the requirements for successful deployment. "While most enterprises are investing in agentic AI, their data architectures can't support the autonomous decision-making capabilities these systems require," he said. "Organisations should prioritise agentic AI solutions that offer easy, secure integration and leverage real-time data for the essential context needed for intelligent action."

Integrating real-time data and AI

Streaming Agents integrates agentic AI directly into streaming data pipelines, utilising Apache Kafka and Apache Flink to enable event-driven automation. This integration provides agents with timely contextual data, allowing them to adapt, make decisions, and communicate with other systems and agents dynamically as business conditions evolve.

The system is designed to operate continuously, handling high volumes of data and delivering context-aware reasoning similar to human operators. One use case highlighted by Confluent is in competitive pricing, where Streaming Agents can monitor prices across multiple eCommerce platforms and automatically adjust a retailer's prices to remain competitive.

Key features and capabilities

Among the central features of Streaming Agents are:

• Tool calling for context-aware automation: Using the Model Context Protocol (MCP), agents are able to invoke the most suitable external tool or system, such as databases, APIs, or SaaS applications, according to current business context and activity.
• Connections for secure integrations: The platform enables secure connections to models, vector databases, and MCP through Flink, with credential protection, connection reusability across multiple assets, and centralised management for large-scale deployments.
• External Tables and Search: This feature allows enrichment of streaming data with external sources, such as relational databases and REST APIs. Organisations can thus provide AI systems with a more accurate and comprehensive data context, supporting retrieval-augmented generation applications and enhancing decision accuracy.
• Replayability for iteration and safety: Teams can develop and evaluate agents using real data without live impacts, supporting dark launches, A/B testing, and accelerated iteration.

Industry implications

By bringing agentic AI into streaming data environments, Confluent aims to assist organisations that have found it difficult to move beyond experimentation with AI and into full-scale, reliable deployments. This technology is positioned to help enterprises automate real-time responses to changing data inputs and business signals, while maintaining security and integration with existing systems.

Streaming Agents is currently available in open preview to customers using Confluent Cloud for Apache Flink, as the company seeks broader feedback from enterprise users before general release.