Google has outlined a way to run generative AI workflows in Dataflow, using a filtering step to limit when an AI agent is called.
The approach is aimed at streaming data pipelines that handle large volumes of events, including customer support messages, system logs, financial transactions and industrial telemetry. Most records in these streams are routine and do not require multi-step reasoning or external tool use, Google said.
Under the model, all incoming events first pass through a lightweight sentiment classifier running on Dataflow worker CPUs via Apache Beam's RunInference framework. Messages classified as positive or neutral are dropped, while negative messages are passed to a downstream agent built with Google's Agent Development Kit and backed by Gemini 3.5 Flash.
The agent then decides what action to take at runtime. In Google's customer service example, it looks up a user in BigQuery, checks order and inventory records, chooses a response such as a refund or replacement, sends an email through the Gmail API and logs the outcome.
Cost focus
Google presented the design as a way to address a core challenge in adding generative AI to streaming systems: the mismatch between high-volume event processing and the cost and latency of large language models. Sending every event to a model with external tools attached would increase token charges, slow pipelines and risk hitting API limits, it said.
The filtering stage is meant to ease those pressures by keeping most traffic on a local CPU path and reserving agent-based processing for a small minority of records. In a typical support scenario, fewer than 5% of messages might be routed onward, while the remaining 95% are handled without incremental external API cost, according to Google.
Google also argued that the setup helps Dataflow maintain throughput. CPU-based inference can be distributed across many worker instances and completed in milliseconds, while the heavier agent path may take seconds because it can involve database queries and other application calls.
Adaptive execution
The design also reflects a broader effort to make fixed streaming architectures more flexible. In a conventional streaming Directed Acyclic Graph, execution paths are defined when the pipeline is deployed, so changing handling logic typically requires code changes, testing and redeployment.
By inserting an AI agent into one stage of the workflow, the pipeline can preserve a static structure for most events while allowing more adaptive behaviour when complex cases arise, Google said. Rather than hard-coding large numbers of conditional branches into Apache Beam, the agent chooses which tools to call once it receives a qualified event.
Google said the pattern could apply beyond customer service. It cited IT operations, where millions of routine logs could be filtered before an agent investigates critical anomalies; fraud analysis, where transactions could be screened locally before suspicious cases trigger database lookups; and industrial systems, where normal telemetry stays on a fast path while erratic readings prompt remediation steps.
Tool chain
The implementation combines several parts of Google's cloud and AI stack. Data is ingested through Pub/Sub, the sentiment model uses a Hugging Face classifier, the downstream agent is assembled with the Agent Development Kit, and supporting data is queried in BigQuery.
The email step uses the Gmail API. In the example architecture, the agent has three tools: one to retrieve a user's email address, one to fetch orders and current inventory information, and one to send a plain-text message to the customer.
The setup is integrated through Apache Beam's native RunInference transform rather than separate orchestration code, Google said. That means the same framework handles model execution, the filtering stage and the downstream agent call inside the pipeline.
Google's example centres on sentiment analysis, but the broader point is that a small local model can act as a gatekeeper for a more expensive AI workflow. In practice, the value of that pattern will depend on whether organisations can define reliable filters that keep false negatives low while reducing the number of events escalated to the agent.
The approach is designed for streams in which a very high share of events are routine and only a small subset need contextual reasoning and tool use at runtime, Google said.