Google Cloud expanded its AI agent monitoring and evaluation tools on November 13, 2026, adding customer-controlled logging, dashboards, privacy safeguards and spending limits across BigQuery and Looker for managed data agents and custom agents built with Agent Development Kit (ADK) or LangGraph.
The system uses a common telemetry path for first-party and custom agents. Managed Conversational Analytics agents emit telemetry automatically, while custom ADK or LangGraph agents can send structured telemetry with single-line SDK instrumentation. The data lands in BigQuery tables owned by the customer for analysis in Google Cloud’s developer and administrator tools.
Several core monitoring features are generally available (GA). BigQuery Studio’s prebuilt dashboards track usage, active sessions, top knowledge sources and latency. Looker System Activity dashboards for Conversational Analytics are also GA in Looker (Google Cloud core) and Looker (original), covering engagement and token usage. User ratings and query review in the Responses & feedback tab remain in preview.
Operational Traces Exclude Conversation Content
Google Cloud separates operational telemetry from conversation content so administrators and agent owners can govern the two layers independently. Cloud Monitoring supplies aggregate metrics for request volume, latency and token usage, while Cloud Trace provides step-by-step spans for each request. Cloud Trace observability scopes can consolidate telemetry across multiple agent projects.
Those operational metrics and sanitized first-party trace spans exclude customer core content, including user prompts, model responses, internal thoughts and generated SQL. This allows platform teams to monitor activity without automatically giving operational tools access to the substance of each interaction.
For deeper auditing, agent owners and editors can opt into conversation logging for individual agents. A Google-managed service agent then streams prompts, responses and turn-level telemetry through the BigQuery Storage Write API into a customer-owned agent_events table. The table inherits the dataset’s identity and access management permissions, customer-managed encryption keys and VPC Service Controls.
Privacy guardrails check each turn before logging and skip logging for that turn when a knowledge source uses row-level security, column-level policy tags or dynamic data masking. If a logging error occurs, the user’s chat session continues uninterrupted. The design therefore distinguishes broad operational visibility from narrower, explicitly enabled access to full interactions.
Google Cloud says custom agents can retain more diagnostic detail through the open-source ADK plugin, which can stream full tool-invocation trees, time-series telemetry and multimodal payloads into BigQuery. First-party conversation logging deliberately strips low-level tool arguments and internal SQL.
Spending Limits Extend Beyond Dashboards
The logs also provide a basis for cost management. Each BigQuery conversation turn records input, output, thought, cached and total token counts alongside total turn latency. Looker System Activity separately reports estimated input and output tokens across business-intelligence workloads.
Per-query caps on BigQuery bytes scanned use dry-run estimates to block oversized queries without incurring on-demand scan charges. Separately, daily input and output token quotas for BigQuery and the Conversational Analytics API are in preview. Those quotas can be applied per project, user or agent through the Google Cloud console or gcloud CLI.
The combination pairs consumption reporting with active limits. Byte-scan caps address query spending before execution, while the preview token quotas extend enforcement to agent consumption across projects, users and individual agents.
Evaluation Results Sit Beside Production Logs
Google Cloud’s open-source Prism tool tests Conversational Analytics agents grounded in BigQuery tables or Looker Explores. It supports reusable test suites, regular-expression assertions and multi-region agent discovery, with evaluations focused on SQL accuracy and answer quality before prompt, model or schema changes reach users.
Prism can write completed evaluation runs to BigQuery tables partitioned by day, placing benchmark results alongside production agent_events records for analysis in BigQuery and Looker. Native first-party BigQuery agent evaluations are separately available in preview through an API and a guided Colab notebook.
The announcement makes customer-owned BigQuery telemetry the common layer for monitoring, cost controls and evaluation, tying agent oversight to existing warehouse permissions and security controls. Managed conversation logging and the main dashboards are GA, while token quotas, native evaluations and some Looker feedback features remain in preview. For production teams, deployment design therefore includes both the boundary between operational traces and full conversations and the differing availability of each control.
Credit: Source link
