Cloud Blog

The Blueprint: How Movix fills a gap in dental skills with specialized agentic AI

Fri, 22 May 2026 16:00:00 +0000

Welcome to The Blueprint, a regular feature where we highlight how Google Cloud customers are tackling unique and common challenges across industries using the latest AI and cloud technologies. We hope to inspire others looking to innovate in their work.

The demand for dental appliances, like crowns and aligners, is booming, but it’s hard for manufacturers to keep up. At Movix, we’re building one of the first agentic AI solutions for dental appliance manufacturers and dental labs to help companies in the sector acquire digital technical expertise so they can scale clinical workflows cost-effectively and consistently.

The challenge:

Movix started in 2025 with a mission to solve a serious shortage of skilled dental technicians in aligner manufacturing through AI and agentic workflows. The need is significant: the global dental market is valued at nearly $400 billion and growing at double digits, yet many operations remain analog - creating enormous demand for co-pilot, agentic solutions.

Before founding Movix, we had previously started a vertically integrated dental aligner company that focused on very difficult dental situations, such as very crooked teeth. Yet even with highly skilled and trained technicians, there were often mistakes that would require remaking an aligner — a process that costs $300, roughly 25% of the retail price. Poor quality control took a real bite out of the company’s margins.

We saw an opportunity with Movix to address these mistakes by providing technicians with AI-powered quality control agents that automate aligner workflows and reduce errors. To achieve this, we needed to solve for a few key technical challenges:

Develop a custom AI model and end-to-end agentic workflow, since off-the-shelf solutions lacked domain expertise,
Ensure scalability would be built into the platform to prevent outages or production delays,
Achieve broad interoperability through a complex hybrid integration strategy since many dental practices are slow to adopt new technology and run on legacy systems.
Optimize security and compliance to comply with medical record regulatory requirements and keep patient data safe.

The solution:

In order to deliver AI agents that can provide expert-level accuracy, we needed to custom build a lot of the tooling ourselves. We started by developing our custom models for deep learning, computer vision, and 3D mesh analysis over a five-month period, using Google Cloud infrastructure. This intensive, methodical time helped ensure the right level of accuracy and quality control.

We use Google Cloud infrastructure across the full pipeline — from dataset storage and model training to evaluation — to build and refine our defect detection models for intraoral scans. Once defects are detected, we use Gemini Enterprise Agent Platform to generate client-facing feedback that reads as if it came directly from a human technician — acting as a digital team member in the quality control workflow.

Our 3D models use Cloud Run with L4 GPUs for the massive compute power we require; notably, performing the 3D segment scans and detecting defects across the entire fabrication process are highly compute-intensive processes. We use Compute Engine VMs to run experiments, along with various other GPUs to train our models, and perform the heavy lifting of model development in this environment.

Cloud Run and other tools like Cloud Storage support our scalability goals as we target large customers who handle high case volumes — some large labs might produce up to 200,000 appliances per year. Google Cloud's global network of data centers also simplifies regulatory compliance across regions and ensures fast delivery of large 3D datasets to clients worldwide.

The architecture:

The outcome:

Our agentic solutions automate data entry and quality control, which are traditionally manual, time-consuming, and error prone tasks. By automating the work of the best dental technicians, we’re ensuring a top quality product that will improve the fit of crowns, aligners, veneers, and implants for many, many patients. We estimate that our automation and the higher level of accuracy our QC agent delivers could save an aligner manufacturer $300 per remake, for example.

We also believe we’re helping to speed the appliance manufacturing process, leading to quicker turnaround times for dental appliances, which helps dental labs receive revenue faster and improve their cash flow. And we already know we’re meeting a critical need: After we launched the QC agent in October 2025, our first customer signed with us in December. That customer, Orthero, an aligner company serving more than 20 countries, has enjoyed significant results.

“Orthero benefits from this automation by making quality control faster, more consistent, and scalable,” Efer Turhan, a co-founder of Orthero, said. “With support from Movix’s QC AI Agent, we detect missing or inconsistent inputs early and flag unusual deviations before they cause delays.”

The details:

Even with the advantages of AI, our goals demand some serious work. Our architecture supports a solution that’s agentic and modular, integrates into existing on-premises dental systems, and ensures security and compliance.

Our agentic approach allows our system to run checks and balances, manage the complex, multi-step process of quality control for dental scans, and eliminate human errors that occur in data handling and quality review. Our goal is to develop five distinct AI agents by 2029 that cover the entire dental appliance workflow, from original patient dental scan to appliance manufacturing. While our first agents focus on data entry and dental scan quality control, our next agents will handle 3D file repair, clinical review, treatment planning, and manufacturing.

Our solution architecture also enables our system to integrate seamlessly with our customers’ existing lab management and manufacturing systems through API integrations. Because we are selling our solution into a conservative market, we decided to bear the burden of responsibility for successful adoption by doing as much of the integration work as possible.

Because we operate in the highly regulated healthcare industry, we built an environment that strictly follows compliance rules, anonymizing protected health information, or PHI, before it enters our machine learning pipeline to prevent health information from being exposed to the processing environment.

We plan to build hybrid solutions to capture a wider market as we move forward. We're designing an architecture that connects our cloud-based AI agents with older, on-premises software that many conservative labs still use — through lightweight local connectors and standardized APIs. This will allow us to access a large market segment that has not yet migrated to the cloud or begun to use new digital dental technologies.

Taken together, we are not just solving a skills gap, we are reimagining what is possible with co-pilot and agentic solutions across the entire dental industry.

What’s new with Google Cloud

Fri, 22 May 2026 16:00:00 +0000

Want to know the latest from Google Cloud? Find it here in one handy location. Check back regularly for our newest updates, announcements, resources, events, learning opportunities, and more.

Tip: Not sure where to find what you’re looking for on the Google Cloud blog? Start here: Google Cloud blog 101: Full list of topics, links, and resources.

aside_block: <ListValue: []>

May 18 - May 22

Chinese Webinar | June 4: AI Command and Control
As AI agents move from experimental pilots to core enterprise functions, governance has become a critical next step. Join Google Cloud on June 4th at 10:00 AM (Beijing Time) to learn how to build a secure AI management layer architecture. We'll explore how to develop governed MCP (Model Context Protocol) endpoints, manage tool access to enterprise data, and leverage robust audit logs to operationalize AI. This session also includes a practical demonstration of these governance frameworks on Google Cloud.

Register here
GCP Announces New Features to Benchmark and Optimize LLMs for On-Device Use Cases
Deploying fine-tuned LLMs from GCP to edge devices like smartphones is complex due to fragmented hardware. Google AI Edge Portal bridges this gap, giving GCP developers the ability to test AI performance on 120+ Android devices, representing the full diversity of high, medium, and low tier smartphones on the market today. This week at I/O, we announced brand new capabilities to benchmark and debug LLM performance across these devices. Sign-up to utilize these new features in private preview today.

May 11 - May 15

Build Your AI & MCP Control Tower for Universal Governance
Master the future of agentic security with Apigee. Join our Community TechTalk on May 21 to discover how Apigee serves as a central "Control Tower" for the Model Context Protocol (MCP). We will explore how new JSON-RPC tool authorization enables fine-grained access policies across your organization, ensuring secure and scalable AI deployments. Whether managing internal tools or external users, learn to govern your agentic ecosystem with absolute precision. This session is designed for global coverage across EMEA and AMER regions.

Register for the May 21 Community TechTalk

Apr 27 - May 1

Master Your Launch: The Apigee Production Go-Live Checklist
Ensure a secure launch with the Apigee production guide. Join Nicola Cardace on May 28 to explore security guardrails, including IAM roles, mTLS configurations, and encrypted KVM migrations. Scheduled at 11 AM EDT / 5 PM CEST to support EMEA and AMER teams, this TechTalk provides the technical roadmap you need to flip the switch with absolute confidence.

Register for the May 28 Community TechTalk
Transforming APIs into Governed Agentic Tools on the Google Cloud Agentic Platform
Turn your APIs into secure, governed agentic tools on the Google Cloud Agentic Platform. Join Specialist Christophe Lalevée on May 7 for a technical deep dive into AI productization. Scheduled at 5 PM CEST / 11 AM EDT to maximize coverage for developers across EMEA and AMER, this session explores the integration and governance frameworks required to scale enterprise-ready AI with confidence.

Register for the May 7 Community TechTalk
Fractional G4 VMs are Generaly Available, providing a highly efficient and cost-effective entry point for AI and graphics workloads. These new configurations, using NVIDIA virtual GPU (vGPU) technology, allow you to leverage the power of the NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs in flexible, smaller increments, so you can right-size your infrastructure to match the specific demands of your applications. By providing more granular access to advanced hardware, fractional G4 VMs let you optimize resource allocation and reduce overhead without sacrificing performance. You can now select from additional GPU slice sizes for your specific needs:
- 1/2 GPU: Ideal for more intensive tasks such as LLM inference, robotics sensor simulation, and high-fidelity 3D rendering.
- 1/4 GPU: Optimized for mainstream workloads, including mid-range creative design, video transcoding, and real-time data visualization.
- 1/8 GPU: Great for lightweight applications such as remote desktops, productivity tools, and entry-level streaming services.
Transitioning AI from a sandbox prototype to an enterprise-grade system is a major hurdle. A monolithic script won't suffice for widespread deployment. To achieve true scale and reliability with Gemini, organizations must adopt service-oriented micro-agent architectures, establish Zero-Trust security, and implement rigorous EvalOps. Master the "Agentic Maturity Ladder" to ensure your AI & Agentic solutions are robust, secure, and ready for the real world.

Watch the deep dive and read the developer blog to learn more.
ML Development in VS Code with Google Cloud Power: Workbench Extension Now Available
Data scientists and developers can now combine the local productivity of VS Code with the scalable infrastructure of Google Cloud. The new Google Cloud Workbench Notebooks extension allows you to connect to and run notebooks on managed cloud environments directly within your local IDE. This integration streamlines the ML lifecycle by eliminating context switching and providing high-performance compute for complex workloads in a familiar interface. As part of our commitment to the developer ecosystem, the extension is fully open-sourced to support community-driven innovation.
- Install from Marketplace: GoogleCloudTools.workbench-notebooks
- Contribute on GitHub: colab-enterprise-vscode

Apr 20 - Apr 24

Announcing the 2026 Google Cloud Partners of the Year
Google Cloud is honored to celebrate the winners of the 2026 Partner of the Year awards! These awards recognize an exceptional group of partners across AI, Security, Infrastructure, and more, who have demonstrated a commitment to customer success. From global system integrators to specialized startups, these winners are leveraging the power of Google Cloud to solve complex challenges and drive digital transformation worldwide. Join us in congratulating these organizations for their innovation, collaboration, and impactful results over the past year.

See the 2026 Partner Award winners

Apr 13 - Apr 17

We're excited to announce the Public Preview of Datastream’s metadata integration with Knowledge Catalog. This is the first step in our vision to provide a centralized, "single pane of glass" for all Datastream assets. The enhancement automatically synchronizes Streams, Connection Profiles, and Private Connections, eliminating data silos. It enhances discoverability, allowing you to search for Datastream assets using the same interface as BigQuery tables. Centralized governance is also provided, making your real-time data estate more transparent and easier to manage.
Upgrading Apigee OPDK to 4.53 with OS Modernization
Modernize your infrastructure using Google’s official, sequential upgrade path. Our Technical expert, Rakesh Talanki outlines how to upgrade Apigee OPDK to v4.53 while migrating to a supported OS (RHEL 8.x/9.x). This guide covers the "build-out" methodology, including multi-data center syncing, to ensure a stable, zero-downtime transition

Read the guide
Cloud Run Worker Pools and CREMA: Powering Serverless AI at Scale
Google Cloud has announced the General Availability of Cloud Run worker pools, a new resource type designed specifically for pull-based, non-HTTP workloads. Unlike traditional Cloud Run services that scale based on request traffic, worker pools provide an "always-on" environment for background tasks like processing message queues or running large-scale AI inference. To support this, Google Cloud also open-sourced the Cloud Run External Metrics Autoscaler (CREMA). Built on KEDA, CREMA enables queue-aware autoscaling for worker pools, allowing them to dynamically scale based on external signals like Pub/Sub backlog or Kafka lag.
Apigee Model Context Protocol (MCP) now Generally Available
Expose enterprise APIs as MCP tools for agentic AI applications with the General Availability of MCP in Apigee. This update allows developers to transform APIs into AI-ready tools using OpenAPI Specifications, removing the need for local MCP servers or additional infrastructure. With managed endpoints and semantic search in API hub, you can now provide AI agents with secure, governed access to enterprise data at scale.

Explore the MCP overview

Apr 6 - Apr 10

Community TechTalk: Powering Retail Agents with ADK, UCP & Apigee X
Move beyond basic chatbots to secure, transactional AI experiences. Join our Community TechTalk on April 16 to learn how Apigee X and Gemini build a "Trust Layer" for AI shopping assistants using UCP standards. We’ll demonstrate how to block prompt injections with Model Armor and implement cost governance via token limits to secure the path from discovery to purchase.

Register for the TechTalk
Implement multimodal capabilities in your AI agents
Explore three new reference architectures for building sophisticated multi-agent AI systems that can process and analyze multimodal data. To analyze disparate multimodal data and produce a high-confidence classification, see Classify multimodal data. To create a fluid conversational AI that processes audio and video streams in real time, see Enable live bidirectional multimodal streaming. To consolidate fragmented multimodal data into a searchable knowledge graph, see Multimodal GraphRAG resource orchestration.
Automate SecOps workflows with an agentic AI system
To accelerate incident response and reduce manual toil for your security team, you need a system that can automate remediation playbooks. Our new reference architecture helps you build an AI agent that orchestrates complex triage and investigation workflows across disparate security tools, such as SIEM, CSPM, and EDR, from a single interface. See the full guide to orchestrate security operations workflows.

Mar 30 - Apr 3

ASEAN Webinar | April 30: Mastering Agentic Governance at Scale with GCP
As AI agents move from experimental pilots to core enterprise functions, governance is the critical next step. Join Google Cloud experts Shilpi Puri & Wely Lau for a webinar on April 30th at 11:00 AM SGT to learn how to architect a secure AI Management layer. We’ll explore developing governed MCP endpoints, managing tool access to enterprise data, and operationalizing AI with robust audit logs. The session includes a live demo of these frameworks in action on Google Cloud.

RSVP here.

Mar 23 - Mar 27

Turn your API sprawl into an agent-ready catalog
As organizations scale, APIs often become scattered across multiple gateways, creating "blind spots" that hinder AI adoption. To solve this, we’ve introduced two new capabilities for Apigee API hub: a new integration with API Gateway to automatically centralize API metadata into a single control plane, and a specification boost add-on (now in public preview). This add-on uses AI to enhance your API documentation with the precise examples and error codes that AI agents need to function reliably.

Read the full blog post to get started.
Webinar | April 16: AI Command & Control
As AI agents move from experimental pilots to core enterprise functions, governance is the critical next step. Join Google Cloud expert Satyam Maloo for a webinar on April 16th at 11:00 AM IST to learn how to architect a secure AI Management layer. We’ll explore developing governed MCP endpoints, managing tool access to enterprise data, and operationalizing AI with robust audit logs. The session includes a live demo of these frameworks in action on Google Cloud.

RSVP here.
Modernizing and Decoupling Event Ingestion with Apigee
In modern cloud-native architectures, decoupling producers from consumers is critical for building resilient systems. While Google Cloud Pub/Sub provides a scalable backbone, exposing it directly to external clients can introduce security and management overhead. This new guide explores how to leverage Apigee as an intelligent HTTP ingestion point. Learn how to handle security, mediation, and traffic control before messages reach your internal bus using the PublishMessage policy or Pub/Sub API.

Read the full guide.

Mar 16 - Mar 20

Gemini-powered Assistant in BigQuery Studio Gets Context-Aware Upgrades
The Gemini-powered assistant in BigQuery Studio has been transformed into a fully context-aware analytics partner, supporting your entire data lifecycle. The new capabilities include intelligent resource discovery, which uses Dataplex Universal Catalog search to find resources across projects and deep dive into metadata using natural language. You can now automate tasks, such as scheduling production-grade queries directly through the chat interface, and instantly troubleshoot long-running or failed jobs with root cause analysis and cost control auditing.

Explore the full range of what the assistant can do.

Mar 9 - Mar 13

Want to use Gemini to develop code and don't know where to start?
This article includes a couple of examples of developing code with Gemini prompts; it identified changes that were needed to be made to get the code working. The article also refers to other examples that are available on github.

Mar 2 - Mar 6

Introducing Gemini 3.1 Flash-Lite, our fastest and most cost-efficient Gemini 3 series model. Built for high-volume developer workloads at scale, 3.1 Flash-Lite delivers high quality for its price and model tier. Gemini 3.1 Flash-Lite can tackle tasks at scale, like high-volume translation and content moderation, where cost is a priority. And it can also handle more complex workloads where more in-depth reasoning is needed, like generating user interfaces and dashboards, creating simulations or following instructions.

Starting today, 3.1 Flash-Lite is rolling out in preview to enterprises via Vertex AI and developers via the Gemini API in Google AI Studio.
TechTalk: Implementing Device Authorization Grant (RFC 8628) for Apigee
Learn how to authorize "headless" devices like Smart TVs or AI agents that lack keyboards and browsers. Join our Community TechTalk on March 19 (5PM CET / 12PM EDT) to go under the hood of Apigee X/Hybrid. We’ll cover the real-world mechanics of state management, polling, and human-in-the-loop security patterns for devices and autonomous agents.

Register for the TechTalk

Feb 23 - Feb 27

Pro-level image generation gets faster and more accessible with Nano Banana 2
Nano Banana 2 is our state-of-the-art image generation and editing model. It delivers Pro-level image generation and editing at the speed you expect from Flash — making the quality, reasoning, and world knowledge you loved about Nano Banana Pro more accessible. Learn more about the model here.

The Intelligent Path to Compliance: Transforming Regulatory QC with Google Cloud
Reducing "Refuse to File" (RTF) risks and submission cycle times is critical for life sciences leaders. Google Cloud’s Regulatory Submission Semantic QC Auditor leverages Gemini and RAG architecture to transform Quality Control from a manual burden into an active, intelligent workflow.

By automating semantic cross-referencing, narrative coherence checks, and dynamic guidance-based auditing, this solution ensures rigorous accuracy and auditability. Operating within a secure GxP-ready environment, it empowers teams to detect subtle inconsistencies and generate remediation plans without sacrificing data privacy.

Learn more.
Stop typing, start interacting! The Gemini Live Agent Challenge is here. Build immersive agents that can help you see, hear, and speak using Gemini and Google Cloud. Compete for your share of $80,000+ in prizes and a trip to Google Cloud Next '26!

Submissions are open from February 16, 2026 to March 16, 2026. Learn more and register at geminiliveagentchallenge.devpost.com

Feb 9 - Feb 13

Introducing Gemini 3.1 Pro on Google Cloud.
3.1 Pro is a noticeably smarter, more capable baseline for complex problem-solving. We’re shipping 3.1 Pro at scale, building upon our goal to help you transform your business for the agentic future. Learn more about the model’s capabilities here. Gemini 3.1 Pro is available starting today in preview in Vertex AI and Gemini Enterprise. Developers can access the model in preview via the Gemini API in Google AI Studio, Android Studio, Google Antigravity, and Gemini CLI.
Automate Storage Compatibility with GKE Dynamic Default Storage Classes
Managing storage across mixed-generation VM clusters in GKE just got easier. With the new Dynamic Default Storage Class, Google Kubernetes Engine automatically selects between Persistent Disk (PD) and Hyperdisk based on a node's specific hardware compatibility. This abstraction eliminates the need for complex scheduling rules and manual pairing, ensuring your volumes "just work" regardless of the underlying infrastructure. By defining both variants in a single class, you reduce operational overhead while maintaining peak performance and cost-efficiency across your entire cluster.

Explore automated disk type selection
Community TechTalk: AI-Powered Apigee Development with strofa.io
Join the Apigee community on February 26 for a deep dive into strofa.io. Guest speaker Denis Kalitviansky will demonstrate how this new AI-powered tool automates and orchestrates Apigee development, from local emulators to large-scale hybrid environments. Discover how to scale your API management and streamline team collaboration using the latest in AI-driven automation.

Register now to reserve your spot.

Jan 26 - Jan 30

Simplify API Governance with Native OpenAPI v3 Support
Eliminate integration debt and accelerate deployment velocity with the General Availability of OpenAPI v3 (OASv3) support for API Gateway and Cloud Endpoints. You no longer need to downgrade modern specifications to OASv2. Instead, you can now define API contracts and enforce critical policies—including telemetry, quotas, and security—using native Google-specific extensions directly within your OASv3 files. This update ensures your APIs are secure by design while remaining fully compatible with the modern developer ecosystem and Google Cloud’s AI services.

Get started with OpenAPI v3 on API Gateway and Cloud Endpoints.

Accelerate API Testing with the New Open Source API Tester
Start validating your APIs with API Tester, a simple, YAML-based Test Driven Development (TDD) framework. Designed for the Apigee community, this tool allows you to write human-readable tests, run them instantly via a web client or CLI, and perform deep unit testing on Apigee proxies. With native support for JSONPath assertions and Apigee shared flows, you can verify everything from payload data to internal variables like proxy.basepath without leaving your terminal.

Explore the API Tester guide and start testing your proxies today.
Secure Sensitive Data with Kubernetes Secrets in Apigee hybrid
Enhance security in Apigee hybrid by accessing Kubernetes Secrets directly within your API proxies. This hybrid-exclusive feature keeps sensitive credentials within your cluster boundary and prevents replication to the management plane. It supports strict separation of duties: operators manage secrets via kubectl, while developers reference them as secure flow variables—ideal for high-compliance and GitOps workflows.

Implement Kubernetes Secrets in your hybrid proxies.
See the Console in a Whole New Light: Dark Mode is Now Generally Available in Google Cloud
Elevate your cloud management workflow with Dark Mode, now generally available in the Google Cloud console. We have delivered a modern, cohesive, and accessible experience reimagined for maximum comfort and productivity—especially during extended working hours and low-light environments. Dark Mode can be enabled automatically based on your operating system's preference, or manually through the Settings -> Appearance menu.

Switch to Dark Mode today to enjoy a modern, comfortable, and productive environment!
Apigee X Networking: PSC or VPC Peering?
Deciding how to connect Apigee X? Watch this video to compare Private Service Connect and VPC Peering. We break down northbound and southbound routing, IP consumption, and how to reach targets on-prem or in the cloud. Learn to simplify your architecture and avoid common networking "gotchas" for a smoother deployment.

Watch the video.

Jan 19 - Jan 23

Bridge the Gap: Excel-to-API Conversion in Apigee Portals
Give your customers more ways to connect! This new article by Tyler Ayers explores how to extend the Apigee Integrated Portal to support direct Excel file uploads. By leveraging SheetJS and custom portal scripts, you can enable users to upload spreadsheets, preview data, and submit it directly to your APIs, all without writing a single line of integration code themselves. It’s a powerful way to simplify onboarding for those who aren't yet API-ready.

Learn how to build it.
Elevate your applications with Firestore’s new advanced query engine
We have fundamentally reimagined Firestore with pipeline operations for Enterprise edition. Experience a powerful new engine featuring over a hundred new query features, index-less queries, new index types, and observability tooling to improve query performance. Seamlessly migrate using built-in tools and leverage Firestore’s existing differentiated serverless foundation, virtually unlimited scale, and industry-leading SLA. Join a community of 600K developers to craft expressive applications that maximize the benefits of rich queryability, real-time listen queries, robust offline caching, and cutting-edge AI-assistive coding integrations.

Learn more about Firestore pipeline operations.

How Glance turns hours of video into mobile-ready clips with AI

Thu, 21 May 2026 17:00:00 +0000

Every day, thousands of hours of new video content sits waiting to be discovered. Most of it lives in long-form, horizontal formats, while audiences are scrolling through vertical feeds on their phones.

Glance, a mobile-first content platform, knows this challenge well. The company processes 1-2 hour videos from sources like podcasts, news reports, movies, and web series, and transforms them into 30 to 180-second vertical clips optimized for mobile lock screens. With daily volume projected to grow from 3,500 to over 10,000 videos per day, manual editing wasn’t a realistic path forward.

The solution also needed to go beyond simple cropping. It required the intelligence to identify and center the primary speaker, or dynamically split the screen to stack speakers vertically during conversations, preserving the context that makes content worth watching.

Here’s how Glance’s video generation solution works.

Building for the lock screen era

The goal was to create a complete pipeline that takes a long-form landscape video (16:9) and outputs multiple ready-to-publish short-form portrait videos (9:16). The solution needed to handle:

Key Moment Identification: Finding the most engaging 60-second segments within hours of long-form footage
Active Speaker Detection: Identifying who’s talking in each frame and positioning them at the top of a split screen. This includes distinguishing between a static image and a live person to ensure the crop focuses on the actual speaker.
Split Screen Detection: Recognizing interview layouts (common in news broadcasts) and stacking the frames vertically to preserve conversation context
Intelligent Reframing: Converting a multi-speaker, wide-screen shot into a focused, vertical frame without losing context
Dynamic Caption Highlighting: Generating word-level timestamps for "Karaoke-style" captions that increase engagement on silent-by-default mobile screens
Automated Branding: Applying masks, logos, and overlays programmatically to maintain brand consistency across all videos

The final technical solution uses Google Cloud Speech-to-Text v2, Gemini, and the Google Vision API, combined with custom video manipulation using Samurai (an open-source object tracking tool), OpenCV and MoviePy.

Architecture overview

The pipeline is divided into three distinct modules.

Fig. 2: High-level architecture

Module 1: Video clipping

This module converts long videos to transcripts, identifies key segments, and clips the video. Accuracy matters here: precise word-level timestamps ensure clips start and end exactly where they should.

Fig. 3: Video Clipping Workflow

The process involves audio extraction, speech-to-text transcription, and timestamp identification using generative AI. The module performs the following key functions:

Audio extraction: Extracting the audio from the original video file.
Speech-to-text transcription: Converting audio into text with precise timestamps for each word
Segment identification: Using Gemini 2.5 Flash (aka Nano Banana) to analyze transcripts text and identify optimal start and end timestamps for short video clips
Video clipping: Clipping the video into short segments based on the identified timestamps
Transcript validation: Using Gemini to verify phrases and words are accurately captured (this step does not validate word timing)

The output is a set of short video clips, each paired with its time-aligned transcript, ready for the next stage: the Intelligent Reframing Engine.

Module 2: Intelligent Reframing Engine

The core technical work here is converting a horizontal 16:9 frame into a compelling 9:16 vertical frame. A simple center crop often cuts out key speakers or action, so our solution uses a multi-stage scene analysis pipeline.

Fig. 4: Intelligent reframing engine

Active speaker detection

To know what to crop, we first need to know who’s talking. This happens on a frame-by-frame basis using the face detection capabilities of the Google Cloud Vision API.

Fig. 5: Active speaker detection

The liveness check: Differentiating a live speaker from a static image (like a photo on the wall or a graphic) is essential. This was achieved by tracking facial landmarks:

Mouth movement: Calculating the normalized distance between upper and lower lip landmarks
Head movement: Tracking changes in head pose angles (pan, roll, tilt)
A face must show consistent animation in these cues to be classified as a "live" participant

Quantifying engagement: Once confirmed as live, we calculate an activity score based on:

Mouth openness
Emotional fluctuation (changes in joy, surprise, etc., provided by Vision API)

Primary speaker identification: The final decision uses a liveness ratio:animated frames divided by total frames where the face appears. The person with the ratio closest to 1.0 (meaning they were consistently animated on screen) is designated as the primary speaker.

One edge case addressed during the development was a static background image appearing behind a live news anchor (as shown in Fig. 6). The liveness check handles this correctly because the static image shows no facial animation.

Fig. 6: Scenario with one active speaker and one static background image

Split-screen detection

This step addresses interview scenarios where two subjects appear on opposite sides of the landscape frame. The system detects split-screen layouts and stacks the two halves vertically to maintain conversation context.

Fig. 7: Video reformatting

With active speaker detection complete, the system uses the primary speaker's location to identify split-screen segments. The goal is to find the precise dividing line between panels, enabling the video to be reformatted into a vertical, top-and-bottom layout. Two complementary approaches accomplish this:

Approach 1: Continuous face tracking with Samurai

This method uses Samurai, an open-source object tracking tool, to follow the primary speaker continuously. The trajectory is analyzed for split-screen layouts based on:

Consistent off-center positioning: The speaker remains on one side of the screen (e.g., left or right half), indicating a split panel rather than free movement across the frame.
Vertical dividing line detection: Image analysis identifies a persistent vertical line separating the two panels.
Background discontinuity analysis: Differences in color, texture, and scenery between the speaker’s background and the opposite side confirm two separate video feeds.

Fig. 8: Background discontinuity analysis

Approach 2: Frame-by-frame detection with Google Cloud Vision API

This approach uses Cloud Vision API's face detection to identify split-screen layouts based on the primary speaker's face location:

Off-center face: Consistent face detection in one region (such as the left 40% of the frame) flags a potential split screen.
Proximate dividing line: Vertical lines between the face and the screen center confirm a panel boundary.
Contrasting backgrounds: Inconsistent backgrounds between the speaker's side and the far side confirm the split-screen layout.

The output: Vertical stacking

Once the system recognizes a split-screen, it performs a digital cut-and-paste. This preserves both speakers and their reactions in a mobile-native format.

Automated reformatting

With the scene analysis complete, the OpenCV2-based solution intelligently applies the appropriate reframing rule to each segment:

Single speaker crop: For scenes with one primary speaker, the system anchors the 9:16 frame to the speaker’s face, keeping them centered.
Split screen: When a split is detected, the system slices the frame along the dividing line and stacks the panels vertically (left panel on top, right panel on bottom).
Multi-speaker crop: For scenes with multiple people (not a formal split), the system focuses the crop on the most prominent speaker or the face closest to the center.
Fallback: If no faces are detected (e.g., graphics or wide shots), the system applies a center crop or horizontal padding (letterboxing).

Two final techniques ensure a polished look:

Short scene merging: Segments shorter than a defined threshold merge with the preceding or following scene, eliminating flicker.
Camera smoothing: When focus shifts between speakers, a virtual camera effect creates a slow pan from one position to the next, rather than an abrupt cut.

Module 3: Finishing and branding

The final stage ensures the clips are ready for immediate publication, focusing on viewer engagement and brand reinforcement.

Dynamic caption highlighting

Using the word-level timestamps from the speech-to-text module, the system overlays highlighted captions with MoviePy. This involves:

Fig. 9: Dynamic caption highlighting

Sentence reconstruction: Grouping individual words into readable lines that adhere to character limits
Highlighting: The currently spoken word is highlighted in a distinct color (mustard yellow) against a black background, a proven method for increasing engagement when videos play without sound.

Masking and logo placement

Two overlay techniques maintain consistent branding across all videos:

Mask placement: A PNG mask with an alpha channel resizes the video to fit precisely into the transparent area. The mask's opaque regions (such as colored bars) serve as a dedicated background for captions and persistent graphics.
Logo overlay: The brand logo is placed onto the video based on configurable parameters for position (top-right, bottom-left, and so on), size, and margin.

Fig. 10: Mask and logo placement

Conclusion

Glance’s video pipeline demonstrates what becomes possible when AI handles the repetitive, judgement-intensive work of video editing. By combining speech-to-text transcription, computer vision, and generative AI, the system transforms thousands of long-form videos into mobile-ready clips each day, preserving narrative context while optimizing for vertical viewing.

The approach offers a template for any organization sitting on long-form video archives. Rather than choosing between scale and quality, automated pipelines can deliver both.

If you’re exploring similar video processing, content transformation, or media AI projects, the Google Cloud consulting team is eager to connect and explore the possibilities. For more on the AI products used in solutions Glance’s this, visit our AI & ML Products page.

_{This solution was a collaborative effort between Glance ( Pradeep Tiwari , Himanshu Aggarwal) and Google Cloud Consulting (Sharmila Devi, Jinyeong Yim, Rohit Sroch, Neeraj Shivhare and Kinjal Singh).}

The top announcements for startups from Google I/O ‘26

Thu, 21 May 2026 16:00:00 +0000

Many of the world’s fastest-growing AI startups are choosing to build their future — and the world’s — on Google Cloud because of our complete and open AI stack. We embed AI into every layer of our architecture so your team is fully equipped to build and scale for the agentic era.

At Google Cloud Next ‘26, we focused on providing updates to each layer of the stack to help lean teams move faster and operate more efficiently. We introduced a unified Agent Platform that moves beyond isolated AI tools to a complete lifecycle platform and a new generation of TPUs optimized for both training and inference. This was complemented by the world’s first Agentic Data Cloud, a system designed to take on the shift from human scale to agent scale and updates to our AI-powered cybersecurity platform, now combining Google’s Threat Intelligence and Security Operations with Wiz’s Cloud and AI Security Platform to prevent, detect, and respond to threats in the agentic era.

A diverse set of global startups are already putting this stack to work. For example, Photoroom relies on our highly optimized compute to process over 1,000 images per minute. In the enterprise customer engagement space, Satisfi Labs leverages Gemini, BigQuery, and AlloyDB to deliver intelligent interactions and real-time insights to 800 sports, entertainment, and tourism clients. Meanwhile, global collaboration platform Notion is at the forefront of integrating artificial intelligence into productivity tools, and Mantis AI operates high-throughput inference pipelines to deliver video-native intelligence to major broadcasters.

Now, at Google I/O ‘26, we are expanding on that foundation by introducing more cost-effective frontier models and bringing these agentic capabilities directly into your development workflow. We are bridging the gap between local prototyping and cloud scale, giving your team a straightforward path to build, test, and distribute your applications.

Let’s look at the key announcements for startups from Google I/O and how you can apply them to your business.

1. Smarter, faster models

Models continue to be the foundation of what many startups are building with Google Cloud, so we’re excited to have evolved them once again. This new generation of models deliver maximum intelligence with incredible efficiency.

Gemini 3.5 Flash delivers intelligence that rivals large flagship models at Flash speed. It’s our strongest agentic and coding model yet, ideal for tackling long-horizon agentic tasks, often at less than half the cost of comparable models.
Gemini 3.5 Pro (pre-announcement): We announced our flagship reasoning model officially rolls out next month.
Gemini Omni is a groundbreaking new model that produces dynamic video content by blending text, audio, image, and video inputs. Gemini Omni delivers a highly intuitive approach to video creation and editing — whether you are developing interactive virtual try-ons for e-commerce, streamlining complex post-production workflows, or generating tailored video narratives, Gemini Omni unlocks new ways to create content and drive deeper customer engagement.

What it means for startups: You now have direct access to build with new state-of-the-art models from DeepMind. We are continuing to push the boundaries of what AI can do, ensuring that as we pioneer new frontiers, we simultaneously deliver the speed and cost-efficiencies your startup needs to scale.

Landing in the top-right quadrant of the Artificial Analysis index, 3.5 Flash delivers frontier-level intelligence at exceptional speed — proving you no longer have to trade quality for latency.

2. AI that works for you

At Next ‘26, we introduced purpose-built agents to handle the heavy lifting at every layer of the stack — from threat-hunting SecOps agents and modular data agents, to self-driving cloud infrastructure that autonomously fixes misconfigurations.

But that massive, specialized agentic power needs a coordinator. Enter Google Antigravity — the ultimate control plane. These new updates transform how applications are built, deployed, and managed.

Antigravity 2.0: A dedicated, standalone desktop application for Mac, Windows, and Linux. This acts as an "agent-first" workspace to build, test, and orchestrate complex AI workflows without being tied down to a traditional code editor (IDE).
Antigravity CLI & SDK: For developers who want to stay at the keyboard, the blazing-fast and stack-agnostic command-line interface lets you run and monitor agents seamlessly. Meanwhile, the Python SDK opens up Google's internal agent infrastructure, allowing you to code and control stable agent loops programmatically.
Dynamic subagents: This allows a primary AI agent to automatically spawn smaller, specialized child agents to handle focused subtasks. It unlocks massive parallel engineering output — your main local agent can delegate a database query to a Cloud Data Agent, for example, or spin up a local code-review subagent, all without cluttering its main memory space.
Scheduled tasks: Remove manual maintenance chores by setting background timers and cron schedules. Startups can instruct Antigravity to trigger your cloud-based observability agents or run local repository sanity checks every night at midnight — completely autonomously.
Enterprise-grade security: Antigravity connects local desktop and terminal agent loops directly to your private Google Cloud projects. By inheriting Google Cloud’s standard data privacy protections and Terms of Service, this ensures your customer data is in your control and agent activity runs within your secure cloud boundary by default.

What it means for startups: You no longer just have a coding assistant; you have an entire fleet of specialized AI engineers at your fingertips. By combining the purpose-built cloud agents launched at Next with the Antigravity control plane, a lean team can orchestrate data pipelines, manage security, and execute massive parallel coding tasks — all from a single, secure environment.

Antigravity 2.0 can deploy simultaneous, agent-driven execution. This example shows automated code generation for your website, creation of on-brand assets, and personalized customer email development. Sequences shortened throughout

3. More ways to accelerate development

We are streamlining the developer workflow to help you move from a prompt to a production-ready application with far less friction. This year’s updates focus on removing the infrastructure setup that traditionally slows teams down.

Native Android support in AI Studio: You can now go directly from a natural language prompt to a fully native Android app within the browser. This includes support for the Google Play Console, allowing developers to publish apps directly to the test track without managing local SDK environments.
The seamless handoff: Through a new integration, you can export entire projects from Google AI Studio directly to your local Antigravity environment with a single click. This transfers your complete codebase, files, and conversation context so you can transition from web prototyping to local development without losing your place.
Managed agents: For lean startup teams, building a production-grade agent shouldn't require managing complex infrastructure. Available across both the Gemini API and Google Cloud's Agent Platform, the new Managed Agents API acts as an agent-as-a-service so you can "manage the mission, not the machine." Simply define your instructions and tools, and a single API call will spin up your agent within a secure, ephemeral Google Cloud sandbox. This allows your team to offload the heavy lifting of backend maintenance and focus entirely on building great agentic experiences

What it means for startups: We are providing a straightforward path from prototype to production. You can quickly test app concepts in the browser, move them to a local workspace for deep orchestration, and deploy user-facing agents using managed cloud infrastructure — saving your engineering team weeks of setup and maintenance.

4. A boost for your personal productivity

While your engineers are accelerating product development, we also want to help founders and operators manage the daily noise of running a company.

To help with this, we are introducing Gemini Spark — a new 24/7 personal AI agent that works in the background across Google Workspace and other daily tools. Instead of just answering questions, it can autonomously execute multi-step workflows on your behalf.

For example, Spark can identify a critical product delay, cross-reference your team's documents to recalculate the timeline, update internal tracking sheets, and draft an update email to your investors — all while waiting for your explicit approval before executing.

What it means for startups: While Antigravity builds your product, Spark acts as your digital chief of staff. It handles the routine, manual operational processes so you can stay focused on high-impact, strategic innovation.

Start building today: The Google for Startups AI Agents Challenge

Open globally to eligible startup founders and developers, this competition equips your team with $500 in cloud credits and access to our new Agent Platform, so you can build autonomous agents and compete for a share of a $90,000 prize pool.

We’re offering separate tracks, whether you want to build a net-new agent from scratch, optimize an existing prototype for production, or prep a business-ready agent for enterprise distribution, there is a track tailored to your exact stage. Submissions are open until June 5, 2026, and will be evaluated on technical implementation, business case, innovation, and your final demo. Learn more and sign up for the challenge here.

AI Studio unlocks full-stack vibe coding with Cloud Run, Firebase, and Cloud SQL, no credit card required

Thu, 21 May 2026 16:00:00 +0000

At Google I/O 2026, we announced updates to the integration between Google AI Studio and Google Cloud:

New users can deploy up to two full-stack applications to the Google Cloud Starter Tier, no billing account required
An expanded choice of databases: Firestore for non-relational data, and Cloud SQL as a new relational database option
Tight integration with Google Workspace tools like Sheets, Calendar, and Gmail using Firebase Auth as the single user login flow

This is an update to the integration we announced in March, which included support for vibe-coded full-stack app deployments from AI Studio powered by Cloud Run, Firestore, and Firebase Auth.

With this expanded integration, you can use AI Studio to build a broader set of applications, using either a relational database with Cloud SQL or a non-relational database with Firestore. You don’t even need to specify a database — the AI agent can infer the right database for your app or feature.

Get started today in AI Studio at no cost with Cloud Run, Cloud SQL for PostgreSQL (coming next month), Firestore, and Firebase Auth for Starter Tier.

Publishing a full-stack app from AI Studio to Cloud Run with a single click

An easy on-ramp: The Google Cloud Starter Tier

You can build applications in AI Studio and deploy your prototypes directly to Cloud Run, authenticate via Firebase Auth, and store your data in a Firestore or Cloud SQL database. No credit card, no Google Cloud account, no friction — just prompt and launch.

If you don’t have an account, AI Studio uses the Google Cloud Starter Tier to create resources for you. You can deploy up to two full-stack apps. If you outgrow the limits of the Starter Tier, you can upgrade to a standard Google Cloud project with a billing account. All your resources will be transferred to your billable Google Cloud project, so that your application can scale as it grows.

Powering full-stack vibe coding with Cloud SQL

We’re introducing an intelligent, automated data foundation that makes it easy for developers to focus on their applications, not their infrastructure.

AI Studio integration with Cloud SQL includes:

An instant on-ramp: Go from prompt to a fully-deployed PostgreSQL database rapidly with instant provisioning.
Zero-cost startup: Try Cloud SQL for the Google Cloud Starter Tier at no cost, without needing a credit card or Google Cloud account.
Flexible cost control: The AI agent uses a new Cloud SQL for PostgreSQL developer edition, which enables the backend to scale to zero automatically, so you only pay while you’re using the app.
Agent-driven experience: To update your application, enter new prompts and the AI Agent automatically creates the schema and executes SQL statements in the database.
Global scalability: While the interface is simple, your app runs on Google Cloud’s robust, highly-reliable, and securely designed infrastructure that can scale to support millions of users.

Creating an app powered by Cloud SQL for PostgreSQL developer edition

Full-stack vibe coding with Firestore and Firebase Auth

When you’re building an app in AI Studio, the agent proactively detects if you need data storage and authentication based on your prompt, and offers to set up a database and user authentication. For apps that benefit from a document database, the agent shows a card to turn on Firestore and Firebase Authentication with your approval.

Enable Firebase for your application when prompted by the agent

By clicking “Enable Firebase,” the agent automatically:

Provisions Firestore, enables authentication, and connects your app to the database
Creates your web app’s sign-in page and configures authentication with Google Sign In
Generates the Firestore code in your app so you can sync data across sessions and devices
Drafts and deploys Firestore Security Rules based on your app’s logic (but you should always double-check these rules before sharing or deploying your app!)

With Firebase Auth, you can:

Connect your apps to Google Workspace using natural language: When you ask for a feature involving Workspace (e.g. Sheets, Calendar, Gmail), the agent implements a “Sign in with Google” flow, powered by Firebase Authentication, designed to securely grant Google AI Studio access to your data.

Connect your app to Google Sheets, powered by Firebase Authentication

Check out more details on the What’s New from Firebase at Google I/O blog.

Getting started in AI Studio

Going from idea to app is now a reality. You can build a full-stack application at no cost using the following steps:

Log into AI Studio: Access the platform to begin your project.
Build with prompts: Start building your application using natural language prompts. For example, “Build an expense tracker app.”
Enable the database: Prompt “Add a database” and AI Studio intelligently provisions a database through an "Enable" widget. You can explicitly ask for a relational database if you’d like to make your preference clear.
Set up the system: Select “Enable” and agree to the terms.
Start sharing: Deploy and share the application through the “Publish” button.

Get started today in AI Studio to turn your ideas into live applications in seconds.

Shipping features to production just got easier with new feature flags in AppLifecycle Manager

Thu, 21 May 2026 16:00:00 +0000

Many development teams are familiar with the hesitation that comes right before pushing a new feature live. As AI helps developers write code faster, the gap between rapid code generation and safe production deployment continues to grow.

Feature flags offer a practical way to manage this risk by separating the act of deploying code from the act of releasing a feature to users. Instead of a single, high-risk launch event that affects all users simultaneously, teams can ship code to production with new features hidden by default in a controlled manner.

To help teams adopt this workflow, we are announcing the public preview of AppLifecycle Manager Feature Flags (ALM FF). This service provides a rule-based solution to manage software behavior across Google Cloud, helping you support rapid development without sacrificing production stability.

Read on to learn four ways these feature flags will help accelerate your deployment.

1. Decouple for safety and velocity

The core mission of ALM FF is to increase development velocity by decoupling your feature releases from your code deployments. Traditionally, releasing a feature requires a binary deployment — a high-risk event that affected all users simultaneously.

With ALM FF, you can ship code to production with new features disabled by default. This allows your team to move faster, deploying code continuously while choosing the exact moment to enable a feature via a toggle. If an issue is detected, the flag acts as an instant kill switch, disabling the problematic feature immediately without the need for a full, time-consuming code rollback.

2. Gradual enablement with precise targeting

Safety is about precision. ALM FF leverages the Common Expression Language (CEL) to implement sophisticated logic for gradual feature enablement.

Percentage feature enablement: Instead of a global launch, you can ramp up a feature to 1%, 5%, or 50% of your traffic. This allows you to monitor system health and performance metrics incrementally, ensuring stability before reaching your entire user base.
Precise allowlisting: You can target specific internal teams, beta testers, or early-access customers by allowlisting their identifiers. This ensures that only the intended audience sees a feature during its initial validation phase.

3. Dynamic configuration for the AI era

Beyond simple toggles, ALM FF offers a dynamic way to inject configuration into your applications. By using string-type flags, you can update application behavior — such as system prompts for LLM integrations—in real-time. This allows product managers and business owners to tweak AI responses and application logic without requiring any code changes or infrastructure rollouts.

4. Built on open standards

We believe safety should not mean lock-in. ALM FF is built on the OpenFeature standard, utilizing industry-standard SDKs and the flagd evaluation engine. This ensures your feature management patterns are portable and follow best practices without adding Google-specific dependencies to your core application code.

Get started

ALM FF is now in public preview. To take control of your releases, you can:

Review the docs: Public Documentation
Onboard today: Quickstart Guide
Give us feedback: Help us shape the future of feature management

Securing Your Gemini and Google API Keys

Thu, 21 May 2026 10:19:00 +0000

Today, AI services rely heavily on API keys. To run AI agents, users provide API keys that signify paid tokens, subscriptions, or paid accounts. While API keys are easy to use, it is just as easy to use them unsafely. The result of a hijacked key is a compromised environment that is misused or abused by perpetrators.

I decided to write this blog post after seeing a thread in the r/googlecloud subreddit asking for a tutorial so users can go and protect themselves. In this post, you will find a few simple steps you can take to reduce your risks and improve the security of API keys created by Google.

You use Google API keys to access Gemini and other AI Google products as well as Google Cloud APIs. In fact, a Gemini API key is actually a standard Google API key behind the scenes. While I will be focusing on Google API key security, you can apply some of these recommendations to API keys and product tokens created elsewhere.

Step 1: Generate a New API Key

Regardless of where you start, you end up creating a new API key in one of Google Cloud projects. You probably will use Credentials under the "APIs & Services" menu in the Cloud console.

Or you may use gcloud services api-keys create command instead. Or there is some other interface which will create a new Google Cloud API key. Regardless of the path and the interface, you need to do the following:

Create the key in a stand alone project that is not used for any other purpose.
Restrict API access and client applications for the new API key.

These steps limit the potential reach of the key and greatly simplify troubleshooting activities when something goes wrong.

API Restrictions

API restrictions specify what services you can access using the key. DO NOT create unrestricted keys, as a stolen key would grant an attacker access to any available service at your expense.

ALWAYS limit the list of the services this key is used for to reduce the potential damage (a.k.a. blast radius) in event the key is hijacked or exposed. Be attentive when you use indirect UI to create a new key. For example, creating an API key in Firebase restricts the use to 24 APIs including Datastore, Firestore, Cloud SQL Admin and others.

If you use Firebase to store your website you probably will not use most of them. When you create an API key to use with AI Studio, restrict it to only "Gemini API".

Attention points:

By default a new API key is created without restriction.
If you search for an API that you want to select but it is missing, this API is probably not enabled in the Google Cloud project that you use. Go to the API Library in your Cloud console, find the API by name and enable it first.
You can do all actions using the Cloud console or gcloud CLI. Other interfaces (e.g. Firebase) may not provide you with access to all parameters of the API keys

Application Restrictions

Similar to API restrictions that limit what services your key can be used for, Application Restrictions limit the applications which can use the key. For example, if you create an API key only for use with Google AI Studio, setting up the application restrictions to the website "https://aistudio.google.com/" will prevent using your key by automations that utilize Gemini and consume a high volume of tokens at scale.

You can set up one or more restrictions of one of the following types:

Website/Web application using the list of URLs
Services using the list of IPv4 or IPv6 address or a subnet masks
iOS applications using the list of Bundle IDs
Android applications using the list of pairs of the package name and certificate fingerprint

Note that you can restrict the key to a single application type only. Create a designated API key for each application type. Having a key per application type helps when observing the key usage and investigating potentially compromised keys.

Step 2: Store API key

I want to reiterate that the API key is not paired with your identity. ANYONE can use it. So, storing the key securely is as important as restricting the key use in Step 1.

The rule is simple: NEVER EVER store the key where it can be easily seen.

If you use an API key in your application, store it in Secret Manager or a similar secret management service. Secret Manager allows you to inject your API key into Cloud Run and GKE environments easily. However, to elevate the key protection you may want to read the key in your code instead. See documentation for an example.

If you use an API key with an external application that asks you to type in the key, take extra steps to explore how the application manages your key. You would need to find out how the key is stored and how it is used in the requests. For Web applications, you may use browser developer tools to inspect application traffic and ensure that the key is never sent in an unencrypted communication channel. For example, Google AI Studio uses encrypted local storage and sends the key via a TLS-encrypted channel.

When Something Goes Wrong

What to do if you suspect that your key is compromised? The straightforward action is the same as with a credit card. First thing ‒ delete the key. You can do it in the Cloud console or using gcloud services api-keys delete command. If you find out that it was a false alarm, you can undelete during the next 30 days.

What if you do not know which key is compromised? In that case you need to do a two-step investigation:

Find out all API keys in your organization or project(s)
Check the graph of API consumption for APIs this key allowing to access

Find out all your API keys

There is more than one way to find your API key resources. You can use Asset Inventory in the Cloud console and filter the dashboard by the Resource type to check apikeys.Key. If you do not see this resource type, find and click on "View more…" to expand the resource type list. Note that the list shows deleted API keys as well.

If you favor CLI, and you know specific project(s) you can use the gcloud services api-keys list command.

To see all active keys in your organization, you will need to use the gcloud asset search-all-resources command and query its JSON output to filter out deleted keys:

code_block: <ListValue: [StructValue([('code', 'gcloud asset search-all-resources \\\r\n --scope=\'organizations/123456789012\' \\\r\n --asset-types=\'apikeys.googleapis.com/Key\' \\\r\n --read-mask="name,displayName,versionedResources" \\\r\n --format=json \\\r\n --order-by=\'createTime\' \\\r\n| jq \'.[] | select(.versionedResources | all(.resource.data.deleteTime == null))\''), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4a60062ac0>)])]>

Find out API consumption

There is a way to track the usage of the API key. You can do it using the Cloud Monitoring metric serviceruntime.googleapis.com/api/request_count. This metric shows a number of times different services have been invoked. To see the number of service requests for a particular API key you will need to use the metric's label credential_id and filter it by the API key unique ID. You can see the metric data using Metrics explorer or use the Monitoring API with the following PromQL expression:

code_block: <ListValue: [StructValue([('code', 'sum(\r\n rate({\r\n "__name__"="serviceruntime.googleapis.com/api/request_count",\r\n "monitored_resource"="consumed_api",\r\n "credential_id"="apikey:00000000-0000-0000-0000-000000000000"\r\n }[${__interval}])\r\n)'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4a60062130>)])]>

You can further filter this metric by service_name label using API name (e.g. mapstools.googleapis.com).

In order to find out the API key ID you will need to use one of the following methods:

Using the Cloud console, open the Credentials page and select the API key that you want. Inspect URL of the API key page in the browser which will look like: https://console.cloud.google.com/apis/credentials/key/[KEY_ID]?project=[PROJECT_ID]. Copy the [KEY_ID] part.
Using gcloud CLI, run the gcloud services api-keys list --format='value(displayName,uid)'command and find the key by its display name. Copy the UID next to the display name.

Abnormally high level of API invocations usually indicates that the API key was compromised and used to access API by a malicious party.

Step 3: API key management hygiene

Whether you are an engineer, an experienced cloud user or just came to experiment, keeping proper API key hygiene is important to avoid your environment being hijacked from you.

If you already use Google API keys do the following right now:

Find out all API keys that you have
Delete all keys that you no longer use or do not recognize (do not worry, you can restore them during next 30 days)
Restrict API keys to only APIs that you intend to use. Narrow the list of clients that can use the APIs if you can
If you administer your Google Cloud projects or organization, consider setting up the apikeys.googleapis.com/Key org policy to minimize wrangling API keys
Consider periodically rotating (refreshing) your API keys by replacing them with newly created ones that share the exact same restrictions. Just be careful to track down and update all places where your existing key is used before deleting it to prevent unexpectedly breaking your application or abruptly losing access to one.

Wrapping up

Securing API keys is a vital step in protecting your cloud ecosystem. Implementing strict API and application restrictions, utilizing secure storage, and proactively monitoring consumption are highly effective ways to prevent unauthorized access. These practices safeguard your development environment from exploitation and prevent unexpected billing charges.

To help you implement these practices, here are a few practical tools and resources you can explore next:

Check more about APIs: Review Best practices for managing API keys and practice Search for and use Google APIs.
Watch a quick tutorial: Check out this great Google Cloud Tech video on Manage your Cloud Run secrets securely with Secret Manager to see secure storage concepts in action.
Get hands-on with a Codelab: Practice fetching credentials safely in a guided environment by trying Secret Manager with Python or with Spring Boot codelabs.

Dive deeper into the docs: Learn about how to select metrics, create charts and set up alerts to observe your API consumption.

Urban Outfitters achieves major cost savings by moving Sterling OMS to AlloyDB for PostgreSQL

Wed, 20 May 2026 16:00:00 +0000

Editor’s note: Urban Outfitters, Inc. (URBN) recently completed a major infrastructure upgrade, migrating its IBM Sterling Order Management System (Sterling OMS) from an Oracle database to Google Cloud's AlloyDB for PostgreSQL. This strategic move, a testament to the growing partnership between Google Cloud and IBM, delivers significant benefits for URBN, paving the way for increased efficiency, reduced costs, and a future-proofed technology landscape. This success story showcases how businesses can leverage AlloyDB for PostgreSQL to modernize their databases and unlock new levels of performance and scalability.

In the fast-paced world of retail, order management is the backbone of a seamless customer experience. Urban Outfitters, Inc. (URBN) relies on IBM Sterling Order Management System (Sterling OMS) as the nerve center of its global ecommerce operations, orchestrating everything from order capture and real-time inventory tracking to fulfillment optimization and post-purchase logistics. This system helps ensure that URBN can efficiently process millions of transactions across its global network of stores, warehouses, and digital channels, delivering on customer expectations for fast, accurate, and flexible order fulfillment.

However, the foundation of this critical system — a massive 11TB Oracle database — was increasingly becoming a bottleneck. High licensing and maintenance costs, growing operational complexity, and the constraints of proprietary technology posed significant challenges to scalability and long-term innovation. To maintain Sterling OMS's high availability, performance, and transactional integrity, URBN needed a modern database solution that could:

Reduce total cost of ownership (TCO): Lower licensing, operational overhead, and infrastructure expenses while maintaining reliability.
Ensure business continuity: Support high availability, rapid failover, and disaster recovery to prevent disruptions in order processing and customer transactions.
Embrace open standards: Transition from proprietary technology and embrace open, flexible, and future-proof solutions.
Maintain feature parity: Ensure a seamless migration without disrupting Sterling OMS functionality, keeping all mission-critical capabilities intact.

For a retail enterprise like URBN, even minor disruptions to order management can have significant financial and operational consequences. A failed transaction, an inventory miscalculation, or a delay in fulfillment can directly impact customer satisfaction, brand reputation, and revenue. Because Sterling OMS is so mission-critical, URBN required a migration approach that was as technically robust as it was precise — demanding a transition with near-zero downtime, data loss, or performance degradation.

The solution: AlloyDB for PostgreSQL

The success of this complex transition hinged on a deep, ongoing collaboration between URBN, IBM, and Google Cloud. This partnership brought together industry-leading expertise and cutting-edge technology, with teams working in lockstep to ensure high-touch engagement throughout every phase. By embedding dedicated IBM and Google Cloud engineers directly with URBN’s technical staff, the teams were able to meticulously plan and optimize the migration of the massive database.

The project’s success was defined by several critical pillars:

First-tier database recognition and feature development: IBM and Google Cloud engineering teams collaborated to ensure that Sterling OMS fully recognized and supported AlloyDB for PostgreSQL as a first-tier database.
Enterprise-grade reliability with two read replicas: To enhance performance and provide high availability and scalability, the AlloyDB deployment architecture includes two read replicas, providing low-latency access to data for reporting and analytics and improving operational resiliency of the entire Sterling OMS application.
Extensive performance tuning: A dedicated performance engineering team from Google Cloud worked alongside URBN and IBM experts to fine-tune database queries and optimize configurations. This level of continuous, high-class support ensured AlloyDB not only met but exceeded the performance benchmarks of the previous Oracle database. This was essential to handle the high transaction volume of the Sterling OMS on AlloyDB for a very large retail customer, the size of URBN.
Rigorous switchover testing and risk mitigation: Google Cloud and IBM teams assisted URBN in a rigorous, iterative swithover testing strategy, which involved running the Sterling OMS system on AlloyDB for a full day before switching back to the Oracle database. This proactive testing allowed URBN teams to identify and resolve potential issues in a controlled environment, significantly reducing risks and increasing confidence in the migrated system.

aside_block: <ListValue: [StructValue([('title', 'Build smarter with Google Cloud databases.'), ('body', <wagtail.rich_text.RichText object at 0x7f4a414cf8e0>), ('btn_text', ''), ('href', ''), ('image', None)])]>

A transformative shift

The migration to AlloyDB has fundamentally reshaped URBN’s data strategy, delivering a more favorable TCO through an optimized storage and compute architecture, without sacrificing performance or reliability. Furthermore, the shift to AlloyDB, a PostgreSQL-compatible database, gave URBN the flexibility of an open-source ecosystem. This move not only provides freedom from vendor lock-in, but also connects URBN to a vibrant community and a vast array of modern tools, ensuring long-term technical agility.

Beyond cost and flexibility, the transition unlocked superior performance and scalability to support URBN’s mission-critical operations. The combination of an optimized database kernel and precise query tuning resulted in significant speed improvements, directly enhancing the responsiveness of the Sterling Commerce system.

A blueprint for success: Planning and testing

URBN’s successful migration of IBM Sterling OMS to AlloyDB serves as a blueprint for organizations looking to modernize their mission-critical infrastructure and future-proof their environment for AI expansion. This journey proves that even the most complex, mission-critical migrations can be achieved through deep cross-organizational partnership and a phased, risk-mitigated approach.

For any enterprise navigating the challenges of modernization, URBN’s experience offers a clear roadmap for success. The use of iterative switchover tests — running the system on AlloyDB and switching back — was the "secret sauce" that built the necessary confidence for the go-live. By prioritizing this level of rigorous testing, businesses can move toward a future of greater agility, efficiency, and innovation.

Learn more:

Discover how AlloyDB combines the best of PostgreSQL with the power of Google Cloud in our latest e-book.
Try AlloyDB at no cost for 30 days with AlloyDB free trial clusters!
Learn more about AlloyDB for PostgreSQL.

Agent Sandbox on GKE is now available for everyone, and a first look at Agent Substrate

Wed, 20 May 2026 16:00:00 +0000

In just a short time, we’ve seen AI transition from simple chat interfaces to autonomous agents capable of function calling, code execution, and persistent terminal use. But to orchestrate these capabilities securely, agents need more than just intelligence — they need a robust, hyper-scalable, secure compute environment in which to execute code.

Since our preview announcement of GKE Agent Sandbox at KubeCon NA in November 2025, the community adoption has rapidly accelerated: we have seen more than 16x growth in sandboxes on Google Kubernetes Engine (GKE) in less than 5 months.

We’ve worked with key customers like Langchain and Lovable, and many others who are rapidly deploying millions of agents into production. Since its unveiling, Agent Sandbox has evolved rapidly, moving from a new project to a mature product with stable APIs. This stability is now fueling its integration into the broader agent ecosystem, where it serves as a critical infrastructure layer.

Today, we are excited to build on this momentum in two ways:

GKE Agent Sandbox is now generally available, giving you a secure, scalable foundation for your agent workloads
Introducing Agent Substrate, a new open source project aimed at continuing to push the limits of agentic infrastructure density

Secure, low-latency execution at scale

Agent Sandbox is an open-source, cloud-native execution environment built on Kubernetes, designed specifically for the unique demands of AI agents. It provides the foundational infrastructure to empower builders to safely and securely execute untrusted logic on top of their own infrastructure with industry-leading speed and efficiency.

With this release, we are delivering on the core requirements of modern agent workloads:

Reduce idle compute with pod snapshots: Agents often have short bursty cycles followed by longer idle periods. Instead of wasting valuable compute to keep the agent running, GKE Agent Sandbox integrates with Pod Snapshots to suspend your idle agent workloads and resume them in seconds upon request.
Low latency sandbox provisioning: Initializing a new sandbox instance for every request introduces unwanted seconds of cold start latency. GKE Agent Sandbox introduces a Sandbox API with an integrated warm pool. The Agent Sandbox API's integrated warm pool enables GKE to allocate 300 sandboxes per second, per cluster, at sub second latency, with 90% of allocations completing in 200 milliseconds.
Cost-effective warm pool: GKE Agent Sandbox warm pools keep pre-provisioned replicas ready to minimize sandbox startup latency. To minimize the cost of maintaining a sandbox warm pool, Agent Sandbox is integrated with standby capacity buffers (suspended VMs) to provide a cold pool of suspended sandboxes that can quickly replenish the warm pool for a fraction of the cost.
Ironclad security & isolation: Agent Sandbox natively supports gVisor and default-deny Kubernetes network policy. Agent Sandbox provides pluggable interfaces for open source sandboxes like Kata Containers, enabling users to customize their kernel isolation.

As the demand for compute continues to rise, this release ensures our customers have access to the broad range of Google Cloud compute options. GKE Agent Sandbox delivers up to 30% better price-performance when running on Axion processors than comparable hyperscaler cloud providers.

The next revolutionary step forward in agentic infrastructure Agentic workloads are simultaneously scaling up to the 10s to 100s of millions of instances while at the same time becoming increasingly idle, waiting for human interactions, events or triggers. These workloads continue to demand strong kernel and network isolation, making dense scheduling a challenge. Handling this level of scale and rapid suspend-and-resume is pushing the limits of the Kubernetes control plane.That’s why we are introducing Agent Substrate, a new open source project aimed at addressing the performance and density needs of ultra scale agents.

Agent Substrate introduces a new level of abstraction that moves agents onto and off of ready compute capacity (running in Kubernetes, of course) in real-time. Agent Substrate takes the core secure runtime and snapshotting capabilities of Agent Sandbox and pairs them with a minimal control plane designed to bypass some of the limitations of Kubernetes, without reinventing the rest of it.

This lets Agent Substrate optimize the critical paths to offer lower latency with higher scale and efficiency. While standard Kubernetes is optimized to handle thousands of long-running services, Agent Substrate is designed for the chatter of millions of sub-second tool calls that would otherwise overwhelm a standard control plane. It provides the perfect foundation for Agents, Agent Harnesses and Agent Runtimes, including the new Agent Executor project.

Agent Substrate’s goal is to explore every opportunity to make things move faster and scale bigger. Achieving this level of scale and efficiency is going to push the bounds of what current compute infrastructure can do, and no rock will be left unturned. One such exploration is to bring data locality into the core of the scheduler, ensuring that agent state and scheduling work together to shave off every possible millisecond of overhead.

Building the future in the open

In the early days of Kubernetes, the feedback and perspective from diverse contributors solving similar challenges was critical to setting the project up for success. We believe that agent infrastructure is at a similar inflection point. Today, we're hoping to recreate that magic of radically open and collaborative innovation to shape the future of agent infrastructure together. By kicking off the Agent Substrate project in the open, we are inviting the community to help design and build this critical next mode of infrastructure.

Get started today

As we look toward a future of autonomous agents, we are excited to continue to build the critical layers of the stack. We invite you to use Agent Sandbox to power your workloads today, and join us in the open-source community to collaborate on Agent Substrate – the next chapter in agent-native infrastructure.

Try Agent Sandbox on GKE
Contribute: Join the Agent Sandbox open-source community
Explore Agent Substrate

Introducing Agent Executor, Google’s distributed Agent Runtime

Wed, 20 May 2026 16:00:00 +0000

As models and harnesses improve, agents are taking on increasingly complex tasks that can run for hours or even days. But as we push agents to do more, this has surfaced a new operational problem: long-running agent workflows are fragile and incredibly hard to manage reliably and efficiently in production.

Today, we’re introducing Agent Executor, Google’s open-source runtime standard for agent execution, resumption, and distributed deployment. Based on what we’ve learned from solving these challenges internally, we’ve built Agent Executor to have the following native capabilities:

Durable execution: Long-running execution requires the ability to resume after outages or agentic interruptions such as human-in-the-loop (HITL) confirmations. Agent Executor provides this backend resilience automatically for any actor (e.g., an agent, agent harness, skill, tool, or sandbox) through its event log and snapshotting.
Secure isolation: Agent Executor isolates components in secure-by-design sandboxes to prevent harmful side effects and help ensure malicious activity cannot compromise the broader service. Sandboxes are especially useful when agents generate code or handle multiple tenants or user data concurrently.
Session consistency: In distributed agent workflows, multiple components may attempt to update shared session state at the same time. Agent Executor’s built-in single-writer architecture helps maintain consistency and reduces the risk of corruption in that state.
Connection recovery: In long-running agentic execution, clients may disconnect for many reasons, including network outages. Agent Executor lets clients reconnect to agents and backfills responses from the last sequence seen by the client for a better user experience.
Trajectory branching: Checkpoints let you branch an agentic trajectory (its decision or workflow path) at any point, allowing agents to test or evaluate different paths without losing context or other state.

In this blog, we’ll share more about Agent Executor and how you can get started.

Federate with Google’s agent runtime

Enterprise adoption of agents requires orchestration across deployment models. Some teams need on-prem infrastructure for proprietary workflows, performance, or compliance, while others prefer pre-built or custom managed agents for faster time-to-value. At Google I/O, we introduced a new suite of such solutions – including Antigravity 2.0 and the Managed Agents API – designed to accelerate how teams build and scale within the agentic enterprise.

Agent Executor bridges these deployment models, letting you mix-and-match between any or all of:

Google Antigravity, Gemini’s state-of-the-art agent harness
Google-built frontier agents, such as the latest Deep Research agent
Custom agents built by you and managed by Google (e.g., via the new Managed Agents in Gemini API)
Custom purpose-built agents, built with LangChain/LangGraph, Agent Development Kit (ADK), etc and any agents using Agent2Agent Protocol (A2A)

Own your agents, models, and compute

With Agent Executor, enterprises have maximum flexibility to maintain sovereignty over workloads and keep proprietary workflows within their self-managed compute and custom sandboxes. Your internal development teams have much more flexibility over how agents are deployed and managed and you benefit from:

Prevent vendor lock-in: Deploy your agents on your own infrastructure without being tethered to a specific provider’s model or compute environment. This allows for full control over data residency and your cost and budgetary controls.
Bring your own harness and agents: Agent Executor is designed to be harness-agnostic, allowing you to bring your own or use those made available by other vendors. It also supports agents developed with industry-standard frameworks and protocols providing a broad ecosystem of compatible agents.
Fully control execution: Agent Executor allows developers to run the entire agentic stack, including MCPs, skills, and other agents, directly on their own data plane. Developers can choose any compute with custom isolation boundaries and workload policy enforcement.

Scale agents up on Kubernetes with an agent-first compute layer

As agent workloads scale into the hundreds of millions and become increasingly long-running, our customers are hitting the limits of traditional compute abstractions because unlike traditional software, agents are nonlinear programs that wait for external inputs. To solve this problem, we’ve partnered with the Google Kubernetes Engine team on Agent Substrate, a new open-source project also announced today.

Agent Substrate introduces a new level of abstraction for Kubernetes that moves agents onto and off of ready compute capacity in real-time, resulting in lower latency with higher scale and efficiency. While standard Kubernetes is optimized to handle thousands of long-running services, Agent Substrate is designed for the chatter of millions of sub-second tool calls that would otherwise overwhelm a standard control plane. Agent Substrate takes core secure runtime and snapshotting capabilities of existing sandbox infrastructure and pairs them with a minimal control plane designed to bypass some of the limitations of Kubernetes, without reinventing the rest of it. Working together, these layers enable you to:

Maximize compute efficiency: Agent Substrate introduces a new control plane designed to handle hundreds of millions of registered agents. Together with Agent Executor, Agent Substrate can provide a foundation for today’s largest agent deployments.
Stay within the Kubernetes ecosystem: Agent Substrate is built on top of Kubernetes and allows scheduling and horizontal scaling of compute with declarative configuration.

In the demo below, we showcase using Agent Executor together with Agent Substrate with a sample workload.

Get started today

Models, agents, harnesses, and the infrastructure around them are all evolving faster than ever. We’re building Agent Executor in the open so we can validate the design in the hands of real developers and improve based on your feedback.

Agent Executor is available now in preview. We invite you to explore the code, test it with your own workloads, and help shape the future of agent runtimes. Head over to our GitHub repo to get started today.

Benchmark and optimize LLMs on-device with AI Edge Portal

Wed, 20 May 2026 16:00:00 +0000

LLMs have become more powerful at smaller sizes, but deploying them to edge devices like smartphones remains a massive challenge. Today, developers have to optimize across a sprawling combination of accelerators, operating systems, and countless System-on-a-Chip (SoC) configurations, often relying on manual testing with just a handful of devices. Google AI Edge Portal helps solve these challenges.

By letting developers test ML workloads across a fleet of over 120 representative Android device types, Google AI Edge Portal provides deep insight into latency and performance across all CPU, GPU, and NPU backends.

Today, we are excited to announce two new capabilities that expand Google AI Edge Portal’s capabilities for the generative AI era: benchmarking and debugging on-device LLMs. These new services give developers what they need to optimize generative AI performance accurately and efficiently across the entire Android ecosystem.

Benchmark LLMs across over 120 different mobile devices

When a user interacts with an LLM-enabled experience in your app, they expect fast and consistent performance on their device. Common challenges like initialization time can result in your app appearing to freeze, or, in a worst case, crash completely if the model consumes all available memory.

With the latest release of Google AI Edge Portal, you can now run automated gen AI benchmarks directly on a physical lab of over 120 diverse Android devices and test for these scenarios specifically. Portal natively supports CPU and GPU benchmarking for LLMs in the LiteRT-LM format.

Customers can benchmark GenAI models on over 120 Android devices, viewing metrics including initialization time, prefill speed, decode speed, and peak memory usage.

When you trigger a gen AI benchmarking job with Portal, it profiles the critical metrics that dictate your end-users’ experience when interacting with your AI application on-device:

Metric	What it measures	Why it matters to you
Initialization time	Measures how long it takes to load your model into memory.	High initialization time can result in delays, or freeze the user interface when your application starts up.
Prefill speed	Captures how fast the device processes prompt tokens to generate the first output token.	Dictates the initial delay before the user sees the first response.
Decode speed	Captures how fast the model generates tokens during a response.	Dictates the speed at which output is generated.
Peak memory	Monitors maximum RAM usage.	Flags potential “out of memory” crash risk, especially prevalent on memory constrained devices.

With these insights, you can confidently decide which devices are ready to host your model and adjust or better optimize your LLMs for device targeting before shipping.

Debug performance easily with Model Explorer

Benchmarking is only useful if you can fix the discovered performance issues. When an LLM performs poorly, finding the root cause within the complex graph of multiple layers and thousands of nodes is a daunting task for developers, involving tedious and time-consuming searching that can take hours if not days.

To bridge this gap, we have added the ability to visualize and compare model graphs in Portal with ease. Through the natively integrated Model Explorer, our graph visualization tool, you can search and locate specific nodes, compare models side-by-side in the same tab, and view tensor shapes, trace inputs and outputs, and more. To further speed up debugging for teams, we also added the ability to take screenshots and share specific views directly with your collaborators in Google Cloud.

These visualizations are one of the most effective ways to identify targets for optimization, including:

Conversion: Model Explorer simplifies the identification of conversion anomalies through its dual-view comparison tool. This interface allows you to traverse complex model architectures by selectively expanding or collapsing specific layers, granting you the ability to analyze internal dependencies and structural nodes with precise granularity.
Quantization: Model Explorer aids in detecting specific operations where quantization may compromise performance. By sorting layers using error metrics, you can pinpoint precision loss, access granular per-layer data, and evaluate various quantization strategies to achieve an optimal balance between model footprint and output quality.
Optimization: Use Model Explorer to visualize hardware compatibility, organize operations by latency, and conduct granular, per-op performance comparisons across different hardware accelerators.

With Model Explorer, you can view model graphs, search for specific layers, and compare models side-by-side to debug performance.

Start benchmarking LLMs on-device today

With the era of LLMs on-device here, we are excited to help close the critical gap in benchmarking to bring the power of AI to the thousands of types of smartphones on the market today. To utilize these latest features, please complete our sign-up form here to express interest.

Google AI Edge Portal is currently available in private preview for allowlisted Google Cloud customers. During this private preview period, access is provided at no charge, subject to the preview terms. All current allowlisted customers will receive access to these new features automatically.

We can’t wait to see what gen AI capabilities you are able to deploy across the full spectrum of devices with Google AI Edge Portal!

_{Thank you to the members of the team, and collaborators for their contributions in making the advancements in this release possible: Akshat Sharma, Ami Kubota, Charlie Xu, Chunlei Niu, Cormac Brick, Derek Bekebrede, Eric Yang, Jing Jin, Kathleen Low, Matthias Grundmann, Marissa Ikonomidis, Na Li, Ram Iyengar, Sachin Kotwani, Sommayah Soliman, Tenghui Zhu, Xiaoming Hu, Zi Yuan}

The agentic era: Architecting the blueprint for mission impact across the public sector

Tue, 19 May 2026 19:00:00 +0000

This is a new era — the agentic era – and the question is no longer, “what’s possible?” but rather, “what creates impact?” Today, organizations across industries around the world are swiftly moving from AI exploration and pilots to real-world use cases that drive impact, at scale. They are doing this with agents. In order to fully meet this moment, and seize the opportunities in this agentic era, we need leadership. To that end, I was honored to share the stage at Google Cloud Next with visionary leaders from the U.S. Food and Drug Administration (FDA), the U.S Department of Transportation (DOT) and the City of Los Angeles who shared how they are disrupting the status quo and driving lasting impact for their organizations and the people they serve. Let’s take a closer look at their stories.

The U.S. Department of Transportation: Strengthening the safety and reliability of our nation’s transportation systems

Pavan Pidugu, the Chief Digital and Information Officer at the U.S. Department of Transportation (DOT), is empowering staff with cloud-based productivity and collaboration tools as the first-cabinet level federal agency to fully transition their workforce away from legacy providers to Google Workspace with Gemini. “Twenty-two days. That’s what it took for us to create a production environment with Google Workspace,” Pavan asserted. In less than six months, the agency migrated more than one billion emails. This transition to Workspace has enabled staff to work smarter and faster, ultimately strengthening the safety and reliability of our nation’s transportation systems.

The U.S. Food and Drug Administration: Accelerating cures and meaningful treatments that benefit everyone

Jeremy Walsh, the Chief AI Officer at the U.S. Food and Drug Administration (FDA), is leveraging agentic AI to accelerate the delivery of life-saving cures, achieving a 80% AI adoption rate across its 18,000-person workforce. “By using AI agents to move the agency toward a real-time regulatory environment, the FDA is compressing decades-old workflows,” underscored Jeremy. While the traditional drug development process can take ten years, the agency now uses AI to analyze candidate data against vast historical datasets in minutes – shrinking 60-day filing reviews down to a matter of hours. This mission to halve the overall time to market for new drugs directly translates into saving more lives.

The City of Los Angeles: Building the technological foundation to serve residents and visitors

Ted Ross, the Chief Information Officer for the City of Los Angeles, is facing a massive logistical challenge: preparing for the 2026 World Cup, the 2027 Super Bowl, and the 2028 Olympic and Paralympic Games. With 15 million visitors expected for the Olympics alone, the city is scaling AI across its 45 departments and 27,500 employees. “The world is going to be looking at Los Angeles, and the reality is you need force multipliers,” Ted articulated. “You need tools like AI, which don’t replace our workforce, but amplify their ability to deliver world-class customer service and the essential functions to run events of this scale.” The City of Los Angeles is embedding Gemini directly into the tools employees use every day to deliver faster, more accessible services for visitors, as well as the city’s four million residents who speak more than 224 languages.

Your blueprint for agentic transformation

These leaders are creating the blueprint for agentic transformation by leveraging AI and agents as a force multiplier to empower their workforce, unlock new levels of productivity, and transform how services are delivered. As evident from these leaders, moving from exploration and pilots to full-scale adoption requires a thoughtful and strategic approach, centered on three critical pillars:

Empower leaders and teams to disrupt the status quo: Transformation requires a cultural shift where leaders and teams are empowered to disrupt the status quo, take calculated risks, and move with speed. As demonstrated by Pavan Pidugu of the DOT, the goal is to ignite a spirit where teams can harness technology to create results in days, not years.
Scale for lasting impact: Moving from AI pilots to agency-wide adoption requires bold leadership. Jeremy Walsh shared how the FDA is empowering its workforce – from reviewers to scientists – with AI and agents to protect public health and accelerate life-saving treatments. As Walsh articulated, agents act as force multipliers, moving the agency beyond document-heavy processes to a state where they can analyze all available data simultaneously. This approach allows the FDA to compress review timelines and solve mission-critical challenges at scale.
Prioritize human-centered adoption: Transformation is fundamentally a “people project” that succeeds by solving immediate employee pain points within existing tools. Ted Ross from the City of Los Angeles shared that by embedding AI into daily workflows, leaders can turn skeptics into champions through small wins that create organizational momentum. This approach raises the collective digital IQ, and ensures that the city can upskill their workforce while delivering faster, more accessible services for residents and visitors alike.

Your mission partner

We are so proud that Google Public Sector is the partner that organizations turn to as they move from AI pilots to total agentic transformation. We believe that bold leadership, coupled with our integrated stack built on a foundation of security, openness and scale, is what makes this kind of transformation possible. We invite you to register to attend our Gemini for Government webinar on June 11 where we’ll dive deeper into the blueprint for transformation and mission impact in the agentic era.

Everything Google Cloud customers need to know coming out of Google I/O

Tue, 19 May 2026 17:45:00 +0000

At Google Cloud Next ‘26, we unveiled the blueprint for the Agentic Enterprise, sharing our eighth-generation TPUs, Gemini Enterprise Agent Platform, a fully reimagined Agentic Data Cloud, Workspace Intelligence, and security built for the AI era.

Today at Google I/O, we’re delivering a new set of powerful AI innovations and models — and putting them directly in the hands of Google Cloud customers via Gemini Enterprise and Google Workspace:

Gemini 3.5: Our latest family of models combines frontier intelligence with action – starting with Gemini 3.5 Flash.
Gemini Omni: Our new model is a leap forward in world understanding, multimodality, and editing, letting you generate any output from any input, starting with video.
Google Antigravity: Google Antigravity’s expanded capabilities and new integration with Agent Platform bring agentic development to your entire organization.
Gemini Spark: For Gemini Enterprise and Workspace customers, Gemini Spark is your 24/7 personal AI agent that helps you work more efficiently by autonomously taking action on your behalf, under your direction.
Google Workspace: Google Pics, our new image generation and editing tool, and new voice features in Gmail, Docs and Keep, help reimagine how you work.
Managed Agents API on Agent Platform: Allows developers to build and run custom agents inside secure, Google-hosted environments that seamlessly integrate with Agent Platform.
CodeMender: A powerful AI security agent provided through Agent Platform, CodeMender can help find and fix vulnerabilities in your code.

Expand what’s possible with Gemini 3.5

We’re kicking off the Gemini 3.5 series with the release of 3.5 Flash, which delivers frontier performance for agents and coding, excelling at complex long-horizon tasks that deliver real-world utility. Google DeepMind engineered these models from the ground up using our purpose-built AI infrastructure. This unique co-design of the model and hardware allows us to train deeper reasoning capabilities faster and more efficiently with every new generation.

Gemini 3.5 Flash delivers intelligence that rivals large flagship models on multiple dimensions at speeds you have come to expect from the Flash series. It’s our strongest agentic and coding model yet, outperforming Gemini 3.1 Pro on key benchmarks (Terminal-Bench 2.1: 76.2%, GDPval-AA: 1656 Elo, MCP Atlas: 83.6%) and leading in multimodal understanding (CharXiv: 84.2%). 3.5 Flash offers a balance of performance and speed, ideal for tackling long-horizon agentic tasks, often at less than half the cost of comparable models.

Gemini 3.5 Pro is currently in testing and will be coming next month.

We have partnered closely with leading organizations across industries to validate the Gemini 3.5 series within their own environments.

Gemini 3.5 Flash is rolling out today:

Developers can build agents using Gemini 3.5 Flash on the Gemini Enterprise Agent Platform, or use it in your projects in Google AI Studio and Antigravity.
Business users can start using Gemini 3.5 Flash in the Gemini Enterprise app to help discover, create, and use the best of Google AI in their workflows starting today.

Generate high-quality video content with Gemini Omni

Gemini Omni is a groundbreaking new model that produces dynamic video content by blending text, audio, image, and video inputs. Building on how Nano Banana reimagined what you can do with images, Gemini Omni delivers a highly intuitive approach to video creation and editing using natural language.

This capability redefines how enterprises produce and refine visual media with intuitive creation and precise editing. Whether you are developing interactive virtual try-ons for e-commerce, streamlining complex post-production workflows, or generating tailored video narratives, Gemini Omni helps your team unlock entirely new ways to create content and drive deeper customer engagement.

Gemini Omni Flash will be rolling out in the coming weeks to developers and enterprise customers via the Gemini API and Agent Platform API.

Make everyone a builder with Google Antigravity

Google Antigravity empowers the next era of enterprise builders. It enables your organization to transform how applications are built, deployed, and managed.

With Gemini 3.5 Flash, Antigravity delivers exceptional computational efficiency. This translates to more rapid development cycles and reduced operational costs for production-scale AI initiatives. Today, we are announcing new tools including:

Enterprise security and compliance: Google Cloud customers can now access Antigravity through Agent Platform. By inheriting Google Cloud’s standard data privacy protections and Terms of Service, this ensures your customer data is in your control and agent activity runs within your secure cloud boundary by default.
Antigravity 2.0 desktop app: At the core of the expanded Antigravity platform is Antigravity 2.0, a new standalone desktop app for builders that provides a centralized workspace to steer, customize, and orchestrate agents. For example, imagine managing a product launch and using Antigravity 2.0 to deploy simultaneous, agent-driven execution across critical project phases, including automated code generation for your website, creation of on-brand assets, and personalized customer email development.

Sequences shortened throughout

Antigravity CLI: For developers who want a more lightweight interface for rapidly building and deploying agents, we are also launching an Antigravity CLI that is tightly integrated with the desktop app.

We’re using Antigravity every day to build at Google, and we’ve already seen Cloud customers and partners using it, too:

“In modern enterprise architecture, the real bottleneck is the cognitive load on the engineer. Antigravity, augmented by Gemini within Accenture’s scaled agentic execution, abstracts away infrastructure complexity and automates delivery mechanics. By moving to a standalone application and consumption model, Google Cloud is making high-velocity engineering accessible at scale—enabling our best-in-class talent to focus on innovation and building resilient digital cores for our clients.” - Chetna Sehgal, Senior Managing Director and Global Practice Lead, Accenture Google Business Group

“Antigravity isn't just an assistant; it’s a core component of our technical DNA. Its ability to simulate user environments via the agentic browser has revolutionized our QA process, while the agent manager has streamlined our entire pipeline. Today, more than half of our production-ready code is generated through these agentic workflows, proving that the future of software isn't just AI-assisted—it’s AI-driven." - Nikunj Shanti, CTO, AirAsia Next

“Antigravity has transformed how our internal and Forward Deployed Engineering teams operate by enabling governed, autonomous software engineering workflows that adhere to Deloitte's enterprise security standards at massive scale. This capability allows us to rapidly accelerate the deployment of high-fidelity, AI-powered solutions for our clients and drive innovation across industries." - Faruk Muratovic, US AI & Engineering Strategy and Services Leader at Deloitte

"Antigravity has fundamentally shifted our team from manual coding to high-level orchestration. By leveraging the agentic reasoning of Gemini, we’ve moved beyond simple code completion to a governed, autonomous workflow that maintains architectural integrity through every sprint. The ability to transform a functional requirements document directly into high-fidelity code while automating unit tests and documentation has drastically reduced our concept-to-deployment lifecycle. Antigravity acts as an autonomous coding partner that empowers our engineers to shift from manual syntax entry to governed, intelligent execution." - John O'Rourke, Technical Director, AI Implementation, Monks

"We're moving past simple AI code completion to true agent orchestration. By using Google Antigravity to run engineering pipelines in the background, our teams eliminate traditional development friction, allowing us to build custom client solutions and deliver value at unprecedented speed." - Vikas Agarwal, CTIO, PwC Advisory, PwC US and Global

“WPP has integrated Antigravity into WPP Open, our agentic marketing platform to supplement our product development lifecycle. Leveraging the power of Gemini, it has streamlined workflows, automated repetitive tasks and empowered engineering teams to deliver high-quality solutions for our clients, faster.” - Callum Anderson, Head of Engineering, WPP Open

To get started, download Antigravity and log in to the desktop application or Antigravity CLI using your standard Google Cloud credentials. Antigravity will be available in Gemini Enterprise in the coming months.

Gemini Spark, your 24/7 personal AI agent

To realize its full potential, AI must serve as the connective tissue that empowers every individual within your enterprise. This requires intelligent personal agents that grasp your specific business context – like what you’re working on, who you’re working with, and even your writing style. These agents can autonomously execute multi-step workflows on your behalf, with your permission, freeing your teams from routine, manual processes so they can focus on the high-impact, strategic innovation that drives your business forward.

Gemini Spark in Gemini Enterprise is our new 24/7 personal agent that can work in the background across Workspace, custom connectors, and the open web. It enables you to:

Delegate complex work: Set recurring tasks, teach the agent new skills, and let it execute multi-step work on your behalf.
Maintain complete control: Spark proactively sends critical updates and requires explicit approval for high-risk actions like sending emails.
Personalize your agent-led experience: The more you use Spark, the better it learns your unique preferences and interactions to become a more accurate, helpful extension of your work style.
Connect to your tools and apps: Spark can use your existing Gemini Enterprise connectors, including Microsoft Sharepoint, OneDrive, ServiceNow, and many others.
Security and governed sandbox: Spark operates in a fully managed, secure runtime on Google Cloud, meaning you get enterprise-grade security without ever having to manage the underlying infrastructure. Every task executes in a fresh, strictly isolated, ephemeral VM to help ensure data never overlaps between sessions. To protect your enterprise, all traffic routes through our secure Agent Gateway that enforces Data Loss Prevention (DLP) policies, while user credentials remain fully encrypted and are never exposed directly to the agent.

Draft an email to stakeholders with Spark in Gemini Enterprise

Spark can assist with tasks for many different processes in your organization to help teams focus on more strategic work. For example:

Spark can identify a new product request related to critical functionality that needs to be fixed before launch and requires a major timeline change. In the background, it can automatically suggest code changes by working with Antigravity, create a Jira ticket for the development team to review the changes, cross-reference your team’s documents to recalculate the launch timeline, and update the internal status across relevant Sheets and Docs. It can then draft an email to relevant stakeholders confirming the actions it took and updating the team.
Spark can help someone working in IT operations by monitoring system health via ServiceNow, checking for relevant tickets. When it detects a recurring critical issue, it creates an escalated Jira ticket to the developer team, drafts a comprehensive incident report in Docs, and pings the IT manager via Chat to review and approve the stakeholder communication plan.
For salespeople, Spark can proactively prepare for client meetings by pulling account history from Salesforce and recent support tickets from Zendesk. Identifying a potential churn risk, Spark drafts a tailored account retention strategy in Docs and a customer-ready email, awaiting explicit user approval to send.

Gemini Spark in the Gemini Enterprise app is rolling out to customers soon. Gemini Spark in Google Workspace will be available soon in preview for business customers in the Gemini app.

Reimagine how you get work done with Google Workspace

Today, we also announced new ways AI can help you accomplish even more directly in your Workspace apps, available in preview for business customers this summer:

Google Pics: Our new AI-powered image generation and editing tool gives you precise control over your images. Pics allows you to easily move, resize, and transform individual objects as well as modify and translate text independently. Built right into apps like Drive, Docs, and Slides, it makes complex editing easy so teams can quickly update global marketing campaigns, swap out products in a photo, or resize backgrounds to fit different ad layouts.
Voice capabilities in Gmail, Docs and Keep: Brainstorm, organize and execute tasks hands-free in your favorite apps. You can quickly find a project due date from your inbox, iterate on a draft of your weekly report, or turn a messy brain dump into a structured list.

Unlock developer velocity with fully managed agents

With the Managed Agents API on Agent Platform, builders can now instantly spin up custom agents that reason, call tools, and execute code inside secure, Google-hosted remote environments using a single API call. This allows technical teams to offload complex infrastructure management and focus entirely on agent behavior. Managed Agents API will automatically inherit Agent Platform's enterprise-grade data privacy, governance, and security protections.

Visit the documentation to get started.

Secure the Agentic Enterprise

Advanced models, such as Gemini 3.5, unlock powerful new capabilities across your enterprise, including transforming AI security and vulnerability detection. Strategic risk mitigation, robust compliance, and trusted AI deployment are critical for sustained business growth.

In addition to sharing our findings and mitigations with the larger security and AI community, we are integrating CodeMender into Agent Platform. CodeMender is an AI code security agent, originally developed by Google DeepMind. Leveraging Agent Platform capabilities and advanced Gemini models, CodeMender autonomously identifies vulnerabilities within your code. It then recommends precise fixes, securely tests them, and can apply patches and necessary changes across dependent systems, with your approval. This entire process automates secure deployment while ensuring your developers retain control.

Several Gemini Enterprise customers are already testing CodeMender, and we will have more to share about expanded availability soon.

We are also making significant strides in ensuring platform trust and safety for synthetically generated media with a new AI Content Detection API, rolling out today on Agent Platform. This provides businesses with a powerful method to identify AI-generated content from both Google and other popular models, supporting responsible media governance. Learn more about our expanded tools for content transparency and verification.

Looking ahead

With the latest Google Cloud updates unveiled at I/O, we’re making it easier than ever to move from initial idea to meaningful impact. These new tools are designed to streamline your workflows and empower your teams. For a deep dive into all the news and live product demonstrations, catch the full I/O keynote.

What Google I/O '26 means for developing agents on Google Cloud

Tue, 19 May 2026 17:45:00 +0000

At Google I/O, we introduced a unified development toolkit featuring Antigravity 2.0 and the Managed Agents API, giving developers better ways to build locally and deploy securely to the cloud on a shared protocol layer. In this blog, we’re going to show you how Gemini Enterprise Agent Platform and the new developer tools shared at I/O fit together, unpack the spectrum of choice for building, and share what we’d actually try first.

Following the evolution of Vertex AI into the Gemini Enterprise Agent Platform – a comprehensive platform to build, scale, govern, and optimize agents with new features like session memory and centralized governance – we are now extending these capabilities directly into your local development tools. Our goal is to bridge the gap between high-speed prototyping and secure, compliant corporate deployment, offering a modular approach where you can choose between quick-start workflows or full production control to fit your stack's specific needs.

Here’s how those pieces now lay out across the entire spectrum of choice.

The four rungs: The spectrum of how to build agents

We like to think of the agent development ecosystem as four rungs on a ladder, designed to give you a clear slider between out-of-the-box configuration and complete code-first control. They're deliberately additive, meaning that starting fast on the lower rungs above never locks you out of graduating to the deeper customization of the rungs above.

Underneath all four rungs is the A2A protocol. This interoperability ensures that an agent built on the first rung can be called as a sub-agent on the fourth rung, allowing your entire architecture to scale seamlessly on the same infrastructure.

Rung one: Agent Studio (low code)

A visual workspace inside Agent Platform. You discover models in Model Garden, engineer prompts, wire up tools, and ship an agent without writing code. Best for business-facing teams and rapid prototyping. The agent you build here runs on the exact same runtime as everything below it.

Rung two: Managed Agents API

New at I/O, the Managed Agents API is for technical teams who want to “manage the mission, not the machine." It allows you to define agentic behavior and let Google Cloud handle the heavy lifting, acting as an agent-as-a-service with nothing to manage.

You use the Managed Agents API to configure your agent, and the Interactions API to invoke it. You package your instructions, skills, and tools, POST them, and Gemini builds and runs the agent.

What makes this deployable is the Google Cloud sandbox, which is secure by design. The agent harness runs on our servers, and each agent has its own ephemeral sandbox provisioned with your skills, Model Context Protocol (MCP) servers, and server-side tools. Full integration with A2A and Agent Platform governance and security are coming soon.

Rung three: Antigravity and friends

Antigravity is our primary solution for developers looking to leverage AI for coding tasks and agent orchestration, enabling teams to transform how apps are built and deployed. We've consolidated our developer-facing coding strategy into this single, powerful harness shared across multiple surfaces.

It’s co-optimized with the Gemini family of models, offering high efficiency to speed up development cycles and reduce costs. Skills you develop with Antigravity are intended to be portable across different surfaces.

This is for development teams who want to utilize Google's advanced reasoning capabilities within their coding workflows, implement custom development loops, and transform how they build, deploy, and manage applications.

Today, we are expanding this with new tools:

Antigravity 2.0: A new standalone desktop application providing a centralized workspace to steer, customize, and orchestrate coding agents. Developers can use this to manage complex tasks, such as orchestrating agents to refactor code, generate unit tests, or even scaffold new service components based on a specification. Agents can spin subagents from a single prompt, while multi-agent orchestration allows tasks to run in parallel.
Antigravity CLI: This brings the full Antigravity experience to the command line: same harness, same agent, same quality of intelligence as Antigravity 2.0, with a product experience tailored for the terminal. It's optimized for speed and lower overhead, and adapts entirely to you. The CLI is tightly integrated with the desktop app, sharing authentication, context, skills, and configurations, providing a consistent experience across both interfaces. Use the Antigravity SDK to build your own runtime.
Enterprise security and compliance: Google Cloud customers can now use Antigravity 2.0 and Antigravity CLI with their Gemini Enterprise Agent Platform project. All you have to do is to log in with Cloud OAuth, set your Agent Platform Project ID and region. This ensures that all agent inference runs via Agent Platform models within your secure cloud boundary, inheriting Google Cloud’s standard data privacy protections and Terms of Service. This ensures your customer data is in your control , and you can utilize regional model endpoints.

Integrating other coding agents

While Antigravity is our recommended agentic coding solution, Google Cloud is designed to work well with any coding agent you choose. Our platform is open, and we provide tools to ensure flexibility:

The Agent CLI and Agent Development Kit (ADK) allow you to build and interact with agents from various sources, including tools like Claude Code. This means developers can often keep their preferred interfaces while running the underlying AI inference on Google Cloud. This approach ensures your workflows benefit from Google Cloud's security, compliance, and infrastructure.
Our Skills for Google products, launched at Next, are designed to be compatible with multiple coding tools, enabling you to enhance different agents with a consistent set of capabilities.

This flexibility allows teams to integrate their existing favorite tools and models, ensuring seamless and compliant operation within their established workflows.

Rung four: Agent Development Kit (ADK 2.0)

Code-first, low floor, high ceiling. If Managed Agents are configuration-first, ADK is engineering-first. This is for software engineers who want to build custom agent meshes from the ground up - any architecture, any model, unconstrained.

ADK enhancements launched at Google Cloud Next are now available for everyone. It introduces a unified graph-based engine that gives you a slider from dynamic, model-led reasoning to strict, deterministic workflows. The framework handles the heavy lifting of multi-agent coordination, managing how sub-agents, tools, and data pass between one another.

Collaborative workflows (Python v2.0.0): Previously called the Task-based Agent Collaboration API, this is how you build self-managing agent teams. A coordinator delegates to subagents using explicit operating modes:
- chat: Full user interaction, manual return to parent, this is “handoff conversation to sub-agents”.
- task: User interaction for clarifications, automatic return to parent, this is a new “collaborate for this assignment” which is the best of both other options.
- single-turn: No user interaction, parallel execution, automatic return, this is “agent as tool”.
Dynamic workflows: Dynamic workflows in ADK allow you to put aside graph-based path structures and use the full power of your chosen programming language to build workflows. With Dynamic workflows, you can create workflows with simple decorators, invoke workflow nodes as functions, and build complex routing logic.
ADK Kotlin (Beta): "ADK for Android." Kotlin support joins Python, Go, and Java, increasing language coverage so your on-device mobile agents can seamlessly coordinate with your backend Python agents.

Finally, the Agents CLI packages Google's expert skills for ADK, eval, deploy, observability, and publishing - turning any AI coding agent (like Antigravity, Gemini CLI, Claude Code, or Cursor) into an expert at agent app building as well as agent ops. It gives your AI Agent skills to understand the Google Cloud agent stack, turning an expansive ecosystem into a seamless assembly line for developers hillclimbing their agent builds.

What we'd actually try first

If we were starting today, here's the order we'd reach for things:

Start with the Antigravity 2.0 desktop app: Explore the interface, add a pre-built agent, and interact with it to understand the core functionality. This provides a more intuitive entry point before diving into API specifics.
Build a mesh: Feel free to explore Managed Agents API through the Agents API skill and Interactions API skill. When you start hitting routing decisions you want to make explicit, or need complex multi-agent orchestration, port your logic to ADK 2.0. The graph model is worth the learning curve as soon as you have more than two branching paths. Don't worry about stringing together a bunch of separate pieces to make this happen - this is exactly where the Agents CLI shines.
Govern and reuse shared domain logic: Check out Skill Registry (public preview): A centralized catalog to govern and promote the reuse of packaged domain logic. Skills are accessible via the Managed Agents API, Agent Platform SDK, and ADK (via SkillToolset). Skill Registry will be part of Agent Registry shortly.
Evaluate: Use the Gemini Enterprise Agent Platform evaluation suite to move beyond basic text-matching vibe checks. Leverage synthetic user simulation to auto-generate multi-turn testing scenarios and safely mock API environments to pressure-test tool resilience. Finally, utilize its LLM-based autoraters and trace logging to evaluate complex logic, group failures, and continuously optimize your agent.
Secure the pipeline: Leverage Gemini Enterprise Agent Platform governance capabilities like Agent Identity, Agent Gateway, Agent Security, and Agent Registry to secure your deployment. Once CodeMender releases, add it to your CI/CD to proactively secure the code your human (and AI) developers are pushing.

Note: You can do this whole loop on a Google Cloud Starter Tier account without a billing account attached. First two app deployments are on us.

We’re excited and hope you are, too

The agent space is evolving rapidly. Agent Platform offers a secure and adaptable foundation. Core components like the Agent Gateway, identity management, and the Skill Registry work together to ensure a robust and controlled environment for your agents, enabling you to innovate flexibly without vendor lock-in.

Pick the rung that fits the project. Bring whatever coding agent your team prefers. The platform you graduate to is the same one either way, and the data stays inside your Cloud project the whole time.

If you only read one set of docs after this post, make it the Agents overview in the Agent Platform documentation. If you build something interesting, show us - the best examples will land in the next round of templates.

We can’t wait to see what you build!

The future of agentic development: Redefining the data practitioner lifecycle with Data Agent Kit

Tue, 19 May 2026 17:45:00 +0000

The modern software development landscape isn’t happening just on one surface — it’s happening across an entire ecosystem of agentic tools. Agents are being developed at an unprecedented scale, and these agents require direct access to enterprise data for context and grounding.

However, the current tooling for building agents and managing data is heavily fragmented. This can make it difficult to access data, increasing security risks, and causing broken developer experiences that hinder innovation.

To address this challenge, we recently launched Data Agent Kit, a unified, open-source collection of data engineering and data science skills, tools and plugins that integrate directly into the environments practitioners already use, such as VS Code, Claude Code, Codex, Gemini CLI and the Antigravity CLI. By seamlessly bringing together these core tools and skills with your enterprise data, the Data Agent Kit effectively serves as a comprehensive harness for agentic context, memory, and personalization. It provides:

Agentic skills: Pre-codified pathways for interacting with your data estate, covering query optimization, ML best practices, data validation, data drift checks, governance, and troubleshooting.
Model Context Protocol (MCP) tools: Secure connections between agentic workflows and cloud data platforms like BigQuery, AlloyDB, and Google Cloud Storage. Developers can now configure connection parameters for their cloud datasets and data processing engines without having to manage complex, manual pipeline code.
Plugins and extensions: Native IDE integrations that enable rich, context-aware developer interactions.

Together, these Data Agent Kit capabilities help data practitioners go from manually writing code to intent-driven data science and engineering: defining the desired business outcomes, constraints, and success criteria, and allowing the AI-augmented system to figure out how to execute it. This shift is critical because today, when building agentic applications that navigate complex data architectures, there’s often a 'context window tax' i.e., developers have to manually paste vast amounts of schema metadata into prompts, eating up token limits and increasing latency. Meanwhile, data practitioners often lack guidance about how to efficiently query, optimize, and troubleshoot cloud data, while specialized, fragmented development environments cannot see across your entire data estate. Data Agent Kit helps with these challenges and others, providing the foundational capabilities data practitioners need for a new agentic way of working.

Read on for an overview of Data Agent Kit’s features and benefits, how to install it and connect your local environment to your data estate, and an intent-driven engineering example.

A unified hub for your data estate and lifecycle

Data Agent Kit makes your entire data estate available in a single view. This goes beyond providing a simple catalog for databases such as BigQuery, AlloyDB and Spanner; rather, it integrates data engineering and science tasks, orchestration pipelines, and jobs into a single interface. This allows practitioners to manage their entire data workflow — from discovery to production — without context switching. Data Agent Kit’s intelligent routing automatically chooses the optimal compute engine for your task — whether that’s BigQuery for SQL-native analytics and ELT, or Spark for custom Python transformations and distributed ML training.

Unified Hub of your entire data estate and lifecycle

Ecosystem-led intelligence: Codified agentic skills

Data Agent Kit offers a library of predefined agentic skills (e.g., ML best practices, ELT, building data apps) based on Google Cloud’s data engineering and science expertise. Rather than relying on generic LLM prompts, it codifies prescriptive guidelines into your workflow. This allows you to inject enterprise-grade data intelligence directly into your IDE or CLI.

Browsing a predefined list of agentic data engineering and science skills

Transforming data exploration through natural language

Grounded in this unified data, Data Agent Kit delivers native conversational analytics directly within your workspace, making it easy to explore your data. Powered by the same Gemini natural language to SQL technology found in our first-party agents (e.g., Conversational BigQuery and Looker), Data Agent Kit lets you run natural language queries to profile, search, and visualize your datasets.

Within Data Agent Kit, you can use Conversational Analytics to explore your data

A practical walkthrough: Unifying data and building models

To see how Data Agent Kit’s skills and MCP tools work together, consider a financial services scenario: Your company is facing rising fraud claims. With your transaction data stored in Cloud Storage, you need to build a high-confidence fraud detection model and schedule orchestration pipelines. Traditionally, this involves hours of data wrangling across multiple consoles. With the Data Agent Kit, you can complete this in minutes, directly within your IDE or CLI. Let’s see how.

Onboarding: The one-minute setup

You can get started with the Data Agent Kit in under a minute through an integrated setup process.

To do so, search for "Google Cloud Data Agent Kit" in your IDE’s marketplace (VS Code) or via the GitHub repo in your CLI (Gemini, Antigravity, Claude, Codex) from the links in the “Get started today” section below. Data Agent Kit automatically configures dependencies and checks your Google Cloud login status.

Click the Google Cloud icon in your activity bar to authenticate via IAM. Once logged in, your Cloud Storage, databases, and catalog assets appear instantly in your workspace.

Use the settings menu to set project IDs, regions, and verify MCP status to ensure all backend services are authorized. Data Agent Kit also includes a quick-start guide on using the tools and skills.

An intent-driven data engineering example

With Data Agent Kit installed, you can skip the manual ETL boilerplate, and directly describe your high-level goal to your coding assistant (e.g., Claude Code, GitHub Copilot) in natural language. The assistant leverages Data Agent Kit’s skills to plan and execute the workflow.

Prompt:

I have the raw transaction logs landing in the GCS bucket gs://fin-clearing-raw/.

First, create a Spark notebook and (1) ingest these logs into an Iceberg table in BigQuery.

Second, create a dbt project to (2) deduplicate them, (3) remove the transactions with invalid transaction id and store them in a separate Iceberg table, (4) standardize the timestamps and perform any other necessary cleanup tasks (5) sync the output to another Iceberg table (6) join this output table with tables that have payer and payees identities and write the output to a final Iceberg table.

Third, I would like you to train an ML model on Spark using a notebook to detect fraudulent transactions in the output table. I am thinking about a LightGBM model but I am open to any suggestions you might have. Use the relevant datasets in the project.

Finally, create an inferencing step using Spark notebook to the above pipeline to perform batch inferencing and write flagged transactions to a Spanner table.

Create an orchestration pipeline that first runs the ingestion then the dbt and next the inference notebook.

Under the hood: Data pipeline steps

Behind the scenes, Data Agent Kit plans a robust multi-step orchestration of the entire data lifecycle, from exploration to inference.

Step 1: Notebook creation, ingestion and initial storage

Find your bronze data — raw, unfiltered data on financial transactions — and bring it into an Iceberg table before doing the transformations.

Automatically create a Notebook to ingest the raw logs from Cloud Storage.
Write the necessary SQL, and store the ingested data into an Iceberg table in BigQuery.

Ingestion into a bronze table

Step 2: Transformation (dbt Project)

Now, clean the bronze data into silver and gold tables:

Data preparation: Deduplicate the transaction logs.
Filter invalid IDs: Identify transactions with invalid IDs and store them in a separate Iceberg table.
Clean and standardize: Standardize timestamps and perform other necessary cleanup tasks.
Sync: Output the cleaned data to another Iceberg table, leveraging the BigQuery MCP server.
Enrichment: Join the cleaned table with payer and payee identity tables.
Final output: Write the joined dataset to a final Iceberg table.

Data transformation to create silver and gold tables

Step 3: Machine learning and inferencing

With your gold table minted, it’s time for some data science: model training and inferencing. Here, the agent hands the clean data from the previous step to the model to spot fraudulent patterns.

Training: Use a Spark notebook to train an ML model.
Inference: Create a Spark notebook inferencing step for batch processing.
Storage: Write all flagged fraudulent transactions to a Spanner table by leveraging the Spanner MCP.

Machine learning and inference

Step 4: Orchestration and execution

Finally, you’re ready to move to production and schedule the whole orchestration pipeline: Ingestion -> Transformation -> Inference.

Orchestration pipelines and scheduling runs

When things go sideways: Agentic incident management and intelligent recovery

If an orchestration pipeline fails, not to worry, Data Agent Kit streamlines resolution using its intelligent incident management capabilities:

Intelligent diagnosis: Automatically conducts root cause analysis to pinpoint failure sources
Autonomous remediation: Drafts and tests fixes, bypassing manual debugging
Automated recovery: Validates and deploys fixes via automated Git workflows

Issue diagnosis and remediation

And there you have it: You’ve gone from raw discovery to a fully automated, fraud-catching machine in a matter of minutes, all from within the same UX. No need to hop between multiple browser tabs, IDE interfaces, or learn data engineering and science best practices — Data Agent Kit orchestrates a clean end-to-end flow leveraging various MCP tools and codified skills. Ultimately, this approach helps you achieve what matters most: shipping innovative, high-performance data applications at scale.

Get started today

Data Agent Kit is available today in preview. Start by installing it in your favorite IDE or CLI:

Then visit the documentation to learn more and get started.

Beyond the Query: 5 Scenarios Laying the Foundation for the Agentic Era

Mon, 18 May 2026 16:00:00 +0000

Accessing enterprise data is shifting from static reports to dynamic use by autonomous systems. To keep up, organizations must route fragmented data from SaaS, IoT, and legacy sources into secure, scalable endpoints.

However, moving to AI-driven exposure requires more than just connecting an LLM to a database, it requires a fundamental architectural shift to manage security, costs, and semantic accuracy.

What we’ll cover

This article explores the technical evolution of data exposure through five architectural patterns: from manual SQL development to autonomous workflows standardized by the Model Context Protocol (MCP).

While the examples use BigQuery and mocked CRM data, the patterns apply to most enterprise data assets transitioning into an agentic workflow.

The 5 Scenarios of Data Evolution

The transition from static reports to agentic insights is defined by two factors: Trust and complexity.

Trust dictates autonomy: Low-trust environments (like external client-facing apps) require deterministic, hard-coded logic to prevent errors. High-trust environments (like internal tools for power users) allow for probabilistic LLM reasoning, where there is more tolerance for non-deterministic outputs.

Complexity defines utility: Simple lookups need fast, cached responses. In contrast, complex, cross-functional problems require an agent to orchestrate multiple tools and data sources.

To navigate this shift, we will examine five technical scenarios, starting with the baseline of the static API.

Scenario 1: The Static API Contract

Focus: Maximum stability and deterministic execution

Scenario 1 represents the traditional model of data exposure. A developer acts as the intermediary, translating specific business requirements—such as "Show me top-selling products by category"—into optimized, hard-coded SQL queries.

Isolation and Predictability

This approach provides the highest level of security and performance:

Low logic risk: Because the SQL is pre-written and vetted, there is no risk of a user (or an agent) crafting a query that accesses unauthorized data.
Secure by design: Using parameterized queries instead of string concatenation provides a hard barrier against SQL injection.
Reliability: The output is deterministic. If the development lifecycle is robust, the user is guaranteed to receive exactly what they requested, with predictable execution costs and performance.

Implementation example

This snippet demonstrates the baseline for data exposure: a direct, static API contract. It offers maximum predictability by using parameterized queries to prevent SQL injection and ensure consistent performance.

A note on the code examples: To prioritize architectural clarity, these examples are provided as conceptual blueprints rather than production-ready code. They are designed for pedagogical purposes and intentionally omit "industrial" requirements such as persistent session state, IAM/Auth protocols, and comprehensive exception handling. Use these only as a logic guide before implementing your own hardened and production-ready solution.

code_block: <ListValue: [StructValue([('code', 'from google.cloud import bigquery\r\ndef fetch_products(limit=10):\r\n client = bigquery.Client()\r\n # Use named parameters to ensure security and prevent SQL injection\r\n sql = """\r\n SELECT id, name \r\n FROM `bigquery-public-data.thelook_ecommerce.products` \r\n LIMIT @limit\r\n """\r\n job_config = bigquery.QueryJobConfig(\r\n query_parameters=[\r\n bigquery.ScalarQueryParameter("limit", "INT64", limit)\r\n ]\r\n )\r\n return client.query(sql, job_config=job_config).to_dataframe()'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4a41557730>)])]>

Analysis

Parameter	Rating	Impact
Flexibility	Low	Users cannot change the query logic or filters without code changes.
Cost Control	High	Query plans are static; costs are predictable and easy to budget.
Latency	Low	Low response times leveraging for example BigQuery's query cache.
Maintenance	High	Every new business question requires a developer and a deployment.

When to use Scenario 1?

This approach is the benchmark for external-facing applications, customer portals, and high-traffic production dashboards. It is the best choice when your requirements include:

Strict auditability: You need a version-controlled (Git-based) history of every query executed against your data warehouse.
Performance at scale: You require sub-second latency, leveraging BigQuery’s result caching for high-concurrency workloads.
Deterministic logic: You must guarantee that specific inputs always produce the exact same output, with no room for AI-driven variability.
External multi-tenancy: You are exposing data to third parties and need absolute assurance against data cross-contamination.

Scenario 2: Custom Agent with SQL Generation

Focus: User flexibility and managed autonomy.

To resolve the development bottleneck of manual SQL authoring, Scenario 2 introduces an LLM agent (via the Agent Platform SDK) to act as a dynamic translator. In this model, the developer stops writing individual queries and starts focusing on metadata documentation.

From Query Writing to Metadata Curation

Using the Agent Platform SDK (for Python, for example), developers implement a reasoning engine that maps natural language to schema metadata. Rather than "guessing" the SQL, the agent follows a structured reasoning loop:

Analyze: It parses the natural language intent (e.g., "Which region had the highest growth?").
Retrieve: It looks up the relevant schema metadata provided in the system context.
Construct: It generates a syntactically correct, BigQuery-compatible statement.

For the LLM to generate accurate queries, it must "see" the data structures. You provide this through system instructions that include table names, column types, and—crucially—semantic descriptions (e.g., "created_at: The timestamp when the user first registered"). By curating this metadata space, you define the boundaries of what the agent can explore and execute.

Access control relies entirely on underlying database permissions (like RLS). Because the agent passes generated SQL dynamically, data boundaries must be enforced at the database level.

Implementation example

This marks the first step into agentic workflows, where an LLM acts as a translator between natural language and structured schema.

code_block: <ListValue: [StructValue([('code', 'from google.cloud import bigquery\r\nfrom vertexai.generative_models import GenerativeModel\r\n\r\ndef ai_query(user_prompt):\r\n # Initialize the model\r\n model = GenerativeModel("YOUR_LLM_MODEL")\r\n \r\n # SYSTEM CONTEXT: Grounding the model with schema metadata\r\n # This prevents the AI from guessing table names or column types.\r\n system_instruction = (\r\n "You are a BigQuery SQL expert. Output ONLY raw SQL code without markdown backticks. "\r\n "Context: The \'products\' table in \'bigquery-public-data.thelook_ecommerce\' "\r\n "contains: id (INT), name (STRING), and category (STRING)."\r\n )\r\n \r\n full_prompt = f"{system_instruction}\\n\\nUser request: {user_prompt}"\r\n \r\n # Generate the SQL string\r\n response = model.generate_content(full_prompt)\r\n sql_code = response.text.strip().replace("```sql", "").replace("```", "")\r\n \r\n # Execute the AI-generated query\r\n client = bigquery.Client()\r\n return client.query(sql_code).to_dataframe()'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4a41557d90>)])]>

Analysis

Parameter	Rating	Impact
Flexibility	High	Users can ask virtually any question in plain English.
Cost Control	Low	LLMs may generate unoptimized queries (e.g., missing partitions).
Latency	Medium	Includes LLM "thinking" time.
Maintenance	Medium	Developers manage "prompt schemas" rather than SQL code.

When to use Scenario 2?

Scenario 2 is best suited for internal data discovery and analyst-led exploration. It bridges the gap between raw data and business users when you require:

High-variability querying: When the range of potential business questions is too broad (the "infinite question space") to be efficiently covered by pre-built, static APIs.
Rapid prototyping: When analysts need to quickly explore datasets and validate hypotheses before committing to the development of formal, production-grade dashboards.
Semantic interpretation: When you need an agent to resolve natural language ambiguities—such as mapping "last quarter" or "active users"—into specific, technical filter criteria automatically.

Scenario 3: Conversational Analytics

Focus: Managed reasoning and verified logic.

Scenario 3 shifts the responsibility from a self-managed custom agent to a specialized, platform-native reasoning engine. By leveraging the Conversational Analytics API (currently in Pre-GA), you can deploy Data Agents - intelligent, governed layers that use enterprise-specific metadata and verified SQL to keep the LLM within strictly defined guardrails. This API translates natural language into precise queries across BigQuery, Looker, and Data Studio, while extending support to Google Cloud’s primary database solutions. We’ll consider BigQuery as our primary example for exploring these conversational insights.

The Power of Verified Queries

Unlike generic LLM prompts that guess the SQL structure, these agents are grounded in your organization’s source of truth:

Verified queries: You provide a library of verified queries (vetted, high-quality SQL examples) that the agent uses as a reference for complex joins and business logic. This ensures the agent follows your established coding patterns.
Managed context: The platform handles the retrieval of schema information and documentation, reducing the prompt bloat that often leads to hallucinations in custom-built agents.
Aligned outputs: By grounding the model in existing production SQL, the system ensures that AI-generated insights remain consistent with your official reporting metrics.

This solution inherits existing BigQuery IAM permissions and provides a view of the reasoning and SQL behind every answer.

Can all of this be done with enough work on a fully customized agent? Yes. Is the custom approach practical, and time/cost efficient? Maybe not.

Implementation example

This approach leverages a specialized reasoning engine to handle intent discovery and data grounding. The developer no longer manages the translation logic: they simply call the managed agent.

code_block: <ListValue: [StructValue([('code', 'from google.cloud import geminidataanalytics_v1beta as gda\r\ndef chat_data(user_query):\r\n # Initialize the client for the Data Agent service\r\n client = gda.DataAgentServiceClient() \r\n # Path to your pre-configured Data Agent resource\r\n agent_path = "projects/YOUR_PROJECT_ID/locations/us/dataAgents/YOUR_AGENT_ID"\r\n # Execute: The agent uses its "Verified Queries" and metadata to find the answer\r\n request = gda.ExecuteDataAgentRequest(name=agent_path, query=user_query)\r\n response = client.execute_data_agent(request=request)\r\n \r\n # The agent returns both the natural language answer and the supporting data\r\n return response.answer'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4a433af520>)])]>

Analysis

Parameter	Rating	Impact
Flexibility	Medium	High for the data sources it knows, but restricted by its Verified instructions and metadata scope.
Cost Control	Medium	Grounded queries are typically more efficient than raw LLM generation.
Latency	Medium	Higher than static queries, due to the multi-stage reasoning and summarization process.
Maintenance	Low	Managed by Google; analysts focus on coaching the agent through metadata and verified SQL.

When to use Scenario 3?

Scenario 3 is the ideal path for BigQuery-centric analysis where accuracy is non-negotiable. Choose this when you require:

Governed trust: Business logic (e.g., "Revenue") must follow pre-vetted verified queries every time.
Native intelligence: Users need to perform complex tasks like forecasting or anomaly detection via BigQuery AI using natural language.
Auditability: Stakeholders require a transparent reasoning path to see exactly how the AI arrived at its numbers.

While Scenario 2 requires building a custom reasoning engine from scratch, Scenario 3 provides a platform-native experience that prioritizes verified logic over raw LLM generation.

The limitation: This data companion is ultimately confined to the BigQuery or Google Cloud ecosystem. To scale an agentic workforce across heterogeneous platforms and tools, we must look toward vendor-agnostic standards like the Model Context Protocol (MCP).

Scenario 4: Managed MCP Tools

Focus: Standardized connectivity and decoupled architecture.

Scenario 4 introduces the Model Context Protocol (MCP)—an open-source standard designed to normalize how AI applications interact with data and tools. While previous scenarios rely on custom SDKs or platform-specific APIs, MCP provides a universal interface that separates the reasoning layer from the tool execution layer.

Standardized Abstraction

MCP enables tool discovery by exposing a manifest of capabilities that any compliant agent can ingest. This allows for a modular system where the data logic is "externalized" from the agent itself.

The MCP client: The reasoning engine (the LLM) that identifies the user's intent. Because it uses a standardized protocol, the client can connect to any MCP server and instantly discover what it can do without needing new integration code.
The MCP server: The domain-specific service that exposes data and logic. The managed BigQuery MCP server doesn't just pass queries: it encapsulates the logic required to navigate Google Cloud’s infrastructure safely. It exposes tools such as:

list_dataset_ids: Context-aware discovery of the data environment.
get_dataset_info: Metadata retrieval for semantic grounding.
execute_sql: Controlled execution of data retrieval.

(see https://docs.cloud.google.com/bigquery/docs/reference/mcp for the updated toolset).

Access control is managed via standard IAM service accounts and lacks programmatic logic-checks.

This decoupling future-proofs your AI stack. You can swap your LLM provider or upgrade your agent's reasoning model without rewriting the data access logic, because the interface between them remains consistent and governed.

Implementation example

In an MCP-based architecture, connecting an AI agent to a data source is reduced to a simple configuration handshake. Instead of writing custom integration logic, you provide an MCP-compliant client (such as the Gemini CLI or a modern IDE) with a manifest defining the server’s location and security requirements.

The following manifest allows the client to connect to Google’s managed BigQuery MCP server, enabling it to dynamically discover and execute data tools without a single line of custom code:

code_block: <ListValue: [StructValue([('code', '{\r\n "mcpServers": {\r\n "bigquery": {\r\n "httpUrl": "https://bigquery.googleapis.com/mcp",\r\n "authProviderType": "google_credentials",\r\n "oauth": {\r\n "scopes": [\r\n "https://www.googleapis.com/auth/bigquery"\r\n ]\r\n }\r\n }\r\n }\r\n}'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4a433af6d0>)])]>

Analysis

Parameter	Rating	Impact
Flexibility	High	Agents can contextually explore any table the MCP server exposes.
Cost Control	Medium	Tools are standardized, but a curious agent can still trigger large scans.
Latency	Medium	Includes standard overhead for the protocol handshake and tool-calling.
Maintenance	Low	Uses a managed MCP Server which requires no maintenance. The work is only on the MCP client.

When to use Scenario 4?

Scenario 4 is the architectural choice for multi-agent environments that require standardized data connectivity with minimal maintenance overhead. It is the ideal path when you require:

Managed infrastructure: You want to offload the security, execution, and maintenance of your toolset by consuming a managed BigQuery MCP server rather than building and patching custom data-retrieval code.
LLM portability: You need an open-standard interface, allowing you to use the same tools across different LLMs or agent frameworks without rewriting proprietary function calls.
Autonomous discovery: Your agents must navigate and inspect complex datasets dynamically. MCP’s standardized endpoints allow agents to crawl metadata and schema information autonomously to determine the best path for a query.

Scenario 5: Custom Hosted MCP Servers

Focus: Architectural extensibility and custom tool definition.

Scenario 5 takes the standardized connectivity of Scenario 4 and adds complete control by replacing the managed service with a custom-built MCP server. Typically hosted on scalable infrastructure like Cloud Run, you can rely on open source solutions such as MCP toolbox. This approach removes the guardrails of managed offerings, granting engineering teams full freedom to define specialized tools, integrate disparate third-party APIs, and implement proprietary execution logic within the protocol.

Architectural Advantages of Custom MCP

Shifting to a custom-hosted MCP server moves operational complexity from the LLM prompt to the server-side logic, unlocking three critical capabilities:

Deterministic tool tailoring: Instead of forcing an agent to navigate raw, sprawling schemas, developers define high-level functions with specific data shapes. This replaces probabilistic SQL generation with deterministic execution, virtually eliminating schema-based hallucinations.
Unified source orchestration: A custom MCP server acts as a consolidated gateway. Within a single tool execution, the server can orchestrate calls across BigQuery, external SaaS APIs, and legacy on-premises systems. The agent receives a pre-processed, unified response, abstracting away the multi-source complexity.
Programmable governance: This scenario enables code-level security difficult to implement in managed environments. You can implement granular controls directly within the protocol layer, such as:

Dynamic PII masking: Automatically redacting sensitive data before it reaches the LLM.
Custom authentication: Injecting enterprise-specific middleware.
Contextual rate limiting: Throttling tool usage based on the end-user’s identity or cost center.

Implementation example

In this scenario, when using MCP toolbox, you use a declarative tools.yaml file to define the interface of your custom MCP server. This file acts as the absolute boundary for your agent—it defines the BigQuery connection, enables safe discovery for schema inspection, and wraps complex, multi-table joins into a single, parameterized tool.

code_block: <ListValue: [StructValue([('code', '# ----------------------------------------------------------------------\r\n# Minimal Configuration\r\n# Dataset: bigquery-public-data.thelook_ecommerce\r\n# ----------------------------------------------------------------------\r\n\r\nsources:\r\n bq-thelook-ecommerce:\r\n kind: "bigquery"\r\n project: "${PROJECT_ID}"\r\n location: "${BQ_LOCATION}"\r\n\r\ntools:\r\n # 1. Discovery Tool: Helps the agent understand the database schema\r\n bigquery_get_table_info:\r\n kind: bigquery-get-table-info\r\n source: bq-thelook-ecommerce\r\n description: Retrieves table metadata and schema details. Run this before executing custom queries.\r\n\r\n # 2. Execution Tool: Parameterized SQL for safe, repeatable data fetches\r\n thelook_get_user_orders_summary:\r\n kind: bigquery-sql\r\n source: bq-thelook-ecommerce\r\n statement: |\r\n SELECT\r\n orders.user_id,\r\n COUNT(DISTINCT orders.order_id) AS count_of_orders,\r\n COUNT(order_items.id) AS count_of_items,\r\n SAFE_DIVIDE(COUNT(order_items.id), COUNT(DISTINCT orders.order_id)) AS avg_items_per_order\r\n FROM `bigquery-public-data.thelook_ecommerce.orders` AS orders\r\n INNER JOIN `bigquery-public-data.thelook_ecommerce.order_items` AS order_items\r\n ON orders.order_id = order_items.order_id \r\n AND orders.user_id = order_items.user_id\r\n WHERE orders.status = "Complete" \r\n AND orders.user_id = @user_id\r\n GROUP BY orders.user_id;\r\n description: Retrieves an order summary for a specific user ID, including total completed orders, items purchased, and average items per order.\r\n parameters:\r\n - name: user_id\r\n type: integer\r\n description: The unique identifier of the user.\r\n\r\ntoolsets:\r\n # Binds the tools together for agent use\r\n thelook_core_insights_toolset:\r\n - bigquery_get_table_info\r\n - thelook_get_user_orders_summary'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4a433afee0>)])]>

Analysis

Parameter	Rating	Impact
Flexibility	High	Supports cross-domain orchestration (e.g., BigQuery + legacy APIs) and unlimited custom tool definitions.
Cost Control	High	Allows developers to inject programmatic query cost estimation and budget thresholds prior to execution.
Latency	High	Custom multi-hop orchestration, network transit, and container cold-starts introduce latency.
Maintenance	High	Requires full ownership of the application lifecycle, including CI/CD, dependency patching, and container scaling.

When to use Scenario 5?

This architecture is the power user choice, essential for highly regulated environments and hybrid infrastructures where managed services fall short. Implement this approach when your design requires:

Secure hybrid orchestration: You must bridge BigQuery with private on-premises systems or restricted APIs, returning a pre-processed, consolidated payload that the agent can use immediately without navigating the raw network gap.
Hardened business logic: You need to move complex, non-negotiable calculations off the LLM and into a controlled code environment, exposing only high-level "expert" tools to guarantee absolute accuracy.
Centralized enterprise tooling: You want to maintain a single, governed source of truth for your proprietary tools that can be served uniformly across different LLM providers or internal frameworks without vendor lock-in.

Conclusion: The Foundation of the Agentic Era

The journey from Scenario 1 to Scenario 5 traces a clear technical evolution: we are moving away from rigid, hard-coded data silos and toward a world of autonomous discovery and standardized connectivity. By adopting frameworks like the Model Context Protocol (MCP), organizations can decouple their data logic from their AI models, ensuring that as LLMs evolve, their access to the enterprise "brain" remains seamless, scalable, and vendor-agnostic.

However, increased autonomy does not mean decreased oversight. While we haven’t touched on these points in depth in this article, we must adhere to a fundamental truth: data access must be governed and controlled using governance and security tools. Regardless of the access scenario—more or less agentic depending on the use case—security, credentials, quality management, and standardized governance are absolutely essential.

On a more lighthearted note, it’s worth remembering that the golden rule of computing still applies: "Garbage In, Garbage Out". You can build the most sophisticated, autonomous agentic layer in the world, but if you feed it messy, uncurated data, you’ll simply get "garbage" answers at a much faster and more confident pace. Sophisticated AI doesn't fix bad data: it just makes it more visible. Maintaining high data quality is not just a legacy requirement—it is the fuel that makes the agentic engine actually work.

How Google Does It: Fleet-wide, large-scale A/B experimentation

Mon, 18 May 2026 16:00:00 +0000

When most people think of A/B experimentation, they think of button colors, landing page layouts, or checkout flows. At Google, many fundamental infrastructure improvements also need the rigor of A/B experimentation. Optimizing a memory allocator or a kernel scheduler can unlock massive savings in compute resources and slash latency for millions of users. But experimenting with such critical changes is inherently risky; a buggy kernel update doesn't just result in an unhappy user, it can take down large swaths of machines. To innovate safely and at scale, you must perform A/B experimentation on the infrastructure itself.

In this blog post, we summarize the key lessons from Google's A/B experimentation methodology that we’ve refined over multiple years. This blog post is meant as a resource of best practices to follow and initiate a broader discussion around this topic (similar to our performance tips series). Specifically, we highlight four pillars of Google’s A/B experimentation infrastructure:

Application-level vs. machine-level experimentation
Maintaining a balanced setup
Ensuring binary hermeticity
Selecting the right performance metrics

In the following section, we explore why these aspects are critical and how we have handled these at Google.

Where infrastructure experiments matter

Experiments targeting the core building blocks of your stack — such as the operating system, core libraries, compilers and the cluster management system — help unlock performance and efficiency gains that application experiments simply cannot achieve alone. These experiments provide a reliable and safe way to measure impact at a massive scale using a representative subset of the fleet. This helps inform which optimizations are worthwhile.

At Google, we see high value in optimizing the following specific components:

Core libraries: Optimizing core libraries, such as TCMalloc, has a profound impact on all binaries running in the fleet.
Compiler: Tweaking different compilation and build flags can yield fleet-wide performance gains without changing a single line of source code.
Kernel: Improving kernel subsystems, such as memory management or scheduling, can dramatically boost machine efficiency across the fleet, while also helping to uncover large-scale regressions.
Cluster management system: Decisions made by entities like kube-scheduler and kubelet have a significant impact on locality and antagonism, which in turn impact cluster utilization and performance.

Scale of improvements

It’s easy to measure changes that bring large improvements, such as rewriting an entire memory allocator for a 2x efficiency gain. At Google scale, however, the reality is a lot of our infrastructure improvements are much smaller — typically sub-1% gains. But while an individual change may seem minor, a sustained sequence of small optimizations accumulates with time, leading to moonshot-like returns.

Achieving these sub-1% gains demands careful thinking about experimentation and measurement. To do so, we built a robust framework for reliably performing A/B experiments and measuring the impact of small changes.

The limitations of application-level experimentation

To evaluate infrastructure changes, you could enable the change on a specific set of applications and observe metric shifts. However, this application-centric approach has several critical drawbacks:

Selection bias: Individual applications may not be suitable for evaluating specific changes. For instance, an application that allocates memory only during startup is a poor candidate for testing updates to a memory allocation library.
Lack of fleet representation: A small group of applications rarely reflects the behavior of the entire fleet, leading to inaccurate estimates of potential improvements.
Invisible system-wide benefits: Measuring isolated applications fails to capture concomitant effects. For instance, a change that improves hardware cache performance may benefit all applications running on a machine, not just the one being measured.
Technical constraints: Fundamental system changes, such as those made to the kernel or cluster scheduler, simply cannot be evaluated effectively through individual applications alone.

Our approach: Machine-level experimentation

At Google, we overcome these hurdles by enabling changes on individual machines rather than specific applications. All workloads running on a chosen machine actuate the change, allowing us to measure the impact across the entire fleet. This approach captures concomitant effects for all co-located applications and enables the evaluation of system-specific changes that application-level tests would miss.

Infrastructure experiments at Google

A typical experiment selects 1% of the fleet for both the experiment and control groups. The experiment is then gradually rolled out in waves following internal best practices. During the rollout, the framework starts collecting data for both groups, and continuously analyzes it to measure impact on performance and detect production regressions.

The importance of a balanced setup

The validity of an infrastructure experiment hinges on how you select machines. You need to have balanced experiment and control groups, each targeting a representative subset of your machine fleet. Google’s fleet has several different machine types. The proportion of these machine types in the experiment and the control groups should closely match. When the machine types in the experiment and control groups did not closely match in just two clusters, we noticed a 0.2-0.3% data skew! For any sub-1% improvement, this is enough to invalidate the result.

The experiment and the control groups must also not be too large, or you risk the infrastructure’s overall reliability. Nor should they be too small, leading to statistically insignificant data. At Google, we concluded that a 1% subset of fleetwide machines hits the sweet spot.

Implementation-wise, we select 1% subsets of the fleet with a proportional representation of different generations of machines in each cluster. All these subsets are completely equivalent to one another and are rebalanced periodically using linear programming to limit the churn. When we deploy an experiment, two subsets are picked to serve as the experiment and control groups. Both groups roll out at the same pace so a balanced A/B analysis can be performed at any time.

Ensuring reliability with binary hermeticity

For an experiment that modifies the behavior of a library, binaries must be recompiled with the experimental library change. Crucially, the experimental logic only activates when the binary is running on a machine in the experiment group.

Our experimentation framework includes a critical safety requirement designed to ensure that experiment rollbacks are simple and reliable. Any experiment that alters the behavior of individual binaries must follow a two-step rollout process:

First, the experiment is rolled out to all machines in the experiment and control groups.
Only then should the experimental library change be submitted, so that newly compiled binaries activate the experiment when they are deployed to the experiment machines.

This sequential approach makes it easy to tie any behavioral changes to binary releases. More importantly, it guarantees that rolling back to the previous version of the binary safely and immediately undoes the experiment.

What happens if you don’t follow this rule? When experiments cause production outages, it becomes extremely difficult to debug and mitigate them. To fully remove the experiment, you would need to roll back the machines and restart all the affected binaries. This delay could significantly slow down outage mitigation and could potentially cause even more damage.

To illustrate the risks, imagine running a core library optimization experiment that causes sporadic memory corruption. If the rollout steps above are switched, newly built binaries will contain the experimental change before the machine-level rollout even begins. Owners might incorrectly believe that these binaries are safe, since the experiment has not been activated yet. However, once the machine rollout starts, only the binaries built after the library change will show memory corruption, but others would not. This staggered failure makes it incredibly difficult to isolate the experiment as the root cause. If the owners attempt to roll back to the previous version of the binary, the corruption will persist, making incident response less effective. The sudden appearance of memory corruption and the search process needed for the right version of the binary complicates debugging.

Measuring what matters

Infrastructure experiments do not care about click-through rates, where short-term boosts may come at the cost of long term engagement. Instead, the focus is on the performance and the health of the applications and machines, as measured by:

Application productivity: At Google, we internally debated about the best metric to accurately measure performance. It is well known in academia that Instructions Per Cycle (IPC) is not a suitable metric for multiprocessor/multiprogrammed workloads. Instead, we opted for a robust, application-defined productivity metric that captures the amount of work done by the application. For example, a search web server reports the number of search queries completed per second as its productivity metric.
Machine-level performance: While application productivity is our gold standard, we also measure other metrics like IPC, cache misses, and memory bandwidth to corroborate performance improvements.
Reliability: We track multiple reliability metrics, including abnormal terminations and machine timeouts. Sure, a new kernel might be faster, but if it introduces new crashes, it is considered a failure and must be fixed.

We periodically collect these metrics from all applications on all machines in our fleet.

Statistical tools

Running experiments and collecting data is only half of evaluating an optimization. A significant challenge lies in analyzing the collected data to understand how a change will truly impact the fleet before a full rollout. With thousands of jobs running in our data centers at any one moment, any single change has a varied effect on job performance. Further, it’s impossible to make an informed decision based on just a handful of jobs. To overcome this, we developed advanced statistical tools that meticulously match jobs running on the experiment group with comparable jobs on the control group. We then compare the metrics from these matched pairs and aggregate the results across all jobs to generate reliable metrics about the entire fleet. This comparison is done using data spanning several weeks to ensure that daily fluctuations do not skew our findings.

Furthermore, we regularly study data from A/A experiments to understand the variance caused by daily fluctuations. The data from these A/A experiments is used to establish a “noise floor.” Only changes that produce results significantly above this noise floor are considered robust and worthy of being rolled out.

Summary

As cloud infrastructure continues to expand, squeezing every bit of efficiency out of our resources is no longer just an advantage — it is necessary for sustainable cost management. This demand for efficiency requires a rigorous, data-driven approach to validate new optimizations, making robust, fleet-wide A/B experimentation infrastructure essential. However, this presents substantial challenges, encompassing configuration management, reliability, and statistical analysis — none of which are exclusive to our environment. By opening up about the infrastructure we have built at Google, we aim to spark a new wave of research and collaboration. We hope that sharing these hard-earned lessons will:

Offer researchers and practitioners a window into the intricate, high-stakes world of infrastructure-level experimentation
Provide a proven blueprint that others can adapt and improve upon within their own environments
Ignite bold new innovations that push the boundaries of what's possible in systems performance.

What we announced in streaming AI at Next ‘26

Mon, 18 May 2026 16:00:00 +0000

Every device, user, and microservice generates data. Ingesting this data, extracting meaning and insights, and driving business decisions in real time has the potential to deliver transformational business value.The rise of agentic AI represents an opportunity for users to overcome the challenges inherent in real-time analytics. But while agentic AI has the potential to accelerate adoption, users face a new set of challenges with effectively leveraging real-time data:

Real-time context is hard to implement. Teams will choose to incorporate data from batch-oriented approaches, like periodic database syncs and scheduled refreshes. Agents have to either rely on stale data or require memory-intensive context windows. This “context lag” makes them ineffective for real-time agentic tasks like fraud detection, dynamic e-commerce recommendations, or autonomous supply chain adjustments.
Real-time systems are inflexible. Agentic tools lack the modularity to adapt to customer-specific requirements, forcing organizations to make difficult architectural choices. Data practitioners need a platform to meet them where they are, where they are free to make the tradeoff between latency, accuracy, and cost.

Google Cloud provides a tightly integrated, unified streaming data platform that delivers both fully managed, Google Cloud-native services, as well as open-source-compatible services, and that come together to power large-scale AI training and inference. The platform is comprised of five key services:

Pub/Sub: Highly reliable, serverless, and fully managed service for messaging and event streaming that’s integrated with BigQuery, Dataflow, and Cloud Storage. Pub/Sub is utilized by organizations like Anthropic.
Dataflow: A serverless engine for batch, streaming, and now agentic AI. Leading enterprise organizations like Palo Alto Networks use Dataflow, as do Google services like Waymo and Google Maps. For instance, Waymo cars use Dataflow to help it “see” the world, plan their routes, and predict obstacles. Before a car hits the actual pavement, it “drives” millions of miles in a simulator, with Dataflow generating training datasets and validating the models that are used for autonomous driving.
Managed Service for Apache Kafka: The fully managed way to run the popular open source streaming storage and data integration system on Google Cloud that’s highly reliable, secure, and cost efficient. Across the largest enterprises and startups, Apache Kafka serves as a staging location for critical training data and real time updates to AI agent context.
BigQuery: A unified platform for real-time ingestion and analysis. The Storage Write API provides high-throughput streaming into BigQuery and Lakehouse for Apache Iceberg tables with exactly-once delivery semantics and stream-level transactions. Additionally, BigQuery continuous queries enable real-time AI inference directly within the data pipeline by calling generative functions like AI.GENERATE_TEXT, allowing for immediate insights as data is ingested.
Bigtable: Google’s NoSQL real-time database for processing streaming data from Pub/Sub and Dataflow automatically using continuous materialized views, delivering results in seconds that are ready for low-latency serving using Bigtable’s in-memory tier.

Moving from insight to autonomous action

At Google Cloud Next, we announced a set of streaming AI capabilities to the Agentic Data Cloud, providing autonomous agents with instant context and enabling real-time actions, helping organizations feed real-time context to their AI agents.

For instance, imagine a supply chain agent that doesn't just monitor IoT data, but autonomously reroutes a shipment around bad weather, confirms new delivery windows with the receiving warehouse, and updates the customer's portal — all before a human supervisor is even aware of the problem. Consider a financial services agent that identifies a fraudulent transaction pattern, instantly freezes the account, communicates with the customer via their preferred channel, and initiates a new card shipment — all within seconds of the suspicious activity. Whether you’re creating embeddings on streaming data to power search, or building a sophisticated multi-agent fraud detection system, these new capabilities add powerful tools to your toolbox.

Let’s take a closer look at these new capabilities.

New streaming AI capabilities

At Next ‘26, we launched tightly integrated capabilities to our platform across three key areas:

1. Providing real-time, enriched context for agents

1.1. Pub/Sub AI Inference SMT (GA): You can now run inference on messages streamed through Pub/Sub. Data practitioners can choose any models available on Gemini Enterprise Agent Platform. Pub/Sub makes the inference call and appends the result to each message before sending it downstream, bringing Pub/Sub’s simplicity together with the Gemini Enterprise’s fully managed tools.

1.2. Pub/Sub Bigtable subscriptions (Preview): Stream Pub/Sub data directly to Bigtable. Pub/Sub Bigtable subscriptions directly materialize event data from a Pub/Sub topic into a Bigtable table, eliminating the need for custom pipelines and dramatically simplifying your streaming architecture. For instance, you can easily ingest vector embeddings into Bigtable to power semantic search workloads.

1.3. BigQuery continuous queries stateful data processing (Preview): BigQuery continuous queries can now perform complex correlations between multiple data streams using JOINs and calculate metrics over consistent time intervals with tumbling window aggregations. This enables sophisticated analysis, such as calculating 30-minute averages or correlating events across different streams, directly as data is ingested into BigQuery. Furthermore, you can integrate AI directly into your data pipelines by calling generative functions like AI.GENERATE_TEXT, as well as materialize continuous query SQL results into BigQuery tables or export them to operational sinks like Bigtable, Spanner, and Pub/Sub for real-time reverse ETL.

2. Direct agents to manage your resources

2.1. Model Context Protocol (MCP) support for Pub/Sub, Managed service for Apache Kafka, Bigtable and BigQuery (GA): Your agents can manage Pub/Sub,Managed service for Apache Kafka services, and BigQuery using fully managed MCP endpoints. Agents can also publish messages to Pub/Sub.

2.2. ADK integration (GA): Your agents can interact with your real-time data stored in Pub/Sub, Bigtable, BigQuery, or other Google Cloud services using pre-built ADK integrations. Developers can build agents acting on real-time context without having to implement complex configurations or plumbing.

3. Combine multi-agent systems with your data processing

3.1. Event-driven autonomous agents: As agents become core to our workflows, real-time data pipelines must evolve to incorporate them directly into the stream. We have enabled this capability by treating agentic logic as a first-class citizen within the Dataflow pipeline. You can now incorporate your agent code using the Agent Development Kit (ADK) and deploy it as a specialized node using the RunInference transform and the new ADKAgentModelHandler. Key advantages of this approach include:

- Massive scalability: Leverage Dataflow’s architecture to process high velocity events upstream and keep hundreds of agents sessions active simultaneously, each driven by specific incoming events.
- Pre-processing power: Dataflow handles the heavy lifting of complex data enrichment, delivering a “ready-to-act” context directly to the agent so it can focus on reasoning.

3.2. Dataflow Unified embeddings Sinks: We are introducing unified embedding generation directly within the data stream to eliminate “context lag”. You can now transform incoming data into high-dimensional vectors at low latency using Dataflow. These real-time embeddings are then seamlessly materialized into our expanded suite of high-throughput vector sinks, which now includes Cloud Spanner (featuring its new built-in vector search) and AlloyDB, providing you with an up to date vector database for semantic search needs as well as for your autonomous agents making RAG calls with an instantly searchable and perfectly synchronized long-term memory. This feature works with both remote and local models, for example Gemma.

As we continue to build out the platform, customers can expect to see even tighter integrations and more powerful capabilities. We look forward to seeing what you build with these new capabilities.

Gemini Live Agent Challenge: Announcing the winners and highlights

Fri, 15 May 2026 16:00:00 +0000

The Gemini Live Agent Challenge is officially in the books! We challenged developers worldwide to break out of the traditional 'text box' paradigm by building next-generation AI agents. From our initial announcement to amassing 11,878 participants and 1,536 submitted projects from 151 countries, the results were nothing short of spectacular.

The mission was to seamlessly integrate multimodal capabilities—building agents that help you see, hear, speak, and create in real time — using the Gemini Live API, the Agent Development Kit (ADK), and the robust infrastructure of Google Cloud. Participants pushed the boundaries of interactive AI across three distinct categories: The Live Agent, The Creative Storyteller, and The UI Navigator.

Congratulations to the builders who took home the top prizes! These winning teams combined technical precision with bold imagination, completely redefining how users can interact with and experience agents. Two of these standout developers were even recognized in person at Google Cloud Next 2026. Here’s a look at their experience, alongside the complete list of winning agents.

Celebrating our category winners at Google Cloud Next ‘26

Category winners Jeremiah Somoine and Bryen Param were invited to attend Google Cloud Next 2026 in Las Vegas, where they shared their experiences and insights with the broader developer community. Both winners presented Lightning Talks at the Developer Theatre on the expo floor and sat down for exclusive interviews in the Creator Studio Pod at the GDE and Certified Lounge.

During his time at the event, Bryen discussed the core inspiration behind drone-copilot. He explained that his project was driven by the question of "what if a model could interact with the real world?", showcasing how multimodal capabilities can bridge the gap between AI and physical environments.

Jeremiah, currently a college student, reflected on the development process behind Sankofa, noting that "the best response to a technical limitation was a creative one." When asked what advice he would give to other students looking to build the next generation of AI applications, he emphasized the importance of jumping at any opportunity to get hands-on with the technology. "The best way to learn is by doing," he said, encouraging aspiring developers to simply dive in and start building.

Winners

Grand Prize winner: ORION - Operating Room Intelligent Orchestration Node
By: Aditya Shukla

ORION, or Operating Room Intelligent Orchestration Node, is a voice-directed surgical co-pilot for robotic surgery. Surgeons can speak naturally and instantly receive answers, live data on display, and real-time visual assistance - all without breaking scrub.

The Live Agent winner: drone-copilot
By: Bryen Param

Drone-copilot transforms how users interact with hardware by enabling natural, real-time conversations with a drone instead of using a joystick or complex menus. Simply by speaking, users can instruct the drone to navigate, perform autonomous visual inspections, or describe its surroundings, while the drone verbally responds and confirms its actions in real time.

Creative Storyteller winner: Sankofa
By: Jeremiah Somoine

Sankofa acts as a multimodal AI "griot"—a traditional West African storyteller—transforming fragmented family histories into deeply immersive narratives. Based on just a few user details, it weaves together rich voice narration, watercolor imagery, and ambient soundscapes into a historical story, allowing users to engage in a real-time voice conversation with the storyteller to explore their roots further.

UI Navigator winner: Moonwalk
By: Enaiho Uwas Paul and Aman Kumar Sah

Moonwalk is a conversational, hands-free desktop assistant that helps users intuitively navigate their computer and complete complex tasks using just their voice. By remembering personal preferences and past interactions, it acts as an intelligent co-pilot that can seamlessly control your mouse and keyboard to execute everyday workflows—like booking flights or managing spreadsheets—while you simply sit back and speak.

Best multimodal integration and user experience winner: Wand
By: David Li

Wand is a voice-first, pointer-aware browser assistant that helps you seamlessly navigate and interact with any website using a combination of natural speech and hand gestures. By simply pointing at your screen and speaking — like asking to "play this video" or "zoom in here"—this live agent helps you instantly execute clicks, searches, and commands without ever needing to touch a mouse or keyboard.

Best technical execution and agent architecture winner: JohnKeats.AI
By: Matthew Keats

JohnKeats.AI is a voice-first emotional companion designed to actively listen and hold space for users without rushing to offer solutions. By processing subtle vocal cues like pitch, pacing, and tone, it reacts naturally to a user's emotional state in real time to provide a deeply reflective and empathetic conversational experience.

Best innovation and thought leadership winner: Rayan Memory
By: Yusuf Elnady

Rayan Memory tackles the universal problem of forgetting by turning your daily learnings into a fully explorable 3D "memory palace." A background agent passively listens to your real-world audio to extract important ideas as physical artifacts, allowing you to walk through themed virtual rooms and converse with a dedicated AI companion to easily retrieve your exact memories.

Honorable mention: NagarDrishti
By: Nikita Dongre and Omkar Dongre

NagarDrishti tackles dangerous road conditions by allowing citizens to safely report potholes and waterlogging using a hands-free voice assistant while driving. These real-time reports instantly populate an interactive dashboard, where city officials can use natural language to easily identify hazard hotspots and manage critical repairs.

Honorable mention: Ekaette
By: Bassey John

Ekaette revolutionizes customer service by replacing frustrating hold queues with a conversational, multimodal AI assistant that operates across live phone calls and text messaging. Customers can speak naturally with the agent over a standard phone line while seamlessly sharing photos, reviewing product options, or completing payments via WhatsApp, c

Honorable mention: VibeCat
By: Sejun Kim and Michael Chang

VibeCat is a proactive macOS desktop companion that continuously watches your screen, understands your context, and suggests helpful actions before you even ask. Instead of waiting for a command, it speaks up first — like offering to fix a missing line of code or execute a terminal command — and completes the task only after receiving your permission.

Honorable mention: Call My Parts
By: Sugam Palav, Nikhil Lohar, Siddhant Panday, and Vishal Parekh

Call My Parts automates the tedious, time-consuming process of sourcing used vehicle parts by doing the research and vendor outreach for you. Users simply speak their part request, and the AI agent autonomously searches vendor websites, calls suppliers to check pricing and inventory, and compiles the best options into a ranked, easy-to-read dashboard.

Honorable mention: Relay
By: Faith Ogundimu

Relay is an interactive AI lab partner that uses your webcam to watch and guide your physical electronics projects in real time. It provides step-by-step voice instructions to help you build circuits, catches wiring mistakes before they happen, and reinforces your skills with a built-in 3D simulation sandbox and adaptive quizzes.

Keep the momentum going

Inspired by these incredible projects? Start building and stay connected with the community through our latest programs and events:

Join Gemini Enterprise Agent Ready (GEAR), designed to help developers and decision-makers build and deploy production-ready AI agents.
Catch up on Google Cloud Next 2026: We just wrapped up an amazing Google Cloud Next! If you weren't able to join us in person — or simply want to relive the energy — take a look at our social and livestream recaps to catch up on some of the exciting developer activations straight from the expo floor.
Tune in on Tuesdays: Want to be the first to hear about new tools, product updates, and upcoming hackathons? Join us for our weekly livestream every Tuesday 9:00 A.M. PDT / 12:00 P.M. EDT for the latest in all things Google Cloud.

Congratulations again to all of our winners and participants. We can't wait to see what you build next!

Welcome to BlackFile: Inside a Vishing Extortion Operation

Fri, 15 May 2026 14:00:00 +0000

Written by: Austin Larsen, Tyler McLellan, Genevieve Stark, Dan Ebreo

Introduction

Google Threat Intelligence Group (GTIG) has continued to track an expansive extortion campaign by UNC6671, a threat actor operating under the "BlackFile" brand, that targets organizations via sophisticated voice phishing (vishing) and single sign-on (SSO) compromise. By leveraging adversary-in-the-middle (AiTM) techniques to bypass traditional perimeter defenses and multi-factor authentication (MFA), UNC6671 gains deep access to cloud environments. The group primarily targets Microsoft 365 and Okta infrastructure, leveraging Python and PowerShell scripts to programmatically exfiltrate sensitive corporate data for subsequent extortion attempts. This post details UNC6671’s attack lifecycle and provides defenders with actionable guidance to detect and mitigate these identity-centric threats.

Since emerging in early 2026, UNC6671 has maintained a high operational cadence. GTIG assesses that the group has targeted dozens of organizations across North America, Australia, and the UK.

GTIG previously highlighted UNC6671 as a distinct cluster in a prior report detailing similar SaaS data-theft techniques utilized by ShinyHunters (UNC6240). While UNC6671 has co-opted the ShinyHunters brand in at least one instance to inject artificial credibility into their threats, GTIG assesses that the operations are independent. This distinction is supported by UNC6671's use of separate TOX communication channels, unique domain registration patterns, and the launch of a dedicated "BlackFile" data leak site (DLS).

These compromises are not the result of a security vulnerability in vendor products or infrastructure. Instead, this campaign continues to highlight the effectiveness of social engineering and underscores the critical importance of organizations moving toward phishing-resistant MFA to protect their SaaS and identity platforms.

Initial Access

UNC6671 initial access operations rely on high-volume voice phishing (vishing), often characterized by meticulous social engineering tactics, synchronized with real-time credential harvesting. These vishing calls are typically made by "callers" hired by the threat actor.

IT Deployment Pretext

The callers often call targeted employees' personal cellular phones to bypass security tooling and move the victim away from standard support channels. They typically masquerade as internal IT or help desk personnel, citing a mandatory migration to passkeys or a required multi-factor authentication (MFA) update. This pretext justifies directing the victim to a credential harvesting site and provides a logical cover for any subsequent security alerts generated during the compromise. UNC6671 has shifted from unique, organization-tailored credential harvesting domains to a subdomain-based model. These domains are typically registered with Tucows. Recent campaigns have used subdomains explicitly referencing "passkey" or "enrollment" themes to enhance the legitimacy of the help desk pretext.

<organization>.enrollms[.]com
<organization>.passkeyms[.]com
<organization>.setupsso[.]com

Real-Time MFA Interception

The vishing call functions as a live adversary-in-the-middle (AitM) attack. The process follows a rapid, procedural lifecycle:

Redirection: The victim is directed to a lookalike subdomain mirroring the organization's single sign-on (SSO) portal.
Credential Capture: As the victim inputs their username and password, the threat actor captures these in real-time and immediately submits them to the legitimate SSO provider.
MFA Bypass: When the legitimate portal issues an MFA challenge (Push, SMS, or TOTP), the victim—believing they are completing a setup step—provides the code or approval to the threat actor.
Device Registration: Upon gaining access, the threat actor immediately navigates to the user's security settings to register a new, attacker-controlled MFA device to ensure persistence.

The speed of this execution ensures the threat actor can establish a permanent foothold before the victim or the organization's Security Operations Center (SOC) can identify the anomaly.

Data Theft

Following successful authentication, UNC6671 leverages SSO access to move laterally across the victim's SaaS applications to enable data theft operations. The threat actors appear to be focused on targeting Microsoft 365 and Okta environments, using compromised accounts to access SharePoint, OneDrive, and other connected SaaS applications such as Zendesk and Salesforce. In several instances, the actors specifically queried internal search functions for string literals such as "confidential" and "SSN" to prioritize theft of perceived high-value data.

Programmatic Data Exfiltration

Upon establishing persistence, UNC6671 transitions from interactive browser-based reconnaissance to automated exfiltration. In multiple engagements, we observed the use of scripts to harvest high-value data from SharePoint and OneDrive repositories.

In addition to relying on methods that triggered standard FileDownloaded events, the threat actor has also used less conspicuous approaches. These include the threat actor’s use of formal APIs, such as Microsoft Graph, as well as the python-requests library and PowerShell to issue direct HTTP GET requests against document resource URLs. Notably, by repurposing valid session cookies (e.g., FedAuth) captured during the initial vishing phase, the actor has been able to "stream" file content directly to attacker-controlled infrastructure.

In these cases, the request mimics a standard web client fetch rather than a formal "Download" command. As a result, the activity is frequently recorded as a FileAccessed event rather than FileDownloaded. This 'direct fetch' method naturally blends into routine traffic, which may bypass detection in many Security Operations Centers (SOCs) that prioritize FileDownloaded events and treat FileAccessed as benign.

Forensic Artifacts and Scripting

Analysis of Microsoft 365 Unified Audit Log (UAL) telemetry revealed several consistent forensic indicators of UNC6671 activity, including clear evidence of scripted exfiltration. Most notably, the threat actor frequently showed User-Agent mismatches; while they spoofed the ClientAppId for "Microsoft Office" to bypass basic conditional access filters, the recorded UserAgent strings identified scripting engines such as python-requests/2.28.1 or WindowsPowerShell/5.1. This discrepancy suggests that access was driven by automated scripts rather than human interaction with the SharePoint user interface. Additionally, these access attempts consistently originated from non-standard infrastructure, such as commercial VPN exit nodes and hosting providers.

{
  "CreationTime": "2026-02-24T14:36:15",
  "Operation": "FileDownloaded",
  "Workload": "SharePoint",
  "ClientIP": "179.43.185.226", 
  "UserId": "victim.user@organization.com",
  "UserAgent": "python-requests/2.28.1",
  "ApplicationDisplayName": "Microsoft Office",
  "IsManagedDevice": false,
  "SourceFileName": "2382_REDACTED_MSA_v3.docx",
  "SourceRelativeUrl": "Shared Documents/Legal/MasterMSA/Archive",
  "SiteUrl": "https://organization.sharepoint.com/sites/Legal_Archive/",
  "AppAccessContext": {
    "ClientAppId": "d3590ed6-52b3-4102-aeff-aad2292ab01c",
    "ClientAppName": "Microsoft Office",
    "TokenIssuedAtTime": "1601-01-01T00:00:00"
  }
}

Figure 1: FileDownloaded event observed in early UNC6671 intrusions

{
  "CreationTime": "2026-03-18T20:06:41",
  "Operation": "FileAccessed",
  "Workload": "SharePoint",
  "UserId": "victim.user@company.com",
  "ClientIP": "179.43.185.226", 
  "UserAgent": "python-requests/2.28.1",
  "ApplicationDisplayName": "python-requests",
  "IsManagedDevice": false,
  "SourceRelativeUrl": "Shared Documents/Data Analytics/Power BI Version History",
  "SourceFileName": "Weekly Production Report.pbix",
  "SiteUrl": "https://company.sharepoint.com/sites/ProductionOps/",
  "AppAccessContext": {
    "ClientAppName": "python-requests",
    "CorrelationId": "b94b01a2-2019-c000-2262-5ff1d0ff6cc8"
  }
}

Figure 2: FileAccessed event from later UNC6671 intrusions

The speed and scale of UNC6671’s data exfiltration also reflects the automated nature of these scripts, which allows the threat actors to exfiltrate massive volumes of data at high speeds. In one case, the threat actor used their Python script from a remote IP to access and download over a million individual files from a victim's SharePoint and OneDrive environments. In another case, the threat actor rapidly iterated through tens of thousands of SharePoint file interactions.

Extortion

UNC6671 conducts highly targeted extortion campaigns, beginning with unbranded ransom notes sent from programmatically generated consumer email accounts. Once a victim engages via the unique, encrypted communication channel (such as Tox or Session) provided by the threat actor in the initial ransom note, the operators identify themselves under the "BlackFile" brand. While the operators typically open negotiations with initial demands in the millions of dollars, they often pivot to low six-figure demands when met with active engagement. Notably, while the initial emails typically do not contain errors, at least some follow up emails have contained mistakes suggesting that those are human generated.

In cases where the operator is met with silence or resistance, the group aggressively escalates pressure. During a recent incident, after the victim was unresponsive, UNC6671 pivoted to an aggressive spam campaign. Using dozens of Gmail accounts with randomly generated usernames, the threat actor flooded employee mailboxes with messages before automated restrictions kicked in based on their sending behavior and their accounts were restricted. We have also observed these threat actors sending threatening voicemails to C-suite executives and, in severe cases, utilizing swatting tactics against company personnel.

Subject: [COMPANY NAME] DATA BREACH 72 HOURS TO CONTACT US
From: [pseudorandom_alphanumeric_string]@gmail.com

Hello [Company Name] Executives and HR,

We have managed to export ~[X] TB of data from your network due to your terrible security practices and negligent data storing practices.

Here is a brief overview of data exported from your network:

[X]+ GB of internal company files (SharePoint & OneDrive) containing confidential business processes, NDAs, project cost estimates, subcontractor contracts, and HR records.
Tens of thousands of emails from executive mailboxes, including confidential documents.
Complete CRM and support ticket exports (Salesforce & Zendesk) containing hundreds of thousands of customer records, PII, billing details, and communication logs.
Complete corporate directory (Entra) dumps including employee names, mobile numbers, job titles, and hierarchy.
~[X] ServiceNow IT infrastructure records (computers, servers, cloud resources).

You have exactly 72 hours to contact the [Tox / Session] ID provided below. If you fail to contact the ID provided by us within the timeframe stated, we will be forced to publish your data to the public. We will also be forced to contact each company you work with via the employee team contact phone numbers and email addresses provided and explain how [Company Name] has terrible security protocols and does not care about its customers.

We are willing to engage in good faith negotiation terms. Upon contacting us, a full list of all data exported from your network will be sent to you for review. You will be able to pick up to 3 files to confirm and verify we have what we are claiming.

[Tox / Session] ID: [Unique Alphanumeric String]

Silence may not always be wise in situations like this. We will not be ignored. Make the right choice and cooperate with us so this can be a learning experience for you.

Figure 3: Generalized example initial unbranded extortion note from UNC6671

Subject: [COMPANY NAME] DATA BREACH 72 HOURS TO CONTACT US
From: [pseudorandom_alphanumeric_string]@gmail.com

Dearest executive,

You have picked to ignore the first deadline to contact us. That is not smart do not ignore us it will only make things worse. We are BlackFile. Do not play games with us. We are giving a final deadline of 72 hours to contact us so we can reach an agreement.

We copied over [X] TB+ of data from your SharePoint & M365 instance (legal documents, operational documents, client documents, sales documents, development documents, etc) over [X]gb of Salesforce data, full ZenDesk support ticket export for [X]+ customers, ALL ticket history including old and new tickets and their contents. Total taken from your network is over [X]TB+

Do not be alarmed as you can secure the proteciton of your data by choosing to work with us. Nothing taken from your network has been disclosed to the public or shared with third parties as of now.

Reach out to us on session to receive all details and evidense that we accessed your network. We will use Session to communicate with you. You can get Session by visiting getsession(.)org

Reach out to the following ID using Session: [Unique Session ID]

Do not reply to this email. Instead alert the rest of your HR and SOC/IT Security Team. We give you a final deadline of 72 hours to confirm reciept that you received this email by contacting us on Session.

If you fail to contact us a second time then a majority of the emails taken from your network will receive a notification from us explaining you failed to come to an agreement with us to protect your customers PII and other sensitive information. Additionally we will message journalists about this breach and your failure to come to a resolution with us before finally uploading all data taken from you to our blog for the public.

Do not let a data recovery company tell you not to negotate us we are BlackFile and we do not play games. The data we took from you can seriously damage your reputation if released is it really worth having that happen over ignoring us?

Blackfile

Figure 4: Generalized example follow up extortion email which included branding not present in initial messages

Evolution of Ransom Notes

Throughout their operations in early 2026, UNC6671's ransom notes exhibited an evolution in formatting, branding, and communication methods. Initially, the threat actors used highly aggressive, short-term deadlines, often giving early victims generic 24 or 48 hour windows to respond. This appeared to become more standardized in late January when they gave subsequent targets a strict 72-hour deadline. Their email subject lines also evolved into a formalized, all-caps structure: [COMPANY NAME] DATA BREACH 72 HOURS TO CONTACT US.

During this same period, the group’s identity and preferred communication channels shifted. Early extortion emails were unbranded, with the actors demanding contact via Tox (a peer-to-peer instant messaging protocol). By February 2026, the group formally adopted the "BlackFile" moniker and transitioned their communication demands exclusively to Session (a decentralized, privacy-focused messenger), providing victims with Session IDs and client download instructions. Additionally, while early extortion notes were sent from external emails that could easily be flagged by spam filters or ignored, since at least March 2026, UNC6671 has leveraged hijacked internal corporate email and Microsoft Teams accounts.

The BlackFile Data Leak Site (DLS)

The threat actors launched the BlackFile Data Leak Site (DLS) on February 6, 2026, claiming to operate as "security researchers." Despite maintaining a dedicated DLS, the group's approach to data exposure deviates significantly from the maximum-publicity, high-noise model employed by other actors. UNC6671 does not publicly advertise their leak site or attempt to index it for search engines. Furthermore, the group has typically only leaked limited file samples and directory listings rather than full datasets; to date, GTIG has not observed the actor leak victim data in full.

Figure 5: BlackFile DLS

Figure 6: BlackFile DLS Deletion Process

Notably, the BlackFile DLS site went offline in late April 2026, but briefly came back online on May 11, 2026 to share the below message before shutting down again. In this message, the threat actor stated "BlackFile is shutting down… under this name." As of the time of publication, the DLS site is inaccessible.

Figure 7: BlackFile DLS Shutdown Announcement

Remediation and Hardening

GTIG recommends the following mitigations and hunting strategies:

Deploy Credential Guarding: Configure environment-specific protections to catch credential submission at the point of impact. In Google Workspace, enable Password Alert to monitor for corporate password hashes being entered into unauthorized domains. For Microsoft environments, leverage Microsoft Defender's Credential Protection and SmartScreen to intercept submissions on known phishing or low-reputation sites. These automated technical controls act as a final fail-safe, triggering immediate password resets or security alerts when a user inadvertently interacts with a malicious page.
Implement Phishing-Resistant MFA: Transition away from SMS-based or push-notification MFA. Implement FIDO2-compliant security keys or passkeys, which are resistant to the adversary-in-the-middle (AiTM) and vishing tactics employed by UNC6671.
Monitor IdP Logs: Review identity provider logs for system.multifactor.factor.setup events that are immediately preceded by user.authentication.auth_via_mfa failures or "Abandoned" challenges.
Correlate Infrastructure: Alert on authentication attempts originating from known commercial VPNs or hosting providers that are abnormal for the user's typical geographic location.
Audit SaaS API Activity: Monitor Microsoft 365, SharePoint, and Salesforce audit logs for anomalous, high-volume file downloads (FileDownloaded or FileAccessed events) originating from generic scripting user agents (e.g., PowerShell, Python).
Monitor User-Agents: Monitor for specific IdP SDK User-Agents on devices not previously associated with a user's profile.
Re-Evaluate "Access" Severity: Security Operations Centers (SOCs) should treat FileAccessed events with the same criticality as FileDownloaded when the User-Agent identifies it as a programming library (Python, Go, etc.) or a command-line tool.
Audit for Direct File Streaming: Monitor for FileAccessed logs where the AppAccessContext indicates a headless client or where the volume of "Accessed" files in a short window exceeds human browsing capability.

Outlook and Implications

The recent shutdown of the BlackFile data leak site (DLS) accompanied by the actors' own declaration that they are shutting down "under this name" signals a possible transition phase rather than a permanent cessation of their threat activity. Historical precedents across the extortion ecosystem demonstrate that major threat clusters commonly rebrand or disperse their operations following disruption or voluntary shutdowns. These events can serve several strategic functions: evading law enforcement or competitor scrutiny, quietly resolving pending extortion cases, or preparing to pivot to a more viable brand while simultaneously also allowing time for the threat actors to retool and/or set up new infrastructure. Even if the BlackFile brand is permanently retired, the techniques leveraged by UNC6671, specifically their focus on data theft from cloud and SaaS environments, represent a highly successful trend in the cyber crime threat landscape that we also highlighted in the Google Cloud H1 2026 Cloud Threat Horizons Report. Organizations can review our prior blog post with actionable hardening, logging, and detection recommendations to help protect against these threats.

Indicators of Compromise (IOCs)

To assist the wider community in hunting and identifying activity outlined in this blog post, we have provided indicators of compromise (IOCs) in a free GTI Collection for registered users. At the time of publication, identified phishing domains have been added to Google Safe Browsing.

While this collection provides a comprehensive list of IOCs, defenders should note that the majority of identified IP addresses are commercial VPN nodes, and actual source IPs tend to vary as the actor continuously cycles through new infrastructure. Furthermore, the domains are often stood up and used within minutes of registration; as such, they are provided primarily as examples of past naming conventions and usage patterns rather than as a primary mechanism for real-time blocking.

Google Security Operations (SecOps)

Google SecOps customers have access to broad category rules under the Okta and O365 rule packs that detect the behaviors outlined in this report. The activity discussed in the blog post is detected in Google SecOps under the following rule names:

Okta Admin Console Access Failure
Okta Suspicious Actions from Anonymized IP
O365 SharePoint Bulk File Access or Download via PowerShell
O365 SharePoint High Volume File Access Events
O365 Sharepoint Query for Proprietary or Privileged Information