Customers

ZenteiQ scales batch inference 4x with GKE Inference Gateway powered by llm-d

Wed, 21 Oct 2026 16:00:00 +0000

Today we hear from ZenteiQ, an artificial intelligence startup spun out of the Indian Institute of Science (IISc) and selected under the IndiaAI Mission to build sovereign scientific foundation models from the ground up. To pre-train models that understand physical laws, governing equations, and numerical solvers, ZenteiQ designed and built a large-scale batch synthetic data-generation pipeline that runs across multiple model families and feeds verified datasets into training runs. Google Kubernetes Engine (GKE) and GKE Inference Gateway gave ZenteiQ the inference-routing layer within its pipeline to scale aggregate inference throughput 4.07x and grow daily output from 1.47 billion to 5.97 billion generated tokens for every 1,000 accelerators. - Fisayo Feyisetan, Product Manager, Google Cloud

Unlike general-purpose language models trained primarily on web text and code, scientific foundation models must understand complex physical systems, mathematical governing equations, and numerical solvers across domains such as thermodynamics, computational fluid dynamics, and electromagnetics. At ZenteiQ, we are building sovereign scientific foundation models from scratch that interpret simulation outputs, evaluate equation formulations, and work through step-by-step mathematical derivations that obey physical laws.

Pre-training models on large datasets of governing equations takes hundreds of billions of reasoning tokens. Alongside curated scientific literature and simulation datasets, we also rely on a batch synthetic data-generation pipeline that we independently designed and implemented to generate, evaluate, and filter candidate derivations and solver validation problems before feeding them into our MaxText pre-training jobs.

Because we operate on a fixed compute allocation, every hour our accelerators spend generating and verifying training data is an hour taken away from pre-training. When data generation falls behind, multi-host accelerator clusters reserved for pre-training sit idle, waiting on datasets.

When data generation slows down pre-training

To maximize data generation speed, our pipeline separates prompt preparation on worker pods from model serving on vLLM across multi-node accelerator pools. Each batch moves through four stages: expanding scientific seed concepts into prompt batches, generating candidate reasoning traces with generator models, scoring responses using evaluator models, and running guardrail checks with a separate model family before packaging verified datasets for pre-training.

We originally ran these model pools on Kubernetes with standard ClusterIP round-robin load balancing across inference endpoints. As we scaled our pipeline to feed our 9B- and 35B-parameter model schedules, two routing bottlenecks began slowing down dataset creation.

Uneven engine queues: Our upstream worker pods expand seed concepts into prompts of widely varying complexity and response length. Because standard round-robin load balancing had no visibility into vLLM engine queues, it kept sending new requests to serving pods that were already busy processing long reasoning traces, while neighboring nodes had capacity to spare.
Redundant prefix computation: Prompts within a generation batch share roughly 70% to 80% of their instructional context and task specifications. Round-robin routing scattered requests with identical instructions across different serving instances, forcing each pod to recompute the same shared prompt prefix from scratch instead of reusing warm caches in accelerator memory.

Together, uneven queues and redundant prefill work caused our generation throughput to swing by as much as half within a single run, stretching out dataset creation and delaying the start of scheduled pre-training runs. We needed routing that understood live vLLM queue depths and prompt caches, without pulling our researchers off model development to build and maintain a custom inference scheduler.

Building on GKE Inference Gateway powered by llm-d

We already run our data preparation and MaxText pre-training jobs on GKE, which lets our team manage data generation and model training on the same Kubernetes platform. Following a recommendation from Google Cloud to address our routing bottlenecks without changing our underlying scientific methodology or pipeline design, we deployed GKE Inference Gateway, powered by the open-source llm-d distributed inference framework, in front of our vLLM accelerator pools.

In our pipeline, worker pods pull scientific seed documents and domain taxonomies from Cloud Storage and expand them into batches of multi-step prompts. Those batches flow through a single GKE Inference Gateway deployment that manages traffic across all of our model families and serving nodes.

Rather than maintaining isolated routing for each model tier, we route generation requests, evaluation queries, and guardrail checks across their respective model pools through the same gateway. Once candidate responses pass evaluation and guardrail checks, worker pods deduplicate and package the verified datasets for direct consumption by downstream MaxText pre-training jobs on GKE.

GKE Inference Gateway routes batch generation, evaluation, and guardrail requests across multi-node accelerator pools running vLLM before verified datasets are fed into MaxText pre-training.

From round-robin to load- and cache-aware routing

With GKE Inference Gateway and llm-d, routing decisions use live serving signals from our vLLM instances rather than static round-robin rotation. GKE Inference Gateway continuously monitors real-time queue depth and key-value (KV) cache utilization across our vLLM endpoints. When a serving node is busy decoding a long equation derivation, the gateway routes incoming requests to nodes with open capacity, so heavy prompts no longer pile up behind busy workers while neighboring nodes sit idle.

At the same time, the gateway routes requests with matching scientific instructions to the serving node where that prefix’s KV cache is already warm in accelerator memory. Because 70% to 80% of each prompt is shared instructional context while the generated reasoning trajectories vary from request to request, combining prefix-cache locality with live queue balancing lets our pods skip repeated prefill work without overloading any single replica.

Higher throughput, predictable training runs

Adopting GKE Inference Gateway powered by llm-d transformed the speed and predictability of our data generation pipeline. Across our production clusters, intelligent routing delivered:

4x higher fleet throughput: Aggregate throughput jumped from 16,964 to 69,114 tokens per second per 1,000 accelerators — growing daily output from 1.47 billion to nearly 6 billion generated tokens.
3.5x to 5.5x gains across individual model pools: Real-time queue balancing kept serving nodes evenly loaded throughout each run, preventing the mid-run slowdowns caused by long reasoning derivations.
14-percentage-point higher prefix-cache hit rate: Routing prompts with matching scientific instructions to warm caches avoided redundant prefill work before each request branched into its own derivation.

For our research and infrastructure teams, the most important outcome was schedule predictability. Quadrupling daily output meant verified datasets were ready on time, so our multi-host pre-training runs started on schedule instead of waiting on data generation.

What’s next for physical AI at ZenteiQ

As we scale our scientific foundation models from 9B to 35B parameters and beyond, we plan to use the same GKE Inference Gateway and llm-d routing setup for post-training reinforcement learning (RL) and automated numerical solver evaluation loops. Because each turn in those multi-turn rollouts builds on the context of the turns before it, they depend even more heavily on live queue balancing and warm prefix caches.

To try these capabilities on your own workloads, explore the GKE Inference Gateway documentation and check out the open-source llm-d project. You can also read how the Gemini platform team cut latency by 35% with GKE Inference Gateway.

Building full virtual vehicles on Google Cloud: From AAOS to the smallest MCU

Mon, 19 Oct 2026 16:00:00 +0000

Imagining traditional gearheads getting their hands on software-defined vehicles is an amusing thought experiment. With their socket wrenches and air compressors at the ready, they might be confused to encounter core functions more likely to run on 1s and 0s than oil and belts. But across the auto industry these days, more and more vehicles feature prominent systems built using cloud-native development, continuous integration/continuous delivery, fast digital iteration, and plenty of AI.

But there’s still a big gap to be found in many automakers’ and Tier 1 suppliers’ cloud-based development processes. On the one hand, these companies are commonly using software to develop vehicles’ digital entertainment and information screens, or other onboard high-performance computers.

On the other, the dozens of microchips and control modules that make a car a car these days — such as gateways, body controllers, zone control units, sensor and actuator nodes — are still frequently hardwired to physical chips when they’re constructed, making them very difficult to update, reprogram, or expand after the vehicle drives off the factory floor. Teams have to wait for benches, samples, and rigs, and integration can easily slip to the end of the development process, where defects are most expensive to fix.

This inherent inefficiency has become a leading bottleneck. After all, manufacturers can’t really claim vehicles are software-defined if most of their electronics only function when connected to physical hardware and if developers are leaning heavily on actual hardware.

With Horizon, our agentic innovation platform for developing software-defined vehicles, our vision is to help manufacturers avoid these inefficiencies by working with partners like RemotiveLabs.

The cockpit already runs in the cloud

The Android team has invested a significant amount of time and effort over the last few years on enabling seamless development of the many digital systems in vehicles’ cockpits. Android Automotive OS (AAOS) runs inside Google’s official virtual testing environment for Android, Cuttlefish, which itself runs comfortably on Arm-based Google Cloud instances. With this approach, developers can build, boot, and test virtual versions of their infotainment stacks on the same Arm architecture that ships in vehicles.

And with the recent launch of Android 17 and AAOS SDV, this capability is also extended to further domains, whenever AAOS SDV is used. Deploying it with Cuttlefish, for example, proves the model: representative software, on representative architecture, in the cloud, on demand. The question that still needs to be answered is how to extend that model to the rest of the vehicle, especially when other operating systems are involved.

The harness around the virtual vehicle

That’s where RemotiveTopology comes into play. As the harness around the virtual vehicle, it’s the layer that connects AAOS in Cuttlefish to a full, simulated electrical and electronic (E/E) architecture: the rest of the vehicle’s onboard computers, its data networks, and the live signals that make the system function like a real vehicle rather than a collection of standalone apps.

RemotiveTopology automatically utilizes manufacturers’ existing network design files to build a working, virtual version of their cars’ networks. This is vehicle-as-code: the E/E architecture expressed as an executable specification, versioned and provisioned like infrastructure-as-code. In this way, an entire vehicle can be spun up on demand. Cockpit software talks to the rest of the vehicle exactly as it would in a real vehicle. As a result, developers get full and continuous signal and cross-domain visibility, so they can see real-time system status. And because it is built on standard Linux tooling, the same topology runs on a laptop and in a contiguous integration pipeline on Google Cloud.

Integrate across domains early, not late

The payoff for using this approach is that all onboard systems can be developed and integrated with software before any of them are connected to a vehicle’s hardware. That’s beneficial because the most difficult (and latest) integration in a vehicle’s development is bringing all its systems together. For example, advanced driver assistance systems (ADAS) produce object lists, trajectories, and mode requests that the cockpit has to render and react to, while the infotainment system and information cluster have to reflect driving state while body and chassis nodes sit in the loop.

With the full topology live on Google Cloud, the infotainment system and ADAS can operate simultaneously within a virtual vehicle, exchanging real signals over the simulated network. This means it’s possible to exercise handoffs like an ADAS event surfacing in the cluster or a driver-mode change rippling across domains from day one in continuous integration long before the benches and samples line up. In this way, developers can be confident they’ll discover multi-domain integration long before the end of the process so that they can test continuously from the start.

The full vehicle frontier

At the heart of this work is a shared desire to bring scalable software development to even the smallest and most common nodes in a vehicle, its microchips and control modules. That’s because these are the parts that have traditionally been wired to hardware, and as a result are the biggest blocker to fully software-defined vehicles.

The ultimate goal is that the same software that runs in the cloud can run on vehicles’ actual control modules with no re-tooling or re-validation needed to move between them. That parity is natural with Arm-based Google Cloud infrastructure because the cloud and the car share an instruction set. Google's Axion makes it straightforward: C4A metal, a bare-metal Arm instance, is provided explicitly for automotive systems development and Android software development, giving direct hardware access without unnecessary software layers and delivering the consistent performance that vehicle systems require.

Taken together, this holistic system allows teams, for the first time, to develop and integration-test an entire vehicle’s systems in the cloud and then apply it to the actual vehicle without having to rewrite it.

Why now, and where to start

The payoff is not abstract. Treating the vehicle as code and moving integration into the cloud compresses the slowest, most expensive phase of a program. Early customer engagements with RemotiveLabs have measured more than 10x faster integration cycles. And the development loop is now changing faster than the hardware loop: AI-assisted and agentic workflows can rapidly generate, test, and refine vehicle software, but only virtual systems provide the required scale. Every element that can’t be virtualized can slow the entire development process down.

RemotiveTopology provides this architecture as vehicle-as-code and makes it accessible for developers, testers, and autonomous AI agents alike. Best of all, there is no new third-party tooling to adopt just to get started. Instead, RemotiveLabs publishes an open-source simplified vehicle on GitHub that anyone can run in minutes on standard Linux tooling and platform definitions already in place. Full existing toolchains plug in as a manufacturer or Tier1 supplier scales, and RemotiveTopology can be easily dropped into an Arm-based Google Cloud environment.

For those building software-defined vehicles whose virtual development stops at the cockpit, it’s time to start with an open-source example, bring unique platform definitions, and run an entire vehicle in the cloud. RemotiveTopology is available in Horizon for easy usage in combination with other Horizon modules.

Architecting autonomy: A design pattern for high-compute, governed agents in biotech

Fri, 16 Oct 2026 16:00:00 +0000

In production systems today, most AI agents operate as lightweight dispatchers. They parse a prompt, make an API call, retrieve a record, and hand back control. Their execution is fast, stateless and cheap. Biotechnology does not work this way.

Folding a protein target, screening a 50,000 compound ligand library, or evaluating structure-activity relationships (SAR) are not quick API lookups. They are sustained, stateful, GPU-bound simulations that run for minutes or even hours across specialized silicon. When you ask a general-purpose large language model (LLM) endpoint to coordinate a complex drug discovery workflow, the standard paradigm breaks: Timeouts occur, context windows overflow with atomic coordinates, and proprietary chemical Intellectual Property (IP) risks exposure to external networks.

Building enterprise grade research and development (R&D) agents requires moving past the monolithic assistant model. In this blog post, we will discuss how to use a three-layer architecture: separating cognitive orchestration (the brain) from computational execution (the muscle), and anchoring both inside an enterprise governance plane to build scalable agents for drug discovery use cases in biotech.

A three-layer pattern

The reference architecture below shows how to architect, optimize, and govern high-compute autonomous agents in biotechnology, specifically Drug Discovery.

Figure 1 — The three-layer pattern: a cognitive orchestrator (brain), a scalable GPU execution layer (muscle), and an enterprise governance plane.

The execution layer: scaling the muscle - high-compute execution

The execution layer is responsible for the heavy scientific computations. The first architectural decision is determining where workloads should run:

Strategic, proprietary pipelines require dedicated GPU infrastructure within your security perimeter
Routine or standardized workloads can safely be delegated to managed cloud endpoints

In this architecture, protein folding is driven by ESMFold and generative design by GenMol. Both run as NVIDIA Inference Microservices or NIMs deployed on Google Kubernetes Engine (GKE) using G4 VMs backed by NVIDIA RTX PRO 6000 Blackwell GPUs.

A critical hurdle in containerized GPU workloads is the cold-start penalty, the latency delay required to pull multi-gigabyte model weights from an image registry and stage them into GPU memory (VRAM).

To ensure low latency responsiveness, we build the following as a recommendation into the reference architecture:

Maintain warm pools: Pin the GKE node pool to a minimum of one active node.
Preload weights: Model weights are loaded directly into VRAM at pod initialization, ensuring containers are immediately ready for inference.
Resilient scheduling: GKE manages pod lifecycles and restarts failed instances without tearing down the underlying VM infrastructure.

This approach trades custom, manual CUDA kernel tuning for NVIDIA's pre-compiled, containerized runtimes, dramatically reducing infrastructure engineering overhead while ensuring deterministic throughput.

In biotech, chemical structures represent millions of dollars in proprietary IP. A docking request transmits SMILE strings or 3D coordinate matrices that must never leave company custody.

Containerized NIMs eliminate the friction between rapid prototyping and rigorous data isolation. During early exploration, researchers can query NVIDIA hosted endpoints. When transitioning to proprietary lead candidates, the exact same container is deployed to GKE inside a private VPC. The agent’s orchestration logic remains identical; only the target URI updates from an external proxy to an internal VPC endpoint. No structural data leaves the enterprise perimeter.

The cognitive layer: Multi-agent reasoning

With compute execution isolated, the cognitive layer manages agentic workflow sequencing. Built on Agent Runtime, within the Gemini platform, this reference architecture shows a system that deploys a hierarchical multi-agent structure, one lead orchestrator, and two specialized domain sub-agents (Lead Optimization and Safety Screening) . The multi-agent design can be modified to suit the use case at hand.

Based on this design, when an end-user submits a high-level task such as “ Analyze Candidate 4b against Target X and produce a SAR report”, the system applies four core orchestration patterns discussed below;

In-process native agent handoffs: The lead orchestrator delegates tasks to specialist sub-agents using Agent Runtime’s native SDK rather than network-serialized agent-to-agent HTTP protocols. This eliminates intermediate network hops and keeps internal handoffs instantaneous. Only the outermost boundary communicates outward to user-facing surfaces like the Gemini agent.
Passing bulky payloads by reference: LLMs degrade quickly when forced to ingest massive arrays of 3D spatial coordinates. In this architecture, raw structural data (PDB files, docking poses, trajectory logs) is written directly to Cloud Storage. Agents pass lightweight, signed object URIs and high-level structural summaries, keeping context windows compact and inference costs down.
Decoupling workflow logic from biochemical reasoning: General-purpose LLMs excel at task planning and synthesis, but are prone to hallucinating biochemical nuances. To control this behavior, this architecture maintains strong separation of concerns, namely, workflow and tasks decomposition are handled by Gemini. Domain chemistry and pharmacological reasoning are handled by specialized Nemotron models hosted on dedicated GPU nodes.
Context-driven evaluation over rigid cutoffs: Traditional computational screening enforces arbitrary numerical cutoffs (e.g., discard if binding affinity ,-7.5 kcal/mol) However, therapeutic context fundamentally alters risk tolerance. Preventive cardiovascular therapy taken daily by healthy patients requires extremely strict safety margins, or a novel oncology candidate targeting an intractable cancer may accept higher toxicity if efficacy is demonstrated.

By passing disease context directly into the specialized reasoning models, drug candidates are evaluated against dynamic, target-specific profiles rather than static thresholds, preventing the premature culling of viable drug leads.

Performance best practices for high-compute pipelines

Optimizing an agentic system that interfaces with heavy GPU compute requires challenging common engineering instincts:

Common Assumption	Architectural Reality	Recommended Best Practice
"Swap in a faster driver model to cut latency."	The primary orchestrator's token generation is rarely the bottleneck in multi-step R&D pipelines.	Optimize I/O, not token speed: Focus on reducing payload sizes and intermediate network hops rather than compromising orchestrator reasoning depth.
"Parallelize tool calls across workers."	External scientific services and containerized models often serialize requests internally on dedicated hardware.	Respect execution topology: Align client-side concurrency with backend queue mechanics to prevent pipeline lockouts and connection timeouts.
"Pass full simulation states between tools."	Large JSON bodies cause schema validation failures, prompt truncation, and token bloat.	Pass heavy payloads by reference: Store raw coordinates in object storage and pass lightweight URIs between agent boundaries.
"Return outputs only when the pipeline finishes."	Long multi-minute runs leave scientists blind to whether jobs are proceeding or stalled.	Stream intermediate stage completions: Emit progress updates at each completed milestone (e.g., folding complete, docking running) for real-time visibility.
"Chain multiple granular micro-tools."	Deep tool-chains increase the probability of hallucinated arguments and schema mismatches.	Collapse compute pipelines: Wrap end-to-end simulation pipelines behind a single, clean tool interface that takes clear parameters and returns a consolidated object.

The enterprise governance and asymmetric autonomy

Deploying high-compute agents in enterprise environments requires robust data sovereignty, audit trails, and process control. The Gemini platform establishes these safeguards natively with the following components:

Agent Identity: Every autonomous agent operates with a cryptographically verifiable identity. Thus, for a biotech use case, when a candidate molecule is advanced or rejected, the event is permanently logged to that specific agent principal.
Agent Gateway: This serves the single ingress/egress point for every interaction within the perimeter. Centralized IAM and regional residency rules help ensure sensitive molecular data remains within designated cloud regions.
Agent Registry: This feature maintains a curated catalog of approved agents, validated tool schemas, and authorized Model Context Protocol (MCP) servers, preventing rogue tools from executing in production.
Model Armor: This functionality inspects incoming prompts and outgoing payloads in real time to prevent prompt injection and data exfiltration.

The payoff is clear. Identity, RBAC, auditing, residency, and data protection are native properties of the Gemini platform; this allows enterprise customers to focus on science rather than having to build a compliance stack for every project.

In high-stakes research and development, full autonomy introduces unacceptable liability, while requiring approval for trivial steps can create frictions. The solution addressing this to use asymmetric gating in the following ways;

Automate reversible operations: For example, allow the agent to resolve chemical naming synonyms, pull structural PDBs, and run preliminary Lipinski screens autonomously without interrupting the researcher.

Gate irreversible operations: For instance, initiating multi-GPU docking run, advancing a lead compound to wet-lab synthesis, or archiving a therapeutic program must require explicit human sign-off.

The pattern, and the path forward

High-compute autonomous agents succeed when the boundaries between cognitive planning, numerical computation, and system governance are clean in the following ways:

Keep the orchestrator focused on decomposition and synthesis, leaving intensive biophysical simulation to specialized, containerized microservices and models.
Eliminate context bloat by passing complex chemical data by reference through cloud storage.
Anchor the workflow in enterprise governance, pairing cryptographic agent identity with asymmetric human oversight.

As generative and structural biology models expand toward whole-cell modeling and complex dynamic simulations, this decoupled architecture ensures research systems scale efficiently while keeping proprietary IP secure and auditable.

Reach out to your Google sales representative if you have any requirements to build a use case like this and we will be happy to talk and support you in your agent development journey on Google Cloud.

How Visko streams real-time world models on Google Cloud TPUs

Thu, 15 Oct 2026 16:00:00 +0000

Editor's note: Generative AI startups are moving rapidly beyond static clip generation to build interactive, real-time worlds. In this technical deep dive, we explore how AI startup Visko collaborated with Google Cloud TPU engineers to deploy Orbis — its continuously streaming Live World Model—on Cloud TPUs, going from initial kickoff to production handover in just 30 days.

What makes a live video world model so challenging to serve in production? The bottleneck isn't average throughput — it's beating a relentless real-time clock. Unlike offline video generators where a user submits a prompt and waits in a queue, a live world model streams continuously. Every new chunk of video and synchronized audio must finish rendering before the viewer finishes watching the previous one, or the world freezes.

Today, we're excited to share how Visko, an AI startup pioneering interactive world models, teamed up with Google Cloud to solve this exact challenge. Working side by side, Visko and Google Cloud engineers deployed Visko's end-to-end live inference pipeline — combining video generation, super-resolution upscaling, and synchronized audio—onto a single Cloud TPU slice. Together, the teams brought end-to-end generation down, rendering at 1.07x playback speed.

Here is an inside look at how we engineered the pipeline to hit that milestone in a single month, the architectural optimizations that moved the needle, and practical lessons for other startups building ultra-low-latency generative AI on Google Cloud.

Redefining interactive media with Live World Models

While most generative video platforms focus on rendering fixed, pre-baked clips, Visko is building what it calls Live Models: video world models that generate continuously and respond on the fly.

Visko's flagship model, Orbis, streams live video starting from a single text or image prompt. As the stream plays, viewers can enter new steering prompts in real time, dynamically evolving the scene without restarting or breaking continuity. For instance, in one test session, the livestream opens on a cat napping beside a crackling fireplace; when the viewer steers the scene to introduce a bowl of food, the bowl materializes seamlessly, and the cat stretches, stands up, and meows.

Since launching its public demo on September 1, 2026, Orbis has unlocked new possibilities across interactive entertainment, live commerce, AI companions, immersive education, and robotics simulation. To scale these interactive sessions reliably, Visko partnered with Google Cloud TPU engineers in August 2026 to port the Orbis live pipeline to Google's Tensor Processing Unit (TPU). Within 30 days, the full multimodal pipeline was running in real time on TPUs.

Visko’s ambition is to power a new generation of interactive worlds that people can step into and shape in real time. Streaming continuous video and audio under a strict clock is a fundamentally different system challenge than offline clip generation. Working side by side with Google Cloud's TPU engineers, we brought the full Orbis pipeline to TPUs in just 30 days — giving us the low-latency compute and memory bandwidth required to bring Live Models to creators and developers at scale.

"Visko is bringing an exciting new class of real-time, steerable workloads to the Cloud TPU ecosystem. By co-engineering their live multimodal pipeline on a singleTPU slice using JAX, torchax, and custom Pallas kernels, we proved how quickly a lean startup team can go from PyTorch prototype to render at 1.07x playback speed on Google Cloud's AI Hypercomputer."
– Dave Elliott, Head of Developer Advocacy for AI, Google Cloud

Live video is a hard-deadline problem

When you serve an offline clip generator, latency is forgiving — the user waits once while the backend processes frames. A live world model has no waiting room.

Every single chunk must be denoised, decoded from latents, upscaled to 1664x936 (at 18 fps), and paired with generated audio faster than realtime playback speed, for as long as the session runs.

Two structural constraints make live world models far more demanding than standard batch inference:

Batch size of one: Each live stream represents an individual viewer's interactive session. You cannot mask latency by grouping dozens of concurrent requests into a massive batch — the standard playbook for keeping AI accelerators saturated.
Strict sequential dependency: Every chunk depends directly on the one before it. Because the video model maintains a rolling memory of recent video latents so that chunk N+1 smoothly continues the action from chunk N, you cannot pre-render chunks out of order or parallelize across future timestamps.

For startup infrastructure engineers coming from LLM serving, it helps to think of live world modeling not as a tokens-per-second throughput benchmark, but as an unforgiving per-step latency SLO.

Architecting the multimodal pipeline on a single TPU slice

To keep infrastructure lean and cost-effective as Visko scales, the team designed the live pipeline to chain three distinct generative models on a single Cloud TPU slice running in BF16 precision.

When a viewer submits an initial prompt or a live steering update, all three generative models — the core video diffusion model, the super-resolution upscaler, and the audio generator — share the same chips, taking turns on the hardware for each chunk.

Figure 1: Visko's end-to-end Orbis live pipeline chains video diffusion, latent decoding, super-resolution upscaling, and pure-JAX audio generation sequentially on a single Cloud TPU slice, completing each chunk faster than the playback speed.

To combine rapid startup iteration with bare-metal accelerator speed, the software stack is built on three open-source pillars:

JAX: Google's high-performance Python array library for accelerators, compiled into optimized machine code by the XLA compiler.
torchax: An interoperability layer that runs PyTorch models directly on Cloud TPUs via JAX, enabling startup teams to bring existing PyTorch components over without an upfront, ground-up rewrite.
Pallas: The custom kernel programming language for TPUs in JAX.

Real-time performance breakdown: Hitting the clock

By profiling and optimizing each stage on a single slice in BF16, the engineering team achieved a total per-chunk latency of under the target while — maintaining roughly 7% headroom to absorb variance during live streaming:

Figure 2: Chart of per-chunk latency budget on one TPU slice.

Because video denoising accounts for nearly two-thirds of the total compute budget, that stage became the primary focus of our kernel and parallelism optimizations.

Inside the stack: Three optimizations that unlocked real-time streaming

1. Ulysses context parallelism across all TPU chips

A video chunk expands into a massive sequence of spatiotemporal tokens inside the video transformer. To distribute that workload across the TPU chips at batch size 1, the team implemented Ulysses context parallelism.

Instead of splitting model weights, Ulysses splits the token sequence across chips. Immediately before the attention layer, a high-speed all-to-all interconnect exchange regroups the tensors so that each chip holds the entire video sequence, but only for one-quarter of the attention heads. After each chip computes attention over the full sequence for its assigned heads, a second all-to-all exchange restores the sequence-split layout for the feed-forward layers.

Figure 3: Ulysses context parallelism across TPU chips at batch size 1. High-speed all-to-all Inter-Chip Interconnect (ICI) exchanges transition tensors between a sequence-split layout for feed-forward layers and a head-split layout for Pallas splash attention.

2. Custom Pallas splash attention from MaxDiffusion

To accelerate the attention computation itself, the team integrated a custom Pallas splash attention kernel from MaxDiffusion, Google Cloud's open-source reference library for diffusion models on Cloud TPU.

Splash attention brings FlashAttention-style tiling to TPU architecture: rather than materializing the full quadratic attention matrix in High Bandwidth Memory (HBM), it computes attention in compact tiles that remain inside ultra-fast on-chip VMEM. Startup teams can drop the exact kernel used in this port straight into their own pipelines via custom_splash_attention.py.

3. A 5.8x audio speedup by moving from torchax to pure JAX

We first brought all three Orbis models onto Cloud TPU with torchax, which runs PyTorch code on TPU through JAX and XLA without a rewrite. For the video and upscaling models, torchax paired with Pallas attention kernels met our latency target directly, and both stay in torchax in the final pipeline.

Only the audio model needed a rewrite. Moving the audio stage from torchax to native JAX made it 5.8x faster and brought the full pipeline under the playback deadline.

Field notes for AI builders: Cloud TPU lessons

Takeaways from tuning live multimodal inference on Cloud TPU:

Ulysses context parallelism for single stream video: At batch size 1, splitting the token sequence across chips with all-to-all interconnect exchanges was faster than both tensor parallelism and ring attention.
Native BF16 for continuous streams: FP8 quantization cuts memory bandwidth pressure, but small quantization errors build up across chunks in a continuous stream. Native BF16 kept visual quality intact within the latency budget.
Selective JAX ports: torchax kept the video and upscaling models in PyTorch, so the team only rewrote the audio bottleneck in native JAX.
XLA compilation caching: The first boot spends a few minutes compiling XLA graphs. Caching compiled artifacts brings worker restarts under a minute.

Building real-time generative models on Cloud TPU? Start here

If your startup is pushing the boundaries of live video, world models, or ultra-low-latency multimodal inference, you can build with the same toolkit that powered Visko's port:

Accelerate diffusion attention out of the box: Grab the open-source splash attention kernel used in this collaboration from MaxDiffusion, and explore the Pallas documentation to write custom TPU kernels.
Migrate PyTorch fast, then optimize selectively: Bring your existing PyTorch models onto TPU quickly with torchax, pinpoint bottlenecks with the Cloud TPU Profiler, and port latency-critical stages to native JAX.
Spin up TPUs on demand: Use TPU Flex-start VMs to provision TPU capacity for short development and benchmarking runs without a long-term reservation.
Scale with Google for Startups: Early-stage AI startups looking for cloud credits, technical mentorship, and access to Google Cloud's AI Hypercomputer stack can apply to the Google for Startups Cloud Program.

^{Special thanks to the Visko.ai team for the close technical partnership, and to the Google Cloud TPU engineers who lead the project including: Yixin Shi, James Huang, and Elisa Tsai.}

What the Gemini agent means for startups

Thu, 15 Oct 2026 16:00:00 +0000

Small teams, short runway, and constant pressure to ship faster — it’s the reality we see every day among the founders we work with at Google for Startups. When you are building a company from the ground up, every hour spent on operational overhead or infrastructure plumbing is an hour taken away from your core product.

Last week at Gemini at Work 2026, we introduced the Gemini agent: a single, universal agent designed to work both as your personal assistant and as a dedicated member of your team. And this week, we made our new frontier model, Gemini 4 Argon, widely available — trained to maintain context across long, multi-step tasks.

Here is a look at how the Gemini agent works, five practical takeaways for founders, and how startups are already using it to punch above their weight.

What is the Gemini agent?

Instead of prompting separate chatbots for writing, coding, and data analysis, the Gemini agent is a single agent that takes high-level objectives and returns finished work. Three principles define it:

Unified: A single agent can now handle knowledge work, data analysis, creative asset generation, and writing and running code. For complex jobs, it dynamically spins up temporary, job-specific sub-agents and coordinates their work in parallel or in sequence.
Omnipresent access: It works across the surfaces you use every day — on the web, command line, desktop, and mobile, as well as natively inside Gmail, Docs, Sheets, Slides, Drive, Chat, and Calendar.
Persistent: It runs in the cloud with continuous memory across four dimensions (session, semantic, procedural, and episodic). It onboards itself like a new hire — learning your team, tools, and business context so you don't have to re-explain your startup with every prompt, and keeping multi-day tasks moving even after you close your laptop.

For founders wearing five hats at once, Gemini works like a digital chief of staff that's already briefed on your calendar, projects and documents. Ask it to research market trends, build a financial model in Sheets and turn both into a pitch deck, without repeating what the project or goals are at each step. It also works ahead of you. When an investor emails asking for your latest product update as a deck, Gemini spots the request and offers to handle it; one click and it comes back with completed work, ready for sharing or review.

Five takeaways for startups

1. Coworker agents and skills help you scale your output

Beyond personal assistance, you can create persistent coworker agents that work with your entire team. Each agent comes with its own Workspace account, email address, Drive storage, and directory presence. You can define a specific role — such as a launch coordinator, QA triager, or financial analyst — and your teammates can add agents to a Chat space or @mention it in a Google Doc comment to delegate work.

You can also equip agents with reusable skills — or let Gemini write skills for itself as it learns your processes. For example, with operational reporting skills connected to BigQuery and the Knowledge Catalog, anyone on your team can ask a plain-language question to build and save a live report. Once saved, those reports run on demand without incurring additional token costs. Gemini also offers pre-built specializations, starting with legal and financial services.

What it means for startups: Coworker agents can immediately increase the capacity of your teams, and skills can standardize repeatable processes like investor and operational reporting without burning tokens on every run.

2. Open and third-party model choice

Gemini is the agent, and the model underneath is your choice. As startup architectures mature, sending every task to a single frontier model quickly eats into gross margins. With the Gemini agent, you can dynamically route workloads across Gemini models, third-party models like Anthropic’s Claude, and lightweight open models like Gemma.

What it means for startups: You get flexibility without lock-in. You can optimize your cost per task — using compact models for high-volume, deterministic jobs and frontier models for complex reasoning — and swap models as your product evolves without rebuilding your workflows.

3. Connected to your services and tools

An agent is only as useful as the systems it can reach. In addition to Google Workspace, the Gemini agent connects directly to your engineering, data, and business stack, including partner solutions:

Development and collaboration: Git, Slack, Jira and more
Data and local files: BigQuery, Postgres, Snowflake, Databricks, and more alongside files on your desktop
Custom tools via MCP: Any Model Context Protocol (MCP) server inside or outside your network so your engineers can build and share internal tools.

What it means for startups: Instead of spending engineering cycles maintaining custom connectors, your team can plug the agent straight into your existing codebase, databases, and MCP tools so it can act on live data wherever it lives.

4. Built-in security and governance

For B2B startups, the enterprise security audit is often where deals stall. Under the hood, every agent is assigned a cryptographic Agent Identity for clear audit trails, runs inside an isolated sandbox, and protected with built-in prompt injection defenses. External traffic is routed through Agent Gateway, allowing you to enforce governance policies. And managing coworker agents is as straightforward as managing users today, since they all have a Workspace account.

What it means for startups: When an enterprise buyer’s CISO asks how your AI workflows handle data privacy, prompt injection, and access control, you have an enterprise-grade architecture ready — helping you clear procurement hurdles.

5. Protect your runway with real-time cost controls

Autonomous agents that run sub-agents for hours or days can quickly run up a bill if a loop goes unchecked. You can now set hard, real-time spend caps directly in the Cloud Billing Console; if a cap is reached, the project's agent automatically pauses so a bug never turns into a billing surprise. Underneath, Gemini runs on Google’s specialized AI infrastructure, including our inference-optimized TPU 8i, which delivers 80% better performance-per-dollar than the prior generation.

What it means for startups: You and your developers can experiment with long-running, parallel agentic loops with a built-in financial safety net that protects your cash reserves and builds confidence with your backers.

Founders already building

Startups and AI-native innovators are already putting Gemini to work. Immersive storytelling platform Whenere used Dreamlands, an agentic 3D worldbuilding platform built on Gemini platform, to bring Pride and Prejudice to Fortnite — with Gemini generating textures for each new object in about 7 seconds for under 10 cents; one artist created nearly 250 custom assets in a single day and completed a historically accurate estate in under a week. They join application-building platform Lovable and fellow startups and innovators including OpenEvidence, Recursion, and Replit and more.

Build with Google for Startups

The agentic era gives lean teams the leverage to punch far above their weight — whether you’re deploying coworker agents to cover internal operations or building agentic workflows directly into your product.

While these new Gemini agent capabilities are not publicly available yet, startups can contact our sales team to learn more and discuss how to get ready. In the meantime, you can explore the Google for Startups program to access credits, hands-on technical guidance, mentorship, global founder community programs and more.

Innovation in Ireland: How Irish brands scale with Gemini Enterprise

Thu, 08 Oct 2026 12:00:00 +0000

In recent decades, Ireland has grown into a vibrant hub for global technology. As modernization accelerates, Irish organizations — spanning government agencies, established enterprise brands, and growing startups in AI and other areas — are all advancing their technical foundations. Today, leaders across the country are injecting agentic AI into every layer of their business and operational workflows.

We are seeing organizations move from early prototypes to production-ready deployments using Gemini Enterprise. This progress supports Ireland's competitive position within the European market. According to a recent Implement Consulting Group report assessing Google’s local contribution, public and private leaders estimate between 40 billion to 45 billion euros of economic potential in global AI markets.

This ambition aligns with broader government initiatives. Since assuming the presidency of the Council of the European Union for the second half of 2026, Ireland has steered higher-level technology and policy discussions, outlining a National Digital and AI Strategy to improve productivity and innovation across sectors.

To support this strategy, we are focused on making these capabilities more accessible. Our local teams are collaborating with industry leaders to implement Gemini Enterprise which offers capabilities that range from conversational AI agents and natural language data analysis, to optimizing operations with agentic ecosystems.

The economic impact is already taking shape. In 2025 alone, Google operations — including our AI and cloud services — added an estimated 10 billion euros to the Irish gross domestic product. And this momentum is shared: Organizations across Ireland are using Google Cloud to modernize infrastructure and put AI to work.

Here are several ways Irish organizations are modernizing with Google Cloud and Gemini Enterprise:

Ryanair supports millions of passengers through AI-backed operations

Ryanair, Europe’s largest airline, is modernizing its collaborative operations and infrastructure by deploying Gemini Enterprise and Google Workspace for some 35,000 employees. Flight crews, operations planners, and corporate staff will benefit from streamlined decision-making, optimized crew logistics, and enhanced corporate productivity. At the same time, a dual-cloud strategy strengthens infrastructure resilience as the airline targets growth to 300 million passengers by 2034.

Smyths Toys rethinks the customer experience with agentic support

As one the world’s largest specialty toy retailers, Smyths Toys is reshaping its customer support with Google Cloud and Gemini Enterprise. By introducing an AI agent named Codie, the retailer has resolved more than 60% of its web inquiries and reduced holiday email backlogs. Customer care teams that previously handled up to 1,000 peak-season emails a day can now focus on the roughly 50 cases that need more complex judgment.

The Irish Revenue Commissioners: Advancing public sector data

Public sector agencies are also making strides in Ireland. The Irish Revenue Commissioners — responsible for taxes, customs, and keeping Ireland’s economy running smoothly — is working with Google Cloud to migrate from legacy on-premise systems to a unified AI-lakehouse architecture designed to break down data silos and establish a single source of truth. Strict data sovereignty controls safeguard taxpayer information while Gemini Enterprise powers agentic data analysis, helping Revenue deliver secure, responsive public services.

Virgin Media Ireland: Accelerating workflows

Supported by Liberty Global's partnership with Google Cloud, Virgin Media Ireland is moving AI from exploration to production. Through modern infrastructure and automation, the team now deploys workloads three times faster, is in the process of rolling out multiple AI agents, and is launching new customer services designed to drive business growth.

Irish startups scaling with enterprise tools

While established enterprises are modernizing legacy systems, Irish startups are building on modern technology from day one. Backed by a strong local tech ecosystem, these agile teams are creating services that compete on an international stage.

Hexis: Scaling personalized sports nutrition

Hexis builds technology solutions that help athletes refine their performance and recovery. By storing and processing that real-time biometric and behavioral data in Google Cloud, Hexis uses Gemini Enterprise to power natural language conversations that explain fueling adjustments to users throughout the day. Today, the platform supports 40% of the Tour de France peloton and half of Premier League clubs during the 2026–2027 season.

Kitman Labs: Unifying performance analytics

Founded in Dublin in 2012, Kitman Labs provides sports science and performance intelligence to more than 4,000 sporting organizations across 26 countries. By combining its intelligence platform with advanced analytics and Gemini Enterprise, Kitman Labs helps teams make data-informed decisions that support athlete health, lower injury risk, and extend player career longevity.

Spryt: Automating healthcare coordination

Spryt streamlines healthcare coordination across the U.K. and U.S., using AI built on Google Cloud and Gemini Enterprise to optimize patients’ experiences, reduce missed appointments, and cut administrative burden. The company collaborates with the NHS, operates in more than 70 US clinics, and has analyzed 224 million appointment records as part of a multiyear initiative with Mayo Clinic.

IMPT: Automating carbon offsetting

By connecting AI, blockchain, and Google Cloud, IMPT embeds verified carbon offsetting into everyday hotel bookings and online purchases. Following a 15-week migration to Google Cloud, IMPT supports 8 million listings across 195 countries. With Gemini Enterprise, the company’s lean team handles about 400 daily customer chats with a six-second median response time, allowing staff to focus on growing its business rather than managing systems.

Transform ideas from planning to production

As Ireland continues to expand as a global technology hub, local organizations are demonstrating how to translate AI potential into practical results. Explore our customer stories to see how other organizations are building with Google Cloud, or reach out to our team in Ireland to accelerate your journey.

AI21 achieves an 83% reduction in time-to-start for AI workloads with AI Hypercomputer

Fri, 02 Oct 2026 19:00:00 +0000

Editor’s note: AI21 Labs is a leading global AI lab with a long track record of building foundation models, most notably the Jamba family, and today focuses on specialized LLMs and agent optimization technology. By adopting Google Cloud AI Hypercomputer, AI21 cut high-priority job wait times from 72 hours to 12 and manual scheduling interventions from 20 per week to zero.

At AI21, we build foundation models and agent optimization products that help enterprises run agents at frontier quality, efficiently. Our language models, including the Jamba family, and our agent optimization product suite run demanding production workloads, including our own. We chose Google Cloud AI Hypercomputer to support them at scale.

To keep our model training runs highly utilized, we needed a performant, scalable environment codesigned across infrastructure, orchestration, and consumption models. Our model training runs on one of our shared Google Kubernetes Engine (GKE) clusters, pooling thousands of Google Cloud A3 (powered by NVIDIA H100 Tensor Core GPUs) and A3 Ultra (powered by NVIDIA H200 Tensor Core GPUs) instances, so any team can draw on the full capacity of the fleet rather than being boxed into its own slice. The cluster also trains models and agent-optimization workloads beyond the Jamba family. That approach keeps utilization high, and it makes scheduling hard.

The gridlock of high-utilization clusters

Prior to leveraging GKE for orchestration, we used to negotiate capacity by hand in Slack. If you needed capacity for a training run, you posted in #gpu-resources and hoped for the best.

That worked fine when the cluster had headroom. It stopped working once utilization pinned near 100%, which is where you want a reserved compute fleet to sit.

Over time, every request became a negotiation. Team leads spent their time refereeing compute disputes. Our high-priority jobs — the large, multi-node training runs that need half or more of the cluster at once and serve as the critical path for model projects — could sit blocked for up to 72 hours waiting for enough contiguous capacity to open up.

Figure 1: A typical day on #gpu-resources — manual requests, ad-hoc coordination, and researchers waiting on replies.

Scarcity created two distinct problems, and it took us a while to see them as separate. The first was contention: determining who gets compute access next, which we resolved through negotiation. The second was fragmentation: capacity that was technically free but scattered in pieces too small for a large job to use, a bin-packing problem no amount of negotiation could fix.

Figure 2: GPU fragmentation. 8 free GPUs scattered as 1+1+4+2 across four nodes can’t fit an 8-GPU workload.

Sometimes we had plenty of capacity free on paper, but it was scattered across different machines in chunks too small for a larger job to actually land. Without all-or-nothing admission, the cluster could reach a deadlock, with machines holding resources without doing useful work until someone stepped in manually.

It was clear the status quo wasn’t working and we needed something better.

Choosing the right scheduler

We looked at a few open-source batch schedulers, including Apache YuniKorn, Volcano, and Kueue. YuniKorn didn’t cover all our use cases. And while Volcano had more features, integrating it with our environment would have required replacing core Kubernetes scheduler components.

Kueue won on simplicity and integration. It worked with standard Kubernetes, didn’t require replacing core components, and didn’t force us to rewrite our job specs.

Pairing Kueue with AI Hypercomputer’s flexible, open operations through GKE also contributed to our success. We rely on GKE because it gives us the right level of control for compute-intensive AI work — close access to GPU hardware and drivers, without the overhead of managing raw instances ourselves. Kueue’s native integration with GKE, including with Google Cloud capacity types like Spot VMs and Dynamic Workload Scheduler, meant that once our reserved capacity filled up, the same scheduling logic could reach out to elastic capacity automatically instead of leaving jobs stuck.

Closing the loop with open source

Adopting Kueue turned into something bigger than a simple process change. Because we partner with Google Cloud, we have a direct line to a Technical Account Manager, who saw an opportunity to make AI21 a design partner for the Kueue team. This way, we wouldn’t just be a user, but a source of real production requirements that could help shape where the tool went next.

One of the first things to come out of that partnership was a requirements document we shared with the Kueue team, describing behavior we needed that didn’t exist yet: fair admission ordering across teams for multi-node jobs, without the preemption that usually comes bundled with fairness. The Kueue team built it. Admission Fair Sharing (AFS) reorders the admission queue to favor teams that have historically used less capacity without disrupting jobs already running. Around the same time, we also turned on Topology Aware Scheduling. This existing feature made Kueue aware of our physical cluster layout, so it can refuse to admit jobs that won’t fit on a single node rather than placing them with nowhere to run.

For us, that’s what the best-case open-source feedback loop looks like: Real requirements surfaced through production use, fed directly back into Kueue.

Same fleet, less friction

The Slack #gpu-resources channel is archived now. Every workload, whether a debug pod, a multi-node training run, or an inference deployment, gets queued, prioritized, and scheduled automatically. The results were immediate. Manual interventions dropped from about 20 a week to zero. High-priority jobs that used to wait up to 72 hours now wait 12. Fragmentation across the cluster fell from 15% to 8%, and the “zombie job” problem — workloads partially admitted with nowhere to run — is gone.

None of this changed our total cost. We run our reserved fleet at close to 100% utilization on purpose, so raw spend was never the variable we were optimizing. What changed is where our people’s time goes. Team leads aren’t refereeing compute disputes anymore, and researchers aren’t waiting on replies in Slack. That frees everyone to run more experiments and iterate faster.

Scribd, Inc. classifies more than 400 million documents with Gemini batch inference on Gemini Enterprise

Thu, 24 Sep 2026 16:00:00 +0000

Scribd, Inc. is home to one of the world's largest collections of human-created content.

Scribd’s products leverage one of the world's largest collections of human-created content and intelligent tools to help people move from information access to real understanding and application.

This past year, Scribd used Gemini's native PDF understanding and Gemini Enterprise batch prediction to run trust and safety classification across its entire user-generated content corpus of more than 400 million documents, spanning over 12 billion pages, in a matter of months.

Here were the results:

Classified 400M+ user-uploaded documents (12B+ pages of text and images) across Scribd and Slideshare
Completed the corpus-wide backfill in a matter of months, with Google Cloud scaling batch throughput to meet the timeline
Native PDF input meant more than 99% of the corpus was processed as-is, with no OCR, rendering, or screenshotting pipeline to build
Gemini Enterprise’s batch prediction at a 50% discount to interactive pricing made LLM classification viable at corpus scale

Trust and safety at the scale of an entire corpus

Scribd, Inc. is the parent company to four distinct products: Scribd, Slideshare, Everand, and Fable. Across Scribd and Slideshare, hundreds of millions of user-uploaded PDFs, presentations, and documents help people find information, build understanding, and finish projects. With that scale comes responsibility. We aim to balance access with protecting our communities. We leverage a mix of human and automated methods to review and best ensure the content on our platforms complies with our community rules. As the corpus continues to grow and technology evolves, this challenge requires even more resources.

Understanding a document requires reading its text and its images together, in context. Classification has to work across all possible use cases, all possible languages, all possible contexts. There is no single solution that can translate cleanly across all of it. And each policy area traditionally demanded its own specialized detection model, which meant either years of in-house engineering effort or specialized vendor solutions that don't fit the economics of a 400-million-document backfill. The team evaluated several off-the-shelf moderation tools and open models, but none delivered the quality they needed at their scale.

“This is a genuinely hard problem that we have been working on for a long time. Every category of content behaves differently, and historically each one required its own specialized solution. Gemini collapsed all of that into one model, one prompt, and one pipeline.” – Sachin Sebastian, Senior Engineering Manager, Scribd, Inc.

Why Gemini: PDFs are a first-class input

The turning point was realizing that Gemini treats Scribd's corpus the way it actually exists: as PDFs. Gemini accepts PDF input natively and reads each page as both text and image, so a single multimodal model could evaluate everything from dense text documents to image-heavy presentations, with no OCR pipeline, page rendering, or screenshot infrastructure in between. Because Gemini processes each PDF page at a fixed, predictable token count, costs scale linearly and stay low even across 12 billion pages.

After benchmarking model families and versions, the team selected Gemini 2.5 Flash Lite as the classification workhorse, with Gemini 2.5 Pro serving as an LLM judge in a full second consistency pass over the corpus to validate output quality. In the team's evaluations, Gemini's multimodal understanding caught visual policy signals that text-only moderation endpoints routinely missed.

“Gemini's peculiar advantage is that it meets our content in its native format. It reads the text, layout, and images of a PDF directly. More than 99% of our corpus went in exactly as it lives on our site without any pre-processing” – Sachin Sebastian, Senior Engineering Manager, Scribd, Inc.

Batch prediction, simple enough to bet the corpus on

The execution model was deliberately simple. Documents were staged in Cloud Storage, submitted to Gemini Enterprise batch prediction, and the results flowed back into the team's data platform for downstream analysis. There was no serving infrastructure to operate, no rate-limiting logic to write, and no GPU capacity to manage.

Batch pricing, at 50% below interactive rates, is what made the economics work at corpus scale. The team later layered on Gemini Enterprise’s implicit prefix caching, restructuring prompts so the static policy text hit the cache, which pushed efficiency further with no loss in classification quality.

A partnership measured in throughput

Processing 400 million documents is ultimately a throughput problem, and this is where the partnership with Google Cloud mattered most. Scribd's team connected directly with Google Cloud engineering and product to plan the backfill, advise on region strategy, and make sure the right capacity was in place ahead of launch.

As the backfill ramped up, Google Cloud worked closely with the team to scale throughput to the demands of the project. The effect was dramatic: batch jobs began completing far faster than projected, and for much of the run Gemini Enterprise was not the bottleneck. Scribd's own upstream pipeline was.

“Google Cloud didn't just answer support tickets. They partnered with us on the backfill, and there were stretches where Gemini Enterprise finished work faster than our own systems could produce it. That is a good problem to have.” – Sachin Sebastian, Senior Engineering Manager, Scribd, Inc.

What's next

The backfill is now the foundation of an ongoing program: newly uploaded content flows through the same Gemini classification pipeline, keeping the corpus continuously evaluated rather than periodically cleaned. And because the pattern of PDFs in Cloud Storage, Gemini batch prediction, and results in the lakehouse proved so operationally simple, the team is applying it to a growing set of content-understanding workloads across its platforms.

“This project changed how we think about our roadmap. Work we had classified as multi-year, multi-team efforts is now a prompt, a batch pipeline, and a few weeks of runtime.” – Sachin Sebastian, Senior Engineering Manager, Scribd, Inc.

^{This work was a collaboration between Google Cloud and Scribd. We'd like to thank everyone involved for their support throughout this project:}

^{Scribd Engineering: Anish Kumar, Jeanie Lam, James Watkins, Hima Alladi}
^{Scribd Applied Research: Rafael Pedrosa Lacerda de Melo, Kara Killough, Eric Chang}
^{Scribd Product: Seyoon Kim, Nicole Pauls}
^{Google Cloud AI Batch Inference team: James Liu, Digvijay Singh, Wei-chung Wang, Yan Wang, Kun Shi}
^{Google Cloud Customer Engineer: Jennifer Liang}

How growing Latin American midsize businesses are building in the AI era

Thu, 24 Sep 2026 14:30:00 +0000

Latin America’s small and medium-sized businesses are the heartbeat of the region's economy — accounting for more than 60% of total employment in the region, according to United Nations estimates. And just like their enterprise peers, everywhere you look, ambitious teams are moving fast to embrace AI.

Many have already transitioned from experimenting with generative tools and agentic workflows to using them every day to work smarter, save time, and deliver exceptional customer experiences. These growing businesses are particularly focused on maximizing the benefit they get from their investments in AI, whether that’s using a fast, low-cost model to summarize daily emails or deploying an advanced model for complex data analysis, teams can match the right AI capability to their exact task and budget.

It’s this range of options, and a familiarity with the broader suite of Google business, media, and advertising tools that has led many SMBs to choose Google Cloud, and Gemini Enterprise in particular, as their AI platform of choice. By doing so, they’re able to build custom AI agents, streamline daily tasks and paperwork, and offer customers instant support with the speed and reach needed to compete on a global scale.

With our unique front row seat, we’ve seen the benefit SMBs are getting from leveraging Gemini Enterprise, not only for generative AI, but as a catalyst for adopting other essential cloud tools like Google Kubernetes Engine and BigQuery for complete end-to-end modernization. The number of Latin American-based small and medium businesses using Google Cloud AI tools has grown 8x year-over-year and the number of Brazil based small and medium businesses using Google Cloud AI tools has grown 9x year-over-year. This rapid adoption spans our Gemini models, Gemini Enterprise, and core Cloud infrastructure, and are helping businesses to:

Roll out better customer support systems to help escalate and resolve customer support calls more quickly.
Automate repetitive actions in areas like payroll and accounting.
Help more employees understand and leverage data at work — even those not trained as data analysts.
Rapidly create and implement new designs for marketing collateral.
Help more people build their own AI agents to help them in their everyday jobs.

As we head into today’s Google Cloud Summit in Brazil, we were proud to showcase nearly 20 of our newest Latin American SMB customers using Google AI to reduce busywork, serve their customers faster, and grow their businesses.

Announcing new Latin American customers putting Google AI to work

AdGoat, an Argentina-based adtech company processing more than 10 billion annual ad requests across more than 100 global websites. It uses Cloud Run, the Gemini API, and Gemini Enterprise to automate content analysis, ad bidding, and audience targeting to help e-commerce brands drive higher campaign returns.
Angelus, a Brazilian dental and healthcare manufacturing company, uses Gemini Enterprise to streamline project management across its research and development department. This enables its teams to automatically pull technical project data into pre-approved templates aligned with the company’s brand identity and regulatory requirements.
BunkerDB, a marketing science company operating across Latin America, uses Gemini Enterprise, Cloud Run, and Cloud Storage to power an AI platform that organizes marketing assets, checks brand compliance, generates or adapts multimodal content, and predicts ad performance before launch. All of this helps it reduce creative turnaround times from weeks to hours and cut cost per lead by up to 25%.
Caffeine Army, a Brazilian wellness and high-performance company that connects people with solutions in nutrition, sports, and well-being, deployed BigQuery and Gemini Enterprise on Google Cloud to unify customer purchase insights, enabling faster creative campaign turnarounds and boosting team productivity across the organization.
Convert, a Brazilian marketing and analytics provider, uses Looker, BigQuery, and Cloud Run to power five specialized AI agents that answer complex business questions in natural language, speeding up report deliveries by 65% and reducing operational costs by 32%.
Growth Digital, a Google Ad sales rep operating across 13 Latin American countries, used BigQuery and Gemini Enterprise to build over 113 AI agents, enabling teams to build proposals 5x faster, cut campaign reporting time by 80%, and reduce financial error rates to under 0.01%.
GrupoTusMaquinas.com, an equipment management platform based in Chile, deployed Google Cloud AI tools and Gemini models to create digital tracking profiles for trucks and machinery, allowing businesses to query fleet status in plain language and manage vehicles regardless of brand or location.
HealthAtom, a healthcare technology company, uses the Gemini API, Firestore, and Cloud Functions to power AI assistants across its clinical platforms, automating appointment scheduling and medical record reviews while supporting 80 million annual patient interactions.
KLog.co, a Chilean logistics technology company digitizing freight forwarding across Latin America, uses Gemini Enterprise, BigQuery, and Google Workspace to automate cargo tracking and shipping paperwork, cutting manual data entry errors by over 90% and increasing document processing capacity tenfold.
NEEOH, a leading Brazilian out-of-home advertising communication platform, uses Gemini Enterprise to standardize secure AI usage across its organization, enabling teams to generate campaign copy and build pitch proposals faster while keeping corporate client data secure.
Luxia Agro, an Argentinian foreign trade supplier of crop protection products, uses Gemini 3.5 Flash and Gemini Enterprise to automatically pull key details from complicated shipping emails and update their central business systems. This allows it to automate 80% of foreign trade operations and cut manual processing errors in half.
Macal, a Chilean auction company, uses the Gemini Enterprise, Cloud Run, BigQuery, and Security Command Center to automatically verify property records and modernize its technology systems, cutting software development times from weeks to days and lowering infrastructure costs by up to 30%.
Ninecon, a Brazilian tech consulting firm, deployed Gemini Enterprise to integrate AI directly into employee workflows, allowing managers to track usage patterns and optimize project turnaround times with real-time insights.
Nova Gestões, a customer service and operations provider in Brazil, uses Cloud Speech-to-Text and Gemini Enterprise to translate and analyze 100% of customer calls in real time, reducing post-call manual data entry and boosting team productivity by 30%.
Romi, a Brazilian industrial machinery manufacturer, uses the Gemini API and Gemini Enterprise to power an interactive chat assistant directly on CNC machine HMI (human machine iInterface), giving factory operators instant answers grounded in official manuals and generating QR codes for step-by-step instructional videos.
Supermercados El Dorado, a leading supermarket chain in Uruguay, leverages Google Compute Engine and Gemini Enterprise to modernize legacy testing infrastructure and connect custom AI agents within their daily workflows, boosting team productivity across departments.
Tryvia, a Brazilian IT and business solutions provider, uses Google Cloud, Looker, and Gemini Enterprise to move off legacy physical servers, giving teams real-time reporting dashboards and AI tools that speed up software development and daily tasks.
Via Cristais, a major highway operator in Brazil, leverages Google Contact Center as a Service to speed up emergency routing for highway accidents, reducing caller wait times, improving driver satisfaction, and mitigating the impact of call center staff turnover.
WeSpeak, an AI conversational platform for the hospitality industry in Latin America, uses Cloud Run, Gemini Pro, and Gemini Flash to automate end-to-end guest interactions across messaging channels like WhatsApp and Instagram. This has helped it achieve an 85% resolution rate and a 2x increase in overall sales volume for hotel clients.

Helping your team build AI skills

To help growing teams get the absolute most out of AI, we’ve created easy, no-cost learning programs that anyone can use:

Programs for small and medium businesses: Explore beginner-friendly training paths or join specialized programs to learn how to build custom AI assistants for your day-to-day work.
Google skills for organizations: Access thousands of free, on-demand AI courses and hands-on practice labs designed by experts at Google Cloud and Google DeepMind.
Get certified: Help your staff gain industry-recognized AI certificates through guided courses, expert mentoring, and skill badges.

By offering easy-to-use tools and free training — from everyday office apps in Workspace to advanced AI on Google Cloud — Google is here to help Latin American businesses thrive today and in the future.

The future of orchestration: Pine59’s journey to Airflow 3 on Google Cloud

Thu, 17 Sep 2026 17:00:00 +0000

Operating large data pipelines requires an orchestration layer that scales smoothly as workloads expand. When your pipelines process millions of complex data points every day to feed predictive models, staying up-to-date with your technology stack is a strategic necessity.

Pine59 provides location intelligence data through data pipelines that produce analytical metrics on cadences ranging from hourly to quarterly. One of the company’s most data-intensive metrics, Daily Foot Traffic, computes data for as many as 14 million distinct locations in a single job. To handle this massive volume, Pine59’s system runs entirely on Google Cloud, with the heavy lifting in BigQuery and all of it orchestrated by Managed Service for Apache Airflow (formerly Cloud Composer) running Apache Airflow 3.

As the company’s volume of data and number of machine learning workloads scaled up, Pine59 decided to modernize its monorepo, which contains hundreds of directed acyclic graphs (DAGs). Here is a look at how that transition improved Pine59’s MLOps capabilities, developer workflow, and pipeline speed.

Proactive modernization for growth

Pine59 has long relied on a shared monorepo with code and tooling spanning multiple projects to run its metric production pipelines. As it considered its infrastructure’s future, the company wanted to help its data pipelines run faster and more reliably.

That’s why it decided to stress-test production workloads against the newly available Managed Airflow (Gen 3) architecture running Airflow 3. The initial results were unambiguous: the Gen 3 environment delivered immediate and significant processing speed, task scheduling, and overall stability improvements. Recognizing the clear potential for performance gains, Pine59 initiated a full transition to the new environment.

Orchestrating advanced MLOps

Pine59’s pipelines don’t just move data; they drive complex ML models, so a core aspect of its migration was optimizing the orchestration of its ML inference workloads.

Previously, Pine59 had used standard Kubernetes operators for these tasks. By moving to Managed Airflow (Gen 3), which features a highly optimized and abstracted infrastructure layer, the company’s engineering team refined its MLOps architecture. They did so by setting up a dedicated Google Kubernetes Engine (GKE) cluster that was specifically optimized for model inference and integrated it into the Pine59 pipelines.

This clear separation of orchestration and heavy ML execution compute allows data processing and model inference to run efficiently, showcasing Managed Airflow as a resilient, scalable backbone for enterprise MLOps.

Supporting developers with custom extensibility

Beyond infrastructure improvements, Pine59 was also able to immediately capitalize on Airflow 3’s delivery of a vastly improved developer workflow and user interface. Indeed, managing hundreds of interconnected DAGs requires excellent observability, and Pine59 found Airflow 3’s plugin authoring system remarkably easy to use.

To improve internal developer velocity, the company quickly built a number of custom plugins that it integrated directly into its new Airflow UI:

BigQuery Auto-linkify: A tool that automatically detects internal BigQuery table references within the Airflow Logs and XCom tabs, dynamically generating direct links to BigQuery Studio for faster debugging (available as a public GitHub gist)
DAG Run Configuration Search: A custom search form added directly to the DAG overview page. It allows Pine59 engineers to query specific key-value pairs within DAG run payloads (configs) and instantly surface matching runs. This in turn drastically reduces troubleshooting time.

In addition, the team also deployed a compatibility shim layer within its monorepo. This “compat” module dynamically abstracts logic between Airflow versions, streamlining operator migration across versions.

Faster, more reliable pipelines

For Pine59, migrating to Managed Airflow (Gen 3) with Airflow 3 has yielded clear, quantifiable results.

The most important improvement was the speed of its DAG runs. In the company’s previous setup, tasks often got stuck in a queued state during peak processing surges. With Gen 3, queue latency has dropped dramatically, allowing tasks to start running almost immediately.

Consider the comparison below of total aggregated “queued” & “running” time of more than 300 runs of the same DAG between Managed Airflow (Gen2) with Airflow 2.11 vs. Managed Airflow (Gen3) with Airflow 3.1 below. As we can readily see, the difference in queued time is significant.

Coupled with internal DAG optimizations made during the transition, the performance gains are also highly tangible. For example, the Daily Foot Traffic pipeline previously took nearly 38 minutes to complete. With the new instance, the same workload now takes less than 26 minutes —nearly 32% less processing time.

Today, Pine59 processes all its production workloads on its new Managed Airflow (Gen 3) instance. By moving to this next generation orchestration, the company improved its MLOps capabilities, equipped its developers with better tools, and built a faster, more resilient foundation for future workloads.

If your engineering team spends more time managing infrastructure than delivering value, consider a similar transition and discover how it can help you move from maintaining servers to building the future of your data and AI pipelines today.

^{Special thanks to the following contributor to this post: Alexandre Crespo-Perez}

How a solo founder runs a five-continent tender platform on AlloyDB and MCP

Thu, 17 Sep 2026 16:00:00 +0000

Editor's note: Lucius AI, a tender-intelligence startup covering markets across five continents, runs its entire data platform on AlloyDB for PostgreSQL with a single operator. By migrating semantic search to a ScaNN index and managing database operations through Model Context Protocol (MCP), query latency dropped by 47x while automating day-to-day administrative tasks via MCP.

Executive summary

Lucius AI runs a global tender platform spanning more than 210,000 tenders across the UK, EU, India, and Australia, requiring minimal operational overhead for a solo founder.
Lucius AI deployed AlloyDB for PostgreSQL to consolidate its relational catalog, audit logs, and vector embeddings into a single managed database engine.
Migrating semantic search to a ScaNN index lowered query latency from 1.14 seconds to 24 milliseconds — a 47x speedup on a representative production query.
Connecting an AI agent to AlloyDB using the Model Context Protocol (MCP) helps Lucius AI automate query analysis, data freshness checks, and incident forensics under strict least-privilege permissions.

Making tender intelligence work as a company of one

Lucius AI helps businesses bidding on public contracts evaluate opportunities across global markets. The platform ingests public procurement notices from the UK, the EU, the US and Canada, Australia and New Zealand, India and Singapore, alongside World Bank donor-funded notices across Africa and Asia. Lucius AI analyzes tender documents using Gemini to generate compliance matrices, bid recommendations, and draft responses citing original source pages. For small and mid-sized suppliers, this replaces days of manual document reviews and costly external consulting.

Running a platform of this scope requires extensive operational coordination:

Nightly ingestion from thirteen public procurement sources
A catalog of more than 210,000 tenders, including tens of thousands open for active bidding
Two production regions on Cloud Run: Europe, and an Australian deployment on its own AlloyDB cluster with customer-managed encryption keys (CMEK) for defense-adjacent customers
Ongoing analytics, performance tuning, data validation, and incident response

Managing these responsibilities without dedicated data engineering or database administration teams requires offloading operational maintenance. Lucius AI addressed this challenge on two fronts: using AlloyDB for PostgreSQL as the core system of record, and connecting an AI agent through the Model Context Protocol (MCP) to safely execute database operations.

Consolidating systems into AlloyDB

Rather than deploying separate relational databases, vector databases, and log stores, Lucius AI houses all core data in AlloyDB for PostgreSQL. The relational tender catalog, document metadata, audit logs, and vector embeddings reside in the same database engine. Storing vector embeddings alongside relational rows avoids managing separate vector stores, establishes a unified backup schedule, and centralizes identity management.

Authentication relies strictly on Cloud IAM. Services connect using dedicated Google Cloud service accounts mapped to database roles scoped to specific access requirements, without storing database passwords in application environments. Database reliability is managed natively by AlloyDB through automated backups and point-in-time recovery, avoiding custom disaster recovery procedures.

In production, this consolidated architecture supports:

More than 210,000 tenders in the catalog, with embeddings stored directly alongside them
Rebuilding the semantic index embedded 115,820 records in 10.6 minutes with the Gemini embedding model, for around three dollars in API spend; AlloyDB auto embeddings now keep those vectors current.
Retrieval reranking executed directly inside the database using the ai.rank function — with mean latency of 77-milliseconds - returning the most relevant results for search queries without requiring a standalone reranking microservice

Accelerating semantic search by 47x

Semantic search across the tender catalog initially relied on unindexed vector comparisons, where a representative query took 1.14 seconds. Migrating this workload to a ScaNN index in AlloyDB reduced query latency to 24 milliseconds — a 47x improvement.

The index recommendation originated from the AI agent during an automated performance audit, where it benchmarked the query plan before preparing the index migration.

Automating database operations with MCP

To delegate routine administrative tasks, Lucius AI configured the open-source MCP Toolbox for Databases using the prebuilt alloydb-postgres server.

Operational delegation requires strict access controls. The agent connects using a dedicated PostgreSQL role granted SELECT across the schema and UPDATE on a single operational table. Destructive commands (DROP, DELETE, TRUNCATE) are omitted, restricting agent actions to authorized operational boundaries.

Under this configuration, the AI agent performs regular database operations across four key areas:

On-demand analytics: Compiles retention cohorts, activation funnels, and catalog coverage by country via ad hoc SQL queries, removing the need to build and maintain manual dashboards or complex analytical pipelines.
Performance optimization: Performs query-plan inspections and index analysis, such as identifying the ScaNN indexing strategy.
Incident forensics: In response to an external security probe, the agent parsed audit logs to reconstruct the request timeline in minutes, verifying that tenant isolation remained intact.
Automated data-quality checks: Evaluates ingestion watermarks and freshness across all thirteen procurement sources every morning.

For teams adopting this architecture, establishing a progressive permission structure provides clear guardrails: start with read-only access, expand permissions as requirements dictate, and keep destructive operations restricted to human administrators.

Looking ahead

Lucius AI is planning three technical initiatives to further reduce operational overhead:

Automated vector embeddings in AlloyDB AI: After validating ai.initialize_embeddings across the full catalog, a weekly maintenance job uses ai.refresh_embeddings to update vectors.
Columnar engine acceleration: Having enabled AlloyDB’s columnar engine with auto-columnarization, the database identified and stored 40 frequently queried columns across four tables in memory within a day, accelerating reporting queries without a separate analytical store.
Managed Remote MCP Server: Transitioning from self-hosted Toolbox processes to Google Cloud's fully managed Remote MCP Server for AlloyDB will offload MCP server hosting and maintenance.

By anchoring core data in AlloyDB and managing routine operations through MCP, Lucius AI demonstrates how a single engineer can build and operate a resilient, multi-region procurement platform.

To explore Lucius AI, visit ailucius.com. To evaluate AlloyDB for PostgreSQL, deploy an AlloyDB cluster to test performance against your own workloads.

For SeaVerse, GKE Agent Sandbox reduces infrastructure costs by 60%

Wed, 16 Sep 2026 16:00:00 +0000

Editor’s note: Today we hear from SeaVerse, a gaming startup from SeaArt that is building a platform for playable AI experiences, where users can open lightweight games, character chats, and interactive apps, or create their own experiences from a prompt. To support that creative loop, SeaVerse needed infrastructure that could run dynamic, multi-tenant sandbox workloads with strong isolation, low latency, better observability, and more flexible costs. Google Kubernetes Engine (GKE) and GKE Agent Sandbox gave SeaVerse the managed foundation from which to execute these AI workloads, helping the team reduce their infrastructure costs by up to 60%, while giving creators a faster path from idea to playable experiences. Read on to learn more.

What if AI were a playground? Welcome to SeaVerse, a creation-first platform for playable AI experiences. Here, an AI creation can be as peaceful as drawing a path for a snake to follow, or as chaotic as a music-backed stickman simulation. Some people come to play lightweight games. Others come to chat with AI characters, try interactive apps, create visual patterns, share what they made, or remix an idea into something new.

We built SeaVerse around a simple promise: Every experience should feel immediate and easy to share. A creator should be able to describe an idea in plain language, refine the result, and publish it in moments, without a traditional coding workflow.

Delivering that simplicity requires serious infrastructure. Every creation that users make moves through the same chain: generate, run, preview, debug, publish, remix. If any part of that chain is slow, unstable, or poorly isolated, users feel it immediately. That’s why we turned to GKE and GKE Agent Sandbox.

The infrastructure challenge of instant interaction

What looks effortless to a user is anything but on our end. Every creation on SeaVerse runs as a distinct workload and is expected to behave reliably from the first interaction.

Because each workload runs in its own environment, we needed clear security boundaries between users, creations, and sandboxes. But overly strict isolation could slow the very creative loop we were trying to protect, and when something went wrong, diagnosing it was costly. Our engineers had to trace problems across multiple parts of the execution chain with little visibility into what was happening inside the environment.

We explored existing sandbox approaches, but needed deeper kernel-level isolation and native observability at scale to support fast diagnosis across multi-tenant environments. Something had to change.

Building on GKE and GKE Agent Sandbox

We chose GKE because we needed a reliable, secure way to operate Kubernetes without turning our engineering team into a cluster maintenance team. GKE brought together the proven ecosystem and operational tooling we needed, freeing us to focus on building the platform rather than managing the infrastructure beneath it.

As a Kubernetes primitive designed for agent code execution and computer use, GKE Agent Sandbox addressed our requirement for strong isolation, enforcing strong security boundaries without slowing down the creation experience. By utilizing GKE Agent Sandbox with Kata Containers+Cloudhypervisor (microVM), we’ve achieved the perfect balance of multi-cloud flexibility and robust security, option to switch isolation runtime between microVM and gVisor, running our AI sandboxes safely. GKE empowers us to scale toward our long-term vision of supporting over a million sandboxes. Built on gVisor, it provides kernel-level isolation for dynamic sandbox workloads while preserving the Kubernetes orchestration model, so that they can be managed through the same scheduling, monitoring, and operations as the rest of the cluster.

With SeaVerse, users can generate interactive experiences from a single prompt. After an experience is generated, GKE Agent Sandbox supports the run, test, integration, and verification steps needed to make it ready to preview, refine, and publish. At general availability, it supports allocating up to 300 sandboxes per second, per cluster, with 90% of allocations completing in 200 milliseconds. Together, GKE and GKE Agent Sandbox gave us a reliable foundation for AI-generated interactive workloads that helped keep our team focused on the product experience.

From black box to glass box

Before GKE Agent Sandbox, a failed sandbox workload could feel like flying blind. We could often see that something had gone wrong, but didn’t have enough runtime status, metrics, or failure signals to understand why.

Now, Google Cloud’s native logging and monitoring reach directly into those sandboxed environments, giving us a clearer view of workload behavior, faster issue resolution, and a stronger foundation for managing multi-tenant workloads.

That visibility matters to developers, but it also matters to the platform’s users: A creator never sees the logs, the cluster, or the orchestration layer. They see whether an experience opens quickly, whether it responds when they draw, click, chat, or share, and whether they can keep building without friction.

Flexibility that translates to savings

GKE Agent Sandbox also changed how we think about cost. Previously, running secure sandboxed environments meant stronger dependencies on specific server types, which limited how precisely we could match resources to each workload. With GKE Agent Sandbox, we can run secure, isolated workloads on appropriately sized cloud VMs. This gives us greater flexibility in resource allocation and helped us cut our infrastructure costs by up to 60%.

That same flexibility extended to storage. Not all SeaVerse creations are built in a single session. Some evolve over time as creators return to refine them, build on earlier ideas, or invite others to remix what they’ve made. Our previous architecture didn’t support the persistent file-system capabilities those more complex use cases demanded, but that gap is gone now. We can attach persistent storage where workloads require it while maintaining the isolation boundaries that multi-tenant AI experiences need. For creators, that means experiences that are fast to open and easier to refine, revisit, and build on over time.

The next remix

Supporting creations that can evolve and deepen is central to what we’re building. It’s still early in what playable AI can become. As the platform grows, we need to keep strengthening what matters most: stability, observability, elastic scaling, and cost efficiency, all in service of a creator experience that stays fast, reliable, and expressive.

We’re also exploring additional Google Cloud tools to support smarter analytics and creation assistance. Gemini and agent models could help operators and creators better understand how experiences perform. BigQuery AI and ML capabilities can support use cases such as churn prediction, LTV and ROI prediction, and user segmentation. Multimodal tools such as Imagen and Veo on Gemini Enterprise Agent Platform open up new possibilities for material analysis, creative generation, and AI interactive content production.

Our goal is to make AI experiences feel immediate, expressive, and connected. With GKE and GKE Agent Sandbox, we have a stronger foundation for the next generation of playable AI.

How Orange built FinOps accountability, and why agents are next

Wed, 16 Sep 2026 16:00:00 +0000

At Orange, the leading France-based multinational telecom provider, there are days when engineering teams set aside their delivery backlogs and spend the day cleaning up cloud spend together. There's a leaderboard. There are goodies on the line. Experienced practitioners guide the newcomers, so people learn the work while doing it. By the end of the day, sponsors can see the results.

Orange calls these FinOps Clean Days. Together with gamified hackathons, they've earned the company's 100-plus person FinOps community a Net Promoter Score within the organization that’s above 70.

Those numbers point at something the wider industry is wrestling with. Recent State of FinOps reports identify getting engineers to take action as one of the top challenges organizations face. Moving from awareness to action means finding ways to build FinOps accountability, and to get teams to genuinely care.

That makes FinOps a business change problem. And business change problems have known solutions. We spoke with Camille Marini, the FinOps lead at Orange, to get a deeper understanding of how the company overcame these hurdles to accelerate AI adoption and ROI, and how your organization might follow the same course.

Why the Clean Days work

Orange has held two principles since it set up its FinOps team. First, Cloud FinOps is a shared responsibility, with every stakeholder in a project involved in their own way. And the only path to that shared responsibility runs through communication and a deliberate change effort.

“We insisted on the concept of shared responsibility across the organization for our FinOps practices,” Marini told us. “It’s very similar to how we approach cloud security. We needed to make teams understand that every single stakeholder in a project is involved in FinOps, each in their own way, if we are going to achieve responsible and impactful AI spending and usage.”

Those principles led Orange to create a FinOps Community of Practice, with support from Google Cloud Consulting. The team ran it on standardized communication channels so the methodology reached well beyond the central group, and kept the meetings actionable, sharing optimizations and billing updates so every session provided value.

The Clean Days came from a clear-eyed reading of how agile teams actually operate. In agile environments with deployment running constantly, optimization work rarely wins against the sprint. Delivery priorities, backlogs, and daily operations take the available time first. So Orange created protected time, made it collaborative, and made it fun.

McKinsey's four building blocks of change explain why this approach lands. Any large organizational change, the framework holds, requires action across four areas:

Conviction and understanding: "I know what is expected of me and I agree with it."
Formal mechanisms: "The structures, processes, and systems reinforce the change."
Role modeling: "I see my leaders and colleagues behaving differently."
Talent and skills: "I have the skills and opportunities to behave in a new way."

Map Orange's practice onto those blocks and the pattern is visible. Gamification and rewards give engineers colleagues to emulate: The leaderboard makes different behavior visible, and sponsors see the quick wins for themselves. Experienced practitioners guiding novices builds talent and skills through the community itself. The regular sessions, sharing optimizations and billing updates, build the conviction that comes from knowing where the money goes.

FinOps activities mapped to the four building blocks of change, with the points where AI agents can reinforce them.

What happens beyond 100 people

A community of 100 engaged people is an achievement. But in an organization with thousands of engineers, no central FinOps team can reach everyone directly. The question for leaders is how to extend what a community like Orange's creates — the awareness, the shared ownership, the habit of acting — to people the FinOps team will never meet.

This is where AI agents extend the capabilities of a FinOps team with two core benefits. They take on complex, time-intensive activities that previously needed a human, and they reduce friction around FinOps for individuals across the business.

Getting teams to adopt them takes a strategy aimed at your own organization's pain points, which often come from high cognitive load, unclear accountability, or competing priorities. Start by finding where engagement drops off in your FinOps lifecycle:

An awareness gap: If teams are unsure of their spend impact, an insight agent can push real-time cost data into their daily tools.
A bandwidth gap: If engineers are too busy with backlogs, a remediation agent can identify quick wins and present them as ready-to-merge code changes.
A complexity gap: If reporting feels like a manual chore, an orchestration agent can gather the data and simplify the process.

Start with trust, then add autonomy

The sensible path runs in sequence. Establish the community practice, the way Orange did. Then introduce read-only agents that inform and suggest. Only once those are established across the community should you build agents that execute changes. Direct action carries operational risk, so manage it carefully. It's also where significant wins often sit.

How you build depends on who's building. For teams that want to deploy quickly with minimal code, the Gemini Enterprise App provides a no-code environment for creating agents. For developers who need granular control, the Gemini Enterprise Agent Platform (formerly Vertex AI) offers advanced tools for launching and governing agents built with frameworks like the Agent Development Kit (ADK).

Cloud FinOps is moving beyond centralized reporting toward action that happens where the work does. The organizations getting there start with the culture, then use agents to carry it further than any one team could reach.

Orange's numbers came out of the community work. Building that foundation is the part worth copying first. When you're ready to extend it, Google Cloud Consulting can help you shape the community practice, and the Gemini Enterprise App is a low-lift way to put your first read-only agent in front of your teams.

How KDDI built Buffmee, a faster, reliable consumer RAG app

Tue, 08 Sep 2026 16:00:00 +0000

When building consumer-facing generative AI applications, balancing high generation quality with fast response times across diverse media types, can be challenging. KDDI, a major telecommunications carrier in Japan, tackled this challenge head-on when they developed Buffmee, their consumer Retrieval-Augmented Generation (RAG) app.

Buffmee is an interactive AI service built on the concept of 'AI that helps you grow.' By grounding responses in over 100 sources — including books, magazines, and web media — it helps users search for information, summarize key points, and explore personalized learning and hobby interests. By citing sources, Buffmee alleviates concerns about information reliability, allowing users to safely deepen their knowledge. To achieve this, KDDI collaborated closely with their development partner KDDI iret, Google Cloud Consulting and our specialized AI engineers.

As part of their app launch, the engineer team needed to ground a massive variety of proprietary content, including books and magazines. However, they struggled with latency issues that prevented them from meeting their target response times, and they needed a reliable way to ensure hallucination-free results.

Buffmee App Description and Images

To meet these performance targets, organizations need a systematic approach to AI evaluation and real-time bottleneck identification. That is why we are sharing the automated evaluation framework and performance optimization techniques that helped KDDI successfully launch their application.

The results were inspiring: KDDI reduced total application response latency by 38%, successfully hitting their target response performance. They also achieved a nearly 18% improvement in TTFT.

"Our vision hinged on a platform where content, once ingested, would instantly function as a working RAG system. Google's careful, hands-on guidance made that a reality — we're sincerely grateful for their support." — Shunya Onoda, AI Product Department, KDDI.

With these performance and accuracy improvements, Buffmee now empowers users to safely explore their favorite media through interactive Q&A and deep-dive analysis, delivering a highly personalized experience while maintaining strict trust and compliance for content providers.

Let’s deep dive into how they achieved these results.

Establish automated evaluation for diverse content

Traditional manual testing requires immense effort and cannot scale to accommodate a large content library. To solve this, the development team designed a systematic AI evaluation process using Gemini Enterprise Agent Platform Evaluation Service.

By implementing automated evaluation frameworks like LLM-as-a-Judge and the Rule of Hundreds, the team replaced labor-intensive manual testing with a data-driven process. They ingested their extensive document corpus, constructed hundreds of automated evaluation tests, and built a comprehensive benchmark dataset to measure the reliability of answers for each use case. As a result, the team improved their groundedness scores by 25%, helping deliver highly accurate and reliable outputs.

KDDI's automated evaluation loop: AI generates questions and scores answers, while humans calibrate thresholds and analyze edge-case failures.

Identify bottlenecks and optimize performance with an agentic loop

To improve response speeds, the team implemented BigQuery Agent Analytics and the Agent Development Kit (ADK) log analysis agent. By analyzing actual production logs, they visualized how skill division and prompt bloat—especially with highly complex, multi-page system prompts — impacted the Time To First Token (TTFT).

The team optimized the system prompt, including the inline integration of skills, and reviewed the sub-agent routing. This allowed them to identify and resolve deep-stack bottlenecks in real time without sacrificing response accuracy.

Four core principles for reliable evaluation

To achieve these results, the team implemented four core technical practices:

Transitioning to binary evaluation: By selectively moving away from ambiguous 1–5 ratings to a binary "pass (1) / fail (0)" system for critical metrics, the team minimized variance and noise, helping improve automation accuracy.
Strategic content sampling: Rather than attempting to evaluate every single document, the team classified their entire corpus along a two-dimensional grid: File Format (Web articles, EPUBs, PDFs, structured data) and Media Composition (Text-heavy, image-heavy, or mixed). By selecting representative samples from each cell of this difficulty grid, they reduced the evaluation workload by 75% while maintaining comprehensive test coverage.
Thresholds grounded in product judgment: Instead of relying solely on default tool parameters, the product owner reviewed randomly sampled answers alongside their automated scores to calibrate and establish what "good enough to ship" actually meant for the user experience.
Modular splitting of massive prompts into ADK Skills: Because massive system prompts exceeding 800 lines can cause LLM attention drift and latency degradation, the team split prompts by function into Agent Development Kit (ADK) Skills, dynamically loading only the required logic to optimize response times.

Get started

Building scalable, reliable generative AI applications requires both automated evaluation and deep performance analytics. To apply these techniques to your own applications:

Measure quality systematically with the Gen AI evaluation service
Structure your agents with Agent Development Kit and apply progressive disclosure deliberately
Ground your agents with Agent Search and inspect your retrieval queries

How Yahoo optimizes resources with flexible VMs in Managed Service for Apache Spark

Fri, 04 Sep 2026 16:00:00 +0000

As a global media and technology company connecting hundreds of millions of users to finance, sports, and entertainment platforms, Yahoo operates a massive data infrastructure where analytics workloads must run continuously at high speed. In deadline-driven data environments, relying on fixed virtual machine (VM) configurations creates a brittle system; if a specific machine shape faces a regional capacity constraint, cluster provisioning in Managed Service for Apache Spark (formerly Dataproc) can experience delays and stall critical data pipelines.

Yahoo utilizes flexible VMs in Managed Service for Apache Spark clusters to automatically absorb these resource fluctuations by defining a ranked list of acceptable VM shapes. This allows the system to dynamically search regional zones and maintain pipeline execution without manual intervention. To search for capacity across a region, teams must also enable Auto-Zone placement.

This optimization builds on Yahoo's broader data modernization journey, which involved migrating on-premises Hadoop and big data estates directly to Google Cloud. By transitioning those legacy workloads, the team established a cloud foundation capable of running high-scale batch and streaming analytics with dynamic resource flexibility.

This post provides a technical blueprint for configuring flexible VM instance rankings in Managed Service for Apache Spark to automatically manage capacity constraints and maintain pipeline execution.

Operational trade-offs of static configurations

Configuring clusters with a single, fixed machine type in a specific zone introduces constraints when regional zonal capacity fluctuations occur, potentially impacting cluster provisioning. Rather than manage these capacity variations through custom retry logic or manual intervention, using flexible configurations allows your infrastructure to automatically adapt. By accepting multiple VM shapes and searching across zones in the selected region, flexible configurations help streamline provisioning to better support high-scale analytics workloads.

Rules for configuring flexible clusters

Deploying flexible configurations requires aligning several connected design choices:

Enable auto-zone placement: You must pass a region(--region=${REGION}) or an empty zone string (--zone="") so Managed Spark can search for available capacity across the entire region.
Maintain core and memory symmetry: If your Managed Spark cluster uses autoscaling, all machine types in your flexible list must share a similar core count and memory size, even if they come from different VM families. A uniform CPU-to-memory ratio across primary and secondary workers prevents performance degradation, as the smallest ratio determines your effective container sizing.
Align component properties: Managed Spark calculates system properties based on VM cores and memory. When mixing machine shapes, you may need explicit property overrides to keep YARN and Spark resource allocations aligned with your expected worker behavior.

Two ways flexible VMs support massive workloads

For large-scale data environments, flexible configurations support operations in two ways:

Higher cluster creation success: Instead of failing when a preferred VM type is out of stock, Managed Spark selects from a ranked list to keep provisioning moving.
Better regional resource use: Auto-zone placement searches the entire region to find capacity, which reduces provisioning friction during high-demand periods.

gcloud example

code_block: <ListValue: [StructValue([('code', 'gcloud dataproc clusters create analytics-cluster \\\r\n --region=us-central1 \\\r\n --zone="" \\\r\n --num-workers=10 \\\r\n --master-instance-selection=\'{"machineTypes":["e2-standard-8"],"rank":0}\' \\\r\n --master-instance-selection=\'{"machineTypes":["n2-standard-8"],"rank":1}\' \\\r\n --worker-instance-selection=\'{"machineTypes":["e2-standard-8"],"rank":0}\' \\\r\n --worker-instance-selection=\'{"machineTypes":["n2-standard-8"],"rank":1}'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f88f00573d0>)])]>

API example

You can also build this capacity policy into your automated pipelines or Managed Service for Apache Airflow DAGS using the instanceFlexibilityPolicy field in the ‘Dataproc’ API:

code_block: <ListValue: [StructValue([('code', '{\r\n "projectId": "PROJECT_ID",\r\n "clusterName": "analytics-cluster",\r\n "config": {\r\n "gceClusterConfig": {\r\n "zoneUri": ""\r\n },\r\n "secondaryWorkerConfig": {\r\n "numInstances": 8,\r\n "instanceFlexibilityPolicy": {\r\n "instanceSelectionList": [\r\n {\r\n "machineTypes": ["n2-standard-8"],\r\n "rank": 0\r\n },\r\n {\r\n "machineTypes": ["e2-standard-8", "t2d-standard-8"],\r\n "rank": 1\r\n }\r\n ]\r\n }\r\n }\r\n }\r\n}'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f88e3f0fa90>)])]>

This API policy achieves the same goal: it establishes your preferred shape, documents valid fallbacks, and lets Managed Spark resolve resource constraints without breaking your automation scripts.

Establishing an infrastructure policy

Managing data at this scale requires standardizing a clear resource policy rather than relying on a single rigid machine type. Your configuration standards should outline:

Preferred and fallback VM families for secondary workers.
Default auto-zone placement to enable flexible provisioning.
Identical core and memory configurations when using autoscaling.
Uniform CPU-to-memory ratios across all worker groups to maintain predictable container sizing.
Explicit YARN or Spark property overrides to guarantee consistent runtime behavior across different machine lines.
Shuffle-safe patterns for Spark workloads running on Spot or highly elastic capacity.

By adopting flexible configurations, you turn infrastructure scarcity into a predictable fallback plan, keeping your critical data pipelines up and running.

Yahoo impact and results

By implementing flexible VMs in Managed Service for Apache Spark, Yahoo successfully reduced cluster provisioning failures by 85% which were caused by regional capacity stockouts. This flexible configuration allows their data infrastructure to automatically handle capacity constraints and successfully provision resources without requiring manual intervention. As a result, Yahoo ensures continuous workload execution and prevents downstream processing delays across their massive data pipelines.

"Managing high-scale data analytics at Yahoo requires resilient, automated infrastructure. Moving to flexible VMs in Managed Service for Apache Spark has transformed our approach; instead of stalling when a specific machine shape faces capacity constraints, our clusters now automatically pivot to our ranked fallback options. This has helped us reduce provisioning failures by 85%, providing the reliability we need to keep our global media platforms running smoothly." - Akshay Jain, Senior Software Developer Engineer, Yahoo!

Strategic benefits of flexible infrastructure

Adopting a flexible compute stack transforms your environment into a dynamic pool of resources that adapts to your operational needs. By moving away from rigid, single-machine type configurations, you ensure that your workloads reliably access the compute they need, regardless of supply fluctuations. This shift not only maximizes workload obtainability and reliability but also facilitates seamless hardware modernization by allowing you to prioritize newer VM generations while maintaining older types as reliable fallback options.

Build your resilient data pipeline

Transitioning to a fluid compute strategy ensures your critical analytics remain operational despite regional resource shifts. Here is how you can begin optimizing your infrastructure today:

Audit your workloads: Identify applications tightly coupled to specific VM families or zones and map out viable alternative hardware shapes.
Standardize resource policies: Explore the documentation for Managed Spark flexible VMs to establish your preferred and fallback VM families.
Align financial strategy: Utilize Flexible Committed Use Discounts (Flex CUDs) to maintain cost predictability when workloads dynamically pivot to alternative machine types.
Claim your credits: New customers may be eligible for $300 in credits to try Managed Service for Apache Spark and other Google Cloud products at no cost.

How BlackLine simplifies perimeter policy intelligence with VPC Service Controls

Tue, 01 Sep 2026 16:00:00 +0000

Establishing network-level perimeters with VPC Service Controls (VPC-SC) is a critical step that can help you protect your cloud environment against data exfiltration, compromised accounts, and insider threats.

Today, Google Cloud is excited to share new policy intelligence capabilities in VPC-SC that can help drive even greater operational simplicity. With our latest release of the VPC-SC violation analyzer and violation dashboard, we have simplified policy management and troubleshooting, to make managing and optimizing your security perimeter more efficient and straightforward than ever.

How BlackLine streamlines incident response

BlackLine, a leader in financial operations management, adopted the VPC-SC policy intelligence solution to maintain strict security perimeters. Chosen by over half of Fortune 500 companies, BlackLine uses Google Cloud's full suite of managed services and built-in security capabilities to protect sensitive customer financial data.

VPC Service Controls are the foundation of BlackLine's preventative compliance and security controls in our Google Cloud environment, helping us to mitigate data exfiltration risks and ensure clear separation between our higher and lower environments by establishing strong security perimeters.

Managing these complex perimeters is a continuous process. VPC Service Controls violation analyzer helps BlackLine cloud infrastructure administrators adapt to changing API connection requirements of the business by adjusting security perimeters through approved access levels, ingress policies, and egress policies.

With only the troubleshooting token or unique ID from any VPC-SC violation error message, we can produce a detailed report identifying the principals and target resources involved in a failed API request, and explaining why and how that API request violated BlackLine's service perimeters. We don’t need to write a Cloud Logging SQL query to extract the data.

The clear access context and actionable insights in the violation details report are an invaluable starting point as we collaborate to resolve violations, significantly reducing our mean-time-to-resolution (MTTR) for service perimeter issues, and helping BlackLine maintain our focus on our customers and continue to innovate on their behalf.

Streamlining the perimeter operations lifecycle

Our new policy intelligence tools — the VPC-SC Violation analyzer and Violation dashboard — simplify real-time monitoring and active incident response. These tools provide clear, actionable insights in the Google Cloud Console, offering greater speed and automation to help you confidently enforce least-privilege perimeters, and quickly resolve access denials.

Violation Dashboard aggregates and visualizes all service perimeter violations across your entire Google Cloud organization in a single pane of glass, helping your team identify trends, spot spikes in access denials, and shareable filters on violations by specific perimeters, projects, or identities.

Violation Analyzer streamlines investigating violations, eliminating the need to query Cloud Logging and manually piece together the details. When you click a troubleshooting token from the dashboard (or input a unique denial ID), the analyzer maps out the identity, source, target, and VPC-SC rule triggered, creating a report telling you why that specific request was blocked. This helps your team more quickly take action to determine whether to modify existing policy rules or create a new one, and resolve incidents more quickly.

Together, the new VPC Service Controls policy intelligence tools go beyond automated log analysis to provide unified visibility of violations and actionable insights to investigate them, making your perimeter deployment and management simpler and lower-risk.

Streamlining the VPC Service Controls lifecycle, from deployment to policy refinement.

With the new VPC-SC troubleshooting tools you can more easily:

Test new perimeters (deployment): Use the violation dashboard to visualize the impact of a service perimeter during your initial dry run phase, helping to verify that enforcement is accurate and predictable before it affects production traffic. Filter violations to track and resolve with prebuilt contextual filters for principals, service perimeters, enforcement type, and more.
Track perimeter denials (monitor): The violation dashboard offers a unified view of your perimeter health, allowing your security operations team to monitor status in real time, including dynamic agentic access denials.
Triage an event (investigate): Violation analyzer provides the identity, source, target, and operations for any violation. It cross-references identity and access management (IAM) permissions, resource ancestry, and context evaluation to identify which rule was triggered, reducing manual effort.
Fix the rule (refine policy): Instead of searching through configuration files, violation analyzer maps violations directly to the relevant line in your VPC-SC policy, allowing you to make updates more quickly and with less manual overhead.

The VPC Service Controls violation dashboard produces detailed reports to jump-start perimeter access investigations that are simplified using the violation analyzer.

Core VPC-SC operations: Simple perimeter enforcement

Our new troubleshooting capabilities build on VPC Service Controls’ foundational simplicity for designing, enforcing, and managing strong perimeters.

By using dry run mode, your teams can build precise, contextual ingress and egress rules based on observed traffic — without disrupting vital business workflows. Once you validate these access patterns, moving to full enforcement becomes a more confident, data-driven process. To keep perimeter maintenance more efficient and straightforward, scoped policies allow you to delegate management directly to project-level administrators, empowering the teams closest to the workload.

Getting started

Simplify data security with VPC Service Controls. With the new Violation Analyzer and Violation dashboard, you can spend less time investigating incidents and more time safely scaling your cloud initiatives. Your data is your most valuable asset — protect it with a perimeter that’s as simple to manage as it is effective in enforcing controls.

Learn more and get started with the VPC-SC violation analyzer and violation dashboard in our documentation.

Reimagining work: How Pythian’s internal AI playbook delivers customer ROI

Thu, 27 Aug 2026 16:00:00 +0000

When Pythian rolled out Google Cloud’s Gemini Enterprise across our 500-person company in 27 countries, the goal was simple: use our own company as a proving ground to discover how enterprise AI actually delivers ROI.

What we found changed our strategy entirely.

Since the rollout of Gemini Enterprise and our previous enterprise AI deployments, Pythian observed firsthand why so many enterprise AI initiatives stall out or fail.

Most organizations trap themselves in a tool-centric mindset — buying licenses, making tools broadly available, and assuming value will naturally follow. They get stuck chasing "nickel and dime" micro-efficiencies (like saving 5 minutes per user) while missing structural, high-ROI workflow transformations. Compounding the problem, even when custom agents are built, they frequently stall in pilot mode or break down in production because teams lack the operational capability to manage AI model drift, agent lifecycles, and ongoing observability.

To solve this, we engineered the Pythian AI Operating Model — a multifaceted, end-to-end framework designed to take enterprise AI from high-level strategy all the way into sustained production. While our dual center of excellence (COE) serves as the core execution muscle, it is the application of the entire framework, from Field CTO strategy and tooling deployment to the dual COE and XOps, that consistently unlocks million-dollar outcomes.

By proving this complete model internally first, Pythian drove a 3x surge in active user engagement and cut our database incident resolution times by 80%.

The four pillars of the Pythian AI operating model

To move past the common failure points of enterprise AI, our framework consolidates strategy, execution, and operations into a single continuous loop:

Field CTO strategy ──> tooling deployment ──> dual COE execution ──> production XOps

Field CTO strategy and governance: Generative AI is arguably the most academically challenging architectural shift in IT history. Led by former C-suite tech leaders, our Field CTO practice provides executive advisory to establish steering committees and clear value metrics. The team audits operations using 16 horizontal agentic patterns (like automated document processing and runbook creation) to build a prioritized backlog of high-ROI use cases before development starts.
Tooling and platform deployment: The team establishes a secure, production-grade foundation on platforms like Gemini Enterprise and connects AI directly into CRMs, ERPs, and database estates to ground models in real corporate context.
The dualCOE: This execution muscle is split into two specialized engines:

People productivity COE: This group handles adoption and change management. Instead of expecting non-technical teams (like HR or Procurement) to build its own agents, this COE builds no-code agents for them, focusing entirely on enablement.
Process productivity COE: This team engineers deep, custom-coded AI agents and complex agentic workflows that integrate into core data platforms for autonomous operations.

XOps (AI production management): While deploying an agent is 20% of the journey, maintaining accuracy in production is 80%. Because AI models and prompt structures naturally drift over time, this XOps practice provides the continuous monitoring, prompt tuning, and model observability needed to keep agents performing without breaking core workflows.

The difference between chasing minor, scattered efficiencies and driving structural enterprise ROI comes down to how you align your operating strategy:

Alignment element	Tool-centric approach	Pythian AI operating model
Primary metric	Individual minutes saved per user	High-impact workflow reimagination and ROI
Operational focus	Broad, unguided tool availability	Prioritized backlog via 16 agentic patterns
Execution muscle	Ad-hoc user experimentation	Dual COE (people and process productivity)
Production lifecycle	Unmonitored static deployments	Active XOps (Continuous accuracy and drift management)

Real-world impact: from database ops to global supply chains

Whether managing 70 manufacturing plants or 30,000 enterprise databases, AI succeeds when tied to structural, high-value workflows:

Pythian “as a customer:” Across 15,000 monthly database tickets, our Process COE deployed an agentic workflow that reads tickets, searches knowledge bases, and auto-generates mini runbooks before an engineer touches them. The result was slashed mean time to resolution by 80% and tripled active user engagement.
Knowledge management customer: We deployed autonomous IT support agents across 10,000 consultants. As a result, we were able to automate 10% of 20,000 annual IT tickets into "no-touch" resolutions, saving 1,000,000+ operational hours.
Supply chain customer: By building custom agentic supply chain tools on Gemini Enterprise, we compressed forecast-matching cycles from weeks down to 2–3 days across 70 global manufacturing sites.
Retail customer: We combined Gemini Agentic AI and computer vision to automate store product onboarding. As a result, we transformed a 20-minute manual task into a multi-second flow.

Ready to build your AI operating model?

Scaling AI demands more than tool-level experimentation. It also requires an end-to-end AI operating model. Learn how Pythian pairs with Google Cloud to operationalize strategy, streamline XOps, and fast-track your Gemini Enterprise journey.

How Uber improves network reliability while unblocking cloud migration

Wed, 26 Aug 2026 16:00:00 +0000

Uber has a lot in common with the cities it serves. Both are always changing and growing, both must carefully manage the resulting traffic to prevent congestion and sprawl.

Uber has continuously evolved its technical strategies to manage its expanding network, and this careful planning and constant evolution helps ensure that application traffic across its entire platform runs smoothly. Ultimately, maintaining a reliable, high-scale platform that operates seamlessly at any given time is key to preserving user trust.

One important solution in this effort has been application awareness on Cloud Interconnect. An industry-first tool for application prioritization across hybrid networks, application awareness on Cloud Interconnect has helped Uber prioritize critical traffic to ensure business continuity during potential network congestion events.

Uber acted as an early design partner for application awareness on Cloud Interconnect, helping ensure that this capability met the demands of Uber’s global-scale operations. It not only improved Uber’s daily operations, it also gave Uber the confidence to move forward with a Google Cloud migration, with confidence that there would be less risk of service interruptions during switchovers.

In this post, we’ll explain the features Uber most sought and why, the inner workings of application awareness on Cloud Interconnect, and how it can help other organizations as well.

Prioritizing critical traffic

When migrating distributed, hybrid, or multicloud applications at a global scale, network reliability becomes a primary concern. Even the most worthwhile migrations may not seem worth it if such migrations interrupt ongoing service. For organizations like Uber, moving vast amounts of data to support large data analytics workload — including emerging AI use cases — can saturate network links, resulting in increased reliability risk for their critical application traffic.

With standard cloud interconnect approaches, enterprises typically apply simple bandwidth overprovisioning to meet extreme infrastructure needs. But with today's hybrid cloud demands, and given the size of an organization like Uber, overprovisioning network capacity for peak usage is often too costly and unreliable.

The shortcomings of overprovisioning only become magnified with the integration of cutting-edge AI innovations. Uber needs systems in place that can take on massive data transfers without congesting its network and protecting the performance of business-critical applications.

With the benefit of application awareness on Cloud Interconnect, including the four major features of application awareness — traffic handling, congestion response, latency management, and cost efficiency — Uber was able to achieve the networking optimization its modern tech stack requires.

Starting with a private preview, Uber deployed this feature across its infrastructure, beginning with Google Cloud Interconnect deployments in Phoenix, Arizona, and Ashburn, Virginia. Application awareness on Cloud Interconnect allows Uber to classify and prioritize end-user application traffic over less time-sensitive data using DSCP marking and configured queuing profiles.

In the following chart, we look at the four key features of application awareness on Cloud Interconnect, how they differ from legacy approaches, and how they help provide better operational continuity for organizations like Uber.

Feature	Standard interconnect solutions	Application awareness on Cloud Interconnect
Traffic handling	All traffic treated equally (first-in, first-out)	Traffic classified into six distinct traffic classes
Congestion response	High-priority application traffic may be dropped during bursts	Business-critical traffic is protected via strict priority or bandwidth sharing policies
Latency management	Unpredictable latency for high priority applications	Predictable and consistent low-latency for time-sensitive workloads
Cost efficiency	Requires expensive overprovisioning to absorb peaks	Efficient bandwidth utilization and lower TCO

Uber's key takeaways

For Uber, the business value of being able to prioritize business-critical traffic on its networks by deploying application awareness on Cloud Interconnect was immediate. And in doing so, Uber has also created a blueprint that other enterprises with similar hybrid cloud challenges can replicate. The core elements of that blueprint include:

Ensuring business continuity: Uber can decide in real time which application traffic to prioritize during major, high-traffic events. This means that mission critical applications stay up and running during even extreme events (both planned and unplanned). Uber leadership has called application awareness on Cloud Interconnect important for its global operations.
Efficient bandwidth utilization: Instead of blindly overprovisioning bandwidth to prevent congestion, application awareness allows Uber to better utilize their existing Cloud Interconnect capacity aligned with their expected network bandwidth needs. The result is lower total cost of ownership for network infrastructure.
Unblocked workload migration: By protecting critical applications from network congestion, Uber was able to migrate significant workloads to Google Cloud and, in the process, dramatically reduce operational overhead.

"Application awareness on Cloud Interconnect was the key that unlocked our ability to migrate more strategic workloads to Google Cloud and is critical for maintaining service reliability during peak global demand. By allowing us to intelligently prioritize traffic, it helps us ensure that we can protect our higher priority services and make our infrastructure more efficient, lowering our total cost of ownership. This wasn't just a feature deployment; it was a deep engineering partnership that delivered a solution critical to our business." – Harry Liu, Director of Engineering, Uber

Securing network reliability for AI and beyond

As more enterprises integrate cloud-based AI models, distributed applications, and data analytics, it's becoming a business imperative to be ready to handle the massive data transfers that follow. But in doing so, they also have to ensure they never compromise the reliability of their critical applications.

With application awareness on Cloud Interconnect, Uber demonstrated that moving beyond simple bandwidth overprovisioning to protect business-critical traffic was an essential step to building the stability required to embrace modern hybrid and multicloud strategies.

You can read our blog about the potential of Cloud Interconnect across industries to learn more about what the service can bring to your organization, and if you’re ready to explore more, our team of networking and industry experts are ready to help.

How Box is unlocking multimodal enterprise agents with Gemini Embeddings 2

Tue, 18 Aug 2026 16:00:00 +0000

Enterprise content management is experiencing its biggest architectural shift since the cloud migration era.

For years, enterprises have stored trillions of gigabytes of critical data in Box: financial models, clinical trial protocols, M&A due diligence rooms, engineering schematics, and legal compliance playbooks. Up to this point, text-based search and retrieval-augmented generation (RAG) have successfully unlocked the vast narrative knowledge within these repositories, establishing a powerful and highly effective baseline for enterprise AI intelligence.

Traditional RAG architectures have mastered text processing, but the agentic era demands more. The next logical evolution is to extend this framework to capture the inherently multimodal, deeply spatial, and highly structured elements that exist alongside text. While text embeddings excel at indexing prose, multimodal architectures unlock a major new capability: For example, they preserve the strict row-column semantics of financial tables, interpret visual evidence like clinical data, and map the logic of multi-page flowcharts without losing their spatial layout.

To deliver next-generation capabilities that can handle the vast universe of digital content, Google Cloud and Box are integrating advanced multimodal capabilities into Box's Agentic Platform, powered by Gemini Multimodal Embeddings 2 merging Box’s industry-leading Intelligent Content Management platform with Google Cloud’s advanced AI embeddings.

Benefits of improved embedding: Extending the dimensions of document content

Preserving visual and spatial geometry: Complex document elements like multi-column tables or financial matrices rely on their spatial layout to convey meaning. Converting these elements into a flat string of text can disassociate column headers from their corresponding data points. Multimodal embeddings allow systems to interpret the document exactly as a human does, maintaining the integrity of spatial relationships.
Illuminating the visual modality: Enterprise documents are filled with visual indicators: technical charts, process flowcharts, branding assets, and product photography. Multimodal capabilities ensure that these elements are no longer invisible to search systems, allowing users to query images and text simultaneously.
Connecting hybrid file formats: Real-world business workflows rarely live in a single document format. An agent may need to cross-reference a PDF policy, a spreadsheet tracking log, and a presentation deck. Extending RAG with multimodal embeddings creates a unified understanding across these varied formats.

The Architectural Solution: Gemini Multimodal Embeddings 2

Google Cloud’s Gemini Multimodal Embeddings 2 introduces a unified, multimodal vector space capable of embedding text, raster images, document pages, rendered spreadsheet tables, and visual charts into the same semantic representation space.

Key product capabilities unlocked by gemini-embeddings-2:

Crossmodal retrieval (text-to-visual / visual-to-text): Enables natural language queries to retrieve highly specific visual components, such as locating a target chart or diagram within a massive library of slides, without requiring manual tagging.
Layout-aware document embedding: Rather than breaking files into arbitrary text blocks, the system can embed document page renderings directly, preserving visual hierarchies, callout boxes, and structural context.
Heterogeneous format bridging: Native support for seamlessly bridging content across .docx, .xlsx, .pdf, .pptx, .png, and .csv without losing modality-specific structural information.

Three core patterns of multimodal enterprise agents

By leveraging multimodal embeddings within Box, we have identified three uniqueprimary design patterns that illustrate how organizations can extend traditional RAG to support complex, visual workflows.

Pattern 1: Complex financial & analytical reporting

The challenge

Corporate finance, research, and audit teams analyze highly structured documents where vital data resides in embedded tables, growth charts, and footnote annotations. Text-only indexing can separate these numbers from their context, making automated analysis challenging.

The multimodal advantage

Structural alignment: The embedding model captures the physical structure of tables and charts, allowing financial agents to understand that a column header applies to a specific row of metrics.
Visual trend analysis: Agents can cross-reference written summaries with visual trends in accompanying bar or line charts, identifying and pointing out discrepancies between written claims and source data.
Contextual sourcing: Users can query complex portfolios and instantly retrieve the exact page, table, or chart supporting a specific metric.

Pattern 2: Multimodal clinical decision support & assisted diagnosis

The challenge

In healthcare and clinical environments, critical patient data is fragmented across vastly different, unstructured visual and textual formats — ranging from external physical photos (visual evidence) and microscopic pathology slides (lab reports) to structured risk matrices (triage grids). Traditional text-based systems or isolated analysis tools cannot synthesize these cross-modal relationships simultaneously, which can delay critical diagnoses or risk missing immediate, life-threatening procedural complications.

The multimodal advantage

Cross-modal clinical synthesis: Evaluates physical symptoms alongside cellular-level laboratory evidence simultaneously by indexing clinical photos, histopathology imagery, and triage grids into a single space.
Granular anomaly identification: Connects niche visual patterns under a microscope (like parasitic cyst walls) with medical knowledge to rapidly isolate rare conditions.
Risk-aware decision support: Cross-references findings against triage frameworks to deliver instant warnings about immediate patient risks, such as life-threatening anaphylactic shock.

Pattern 3: Cross-document multimodal synthesis & data reconciliation

The challenge

Enterprise information is fragmented across disconnected files and formats (e.g., PDF minutes, Excel charts, PNG flyers, and email threads). Traditional tools analyze these files in isolation, failing to connect the dots when verifying details or resolving data contradictions across independent documents.

The multimodal advantage

Cross-file synthesis: Connects information across entirely different formats (PDFs, spreadsheets, images, emails) simultaneously to answer complex business queries.
Conflict resolution: Flags and resolves contradictions between assets, such as catching outdated pricing on an image by cross-checking it against the latest financial spreadsheets.
Visual-to-text auditing: Audits visual or scanned files against text-based records (e.g., verifying a signed PDF contract against a legal review email) to catch missing clauses or changes.

The future of agentic enterprise content management

The integration of gemini-embeddings-2 into Box’s Agentic Platform is an important new capability to improve the next era of content intelligence. Multimodal embeddings help Box to move beyond basic search to active, intelligent collaboration.Box's Intelligent Content Management platform represents a fundamental shift in enterprise AI infrastructure — moving beyond passive document storage to deliver a governed, semantically indexed reasoning layer where AI agents can interrogate, cross-reference, and act on content with full compliance and security controls already in place.

Powered by multimodal embeddings and a suite of native AI agents spanning search, metadata extraction, research, analysis, and composition, Box enables organizations to proactively surface insights such as flagging stale pricing data, expiring contract clauses, or cross-document contradictions before they become business risks. For high-complexity industries like financial services, life sciences, and legal operations, Box's ability to reason across text, tables, charts, and images makes multimodal understanding a competitive requirement.

Designed to interoperate with the broader enterprise AI ecosystem, Box serves as the single governed content foundation that ensures every AI-driven workflow is grounded in authorized, auditable enterprise data.

When you think about it, the enterprise data landscape was always multimodal. Now we have the technology to make the most of it. By integrating gemini-embeddings-2, Box helps its users unlock unprecedented value from unstructured enterprise content. Product leaders who embrace multimodal-first architectures, rigorous precision benchmarking, and audit-ready grounding will lead the next wave of enterprise productivity and innovation.

The team would like to thank Ken Ikeda, Afshaan Mazagonwalla, and Samip Thakkar for their work on this project.

Building operational resilience with agentic AI in financial services

Tue, 18 Aug 2026 14:00:00 +0000

For financial institutions, operational resilience has long been embedded in regulatory and supervisory expectations — to say nothing of the high expectations of consumers. With the implementation of the European Union’s Digital Operational Resiliency Act (DORA), those expectations have become even more stringent, with more explicit, harmonized, and evidence-driven requirements. Firms must now demonstrate that their critical business services and supporting digital infrastructures can withstand disruption, support coordinated response, and recover with control.

To meet these conditions, Deutsche Bank developed an AI-powered agentic resilience platform that modernized its regulatory tabletop resilience exercises at scale and turned manual preparation into context-aware and evidence-ready simulations grounded in actual operational data. The platform builds enterprise context from architecture, data flows, logs, incident history, alerting signals, and operational telemetry to generate scenarios, simulated operational evidence, structured session records, and regulator-ready artifacts.

At many large banks with operations that span interdependent applications, data flows, and third-party services, this is a critical and even existential shift. Across financial services, supervisory expectations are evolving and as they do, banks’ tabletop exercises must reflect their production dependencies, real operating conditions, and compliance with consistent evidence standards more directly.

As Deutsche Bank considered how to successfully and efficiently make this shift at scale, it looked to its long-time partner, Google Cloud, and its growing suite of agentic AI tools.

From tabletop exercises to resilience intelligence

With its agentic resilience platform, DB has been able to transform its tabletop exercises from manual preparation to a continuous intelligence model. And it’s been able to extend the same agentic layer to root-cause analysis when real operational context is needed.

This means that every scenario it runs is based on real enterprise signals. The platform can then reflect true system dependencies, failure patterns, and business impact instead of relying on static inputs that are more likely to return assumptions than real-time insights.

By using Gemini Enterprise Agent Platform, DB has been able to migrate this operational context into structured scenarios with clear timelines, decision points, and expected responses. This has ensured that each exercise is grounded in real system behavior that produces consistent, audit-ready evidence that meets regulatory expectations.

Dual orchestration for control and flexibility

In order to deliver both regulator-grade control and operational flexibility, DB’s platform introduced a dual-orchestration architecture that separates workflows into two complementary execution models.

First, for regulator-aligned execution, the bank is using LangGraph to ensure that it generates every scenario through a traceable, deterministic process — with clear lineage from input context to output — that supports the auditability required for supervisory review.

Next, for its adaptive and investigative scenarios, DB is using Google Agent Development Kit (ADK) to enable agent-driven coordination. This approach allows the bank’s platform to dynamically analyze conditions and generate responses without predefined execution paths.

Figure 1. Architecture for context assembly, orchestration, and scenario generation.

With this architectural separation, the platform can combine governed execution with adaptive investigation while preserving a common intelligence layer. The same agents and tools can reason over architecture, data-flow diagrams, logs, and code artifacts across tabletop scenario generation and related incident-analysis workflows. Importantly, this supports a consistent resilience model across both planned exercises and real operational events.

Deutsche Bank’s objective with this platform was to engineer a resilience model for critical financial systems that meets regulatory expectations — even within highly complex, distributed environments. By linking dynamically generated scenarios to real business context and combining governed orchestration with adaptive analysis, the platform has given us an intelligent, continuously adaptive model for operational resilience.” – Sanjay Tripathi, Managing Director, Global Head of Surveillance Technology & Compliance Cloud & AI Transformation Lead, Deutsche Bank

Powering generation and governance with Google Cloud

Google Cloud’s suite of agentic tools is providing the foundation for scaling Deutsche Bank’s platform across its many governed, enterprise-grade resilience workflows. Here’s how:

Cloud Run supports elastic execution of scenario and evidence-generation services.
Gemini Enterprise Agent Platform transforms operational context into structured resilience scenarios.
Google ADK enables adaptive agent coordination.
Cloud SQL provides durable persistence for scenarios, session artifacts, and review records.

Collectively, these services give DB support for the traceable generation, controlled execution, and persistent evidence record required for compliance review and continuous improvement.

Scalable, evidence-ready resilience testing

Every scenario generated by Deutsche Bank’s platform drives a structured tabletop session for the teams that run response, escalation, and recovery. Because these exercises are grounded in real enterprise context, they reflect operational reality while also strengthening consistency across teams and creating audit-ready evidence that meets regulatory expectations. For institutions that operate under DORA or similar frameworks, this makes it easier to demonstrate controlled, coordinated, and disciplined response at scale.

This model is now being applied across multiple DB portfolios, which is helping the bank establish more consistent and scalable resilience paradigms and a replicable blueprint for the broader financial sector.

In this model, root-cause analysis acts as the feedback loop between real incidents and future resilience testing. The resulting insights from production events can inform future tabletop scenarios, while exercise outcomes can strengthen response playbooks, escalation paths, and recovery readiness.

All of this extends the platform’s value from planned resilience exercises to real operational events while keeping scenario-based resilience testing as the primary use case.

As adoption expands, this platform brings consistency by embedding Google Cloud’s methodology for context-aware resilience. It eliminates fragmented manual approaches and establishes a cross-functional, AI-informed operating model across the bank.

Toward resilience intelligence

The bank’s next step is to extend this approach into a broader resilience intelligence layer, which is possible because it can deploy the same patterns to support playbook refinement, recovery-readiness assessments, and continuous validation of controls against evolving system conditions.

For financial institutions, this is a strategic shift. As systems become more distributed and regulatory expectations more demanding, banks must move from periodic resilience testing to continuous, intelligence-driven capabilities. At Deutsche Bank, Google Cloud is making that transition simple across the organization.

Learn more about Google Cloud’s methodology for context-aware resilience in this article.