Networking

When the Kubernetes API went silent: A three-layer networking mystery

Fri, 30 Oct 2026 16:00:00 +0000

Every support engineer has a case that stays with them. The one that comes up at team lunches. The one where, even after it's resolved, you find yourself staring at a packet capture at midnight just to make sure you really understand what happened. This is that case.

It started in January 2026 with a support ticket that read deceptively simply: pods are intermittently failing to reach the Kubernetes API server. The error was a familiar-looking timeout. The customer had already ruled out their own operators. What none of us knew yet was that we were looking at three separate bugs — in three different layers of the stack — all conspiring to produce the same symptom at the same time.

By the time it was over, around 800 developers had been blocked from deploying to production for weeks on end, and GKE clusters across europe-west had ground to a halt. In a race to fix the problem, we involved engineers across Google Kubernetes Engine (GKE), Private Service Connect, networking, hostnet, kernel, and the Linux upstream community, who together tracked down not one but three distinct root causes. And the final fix landed not as a config change or a feature flag, but as a patch to the Linux kernel itself.

This is the story of how we got there.

Part one: The alarm bell that wouldn't stop

The customer ran a large multi-cluster GKE environment in Google Cloud's europe-west region. Their clusters used Private Service Connect — Google Cloud's mechanism for exposing the GKE control plane privately — and had a rich ecosystem of system-level workloads: Twistlock security scanners, External Secrets Operator for secrets management, Cilium for network policy, as well as several internal Kubernetes operators.

The first errors appeared in one of its user acceptance testing (UAT) clusters in europe-west, and a few days later in a second cluster in the same region. They looked like this:

Application logs — pods across multiple namespaces

code_block: <ListValue: [StructValue([('code', "TimeoutError: Connect call failed ('1.2.3.4', 443)\r\ndial tcp 1.2.3.4:443: i/o timeout\r\n\r\n# 1.2.3.4 is the cluster's internal VIP for the kube-apiserver,\r\n# routed through Private Service Connect"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbd615810>)])]>

Initially the errors were intermittent — spikes in the logs, not a hard outage. The customer suspected their own code. Their team had recently deployed a custom operator built with the kopf framework, and it was the first thing they looked at. But the logs showed the errors had started eight days before the operator was ever deployed. When they shut it down completely, the timeouts kept coming.

The issue persists even after completely scaling down the External Secrets Operator. The errors are appearing from customer workloads, and system-level pods we haven't touched.

The growing list of suspects

When a case like this arrives, the first instinct is to ask: what changed? The customer's timeline gave us several candidates, and we worked through them methodically.

The custom operator (kopf): Ruled out immediately — errors predated deployment by a week and continued after shutdown.
External Secrets Operator: Removed entirely on customer request. Timeouts continued in customer workloads that relied on Kubernetes control plane (KCP) connectivity, and cluster-internal system pods.
Security agents: Uninstalled. Same result. We were now watching system-level Kubernetes components fail to reach their own control plane.
conntrack table exhaustion: One theory was that too many half-open connections were filling the OS connection tracking table on the control plane nodes. We checked. The conntrack table was at roughly 50% capacity — of a default maximum of 1.3M entries. Not even close to exhaustion.
Control plane overload: We scaled up the KCP VM size. kube-apiserver latencies looked healthy. No improvement on the timeout side.

What made this case genuinely strange was the scope. We confirmed the timeouts were happening across five different clusters, affecting DaemonSets, Deployments, and system pods alike. External clients — developer laptops, GitHub Actions runners — were also hitting the same wall. This was not a broken workload. This was something wrong with the path between everything and the API server.

The timeline so far

Day 1: First I/O timeout errors appear (One of the UAT clusters in europe-west3. Intermittent, easy to dismiss.)
Day 10: Errors become persistent (Frequency increases significantly. The issue is no longer dismissible.)
Day 15: Case escalated — P0 bug filed (Multiple clusters confirmed affected. ~800 developers blocked. Production releases are frozen. Formal escalation begins.)
Day 20: All obvious suspects ruled out (Custom operator, security agents, conntrack exhaustion, control plane resizing — all cleared.)
Day 22: Packet captures collected (Captures taken simultaneously at the control-plane node NIC and from inside the kube-api container. The investigation enters its most critical phase.)

With the obvious suspects cleared, and the scope undeniably wide, we did what you do when the application layer has nothing left to say: We went to the packets.

Part two: Tracing the ghost in the wire

To understand what the packet captures revealed, you first need to understand the architecture that a GKE pod traverses when it makes an API call. In a Private Service Connect-based GKE cluster, the path is deceptively long for something that looks like a local connection.

The key component here is Private Service Connect. When a pod dials 1.2.3.4:443, it is not actually going directly to the control plane; that IP is a virtual endpoint. The connection is intercepted by Google's networking fabric, NAT'd through a Private Service Connect endpoint, passed through an internal passthrough Network Load Balancer, and ultimately delivered to the KCP VM, where Envoy acts as a local proxy in front of the actual kube-apiserver process.

Private Service Connect adds a proxy protocol header to each new connection, so the kube-apiserver can see the real client IP. This will turn out to be important later.

What the packets revealed

When we captured simultaneously at the KCP VM's NIC (the PIGv interface, Google's programmable virtual NIC) and from inside the kube-apiserver container, we saw something striking.

SYN packets from the pods were arriving at the KCP VM's NIC. They were getting that far. But instead of receiving a SYN-ACK in response — the expected second step of a TCP handshake — the client was getting back an out-of-place ACK. Not an RST. Not silence. An ACK that logically belonged to a different, earlier connection.

Wireshark, the network protocol analyzer, was flagging these streams with a label that initially seemed innocuous: "TCP Port numbers reused."

Wireshark — packet stream showing the failed handshake

code_block: <ListValue: [StructValue([('code', 'Time Source Port → Destination Port Info\r\n09:13:41 10.0.13.77 53426 → 10.117.0.2 443 [SYN] Seq=0 Win=42600\r\n09:13:41 10.117.0.2 443 → 10.0.13.77 53426 [SYN, ACK] Seq=0 Ack=1 Win=64768\r\n ... (data exchange, connection completes)\r\n09:13:42 10.117.0.2 443 → 10.0.13.77 53426 [TCP Retransmission]\r\n09:13:42 10.117.0.2 443 → 10.0.13.77 53426 [TCP Retransmission]\r\n ... (many retransmissions of the same ACK)\r\n\r\n# 2 minutes later, same port 53426 is reused for a new connection\r\n09:13:44 10.0.13.77 53426 → 10.117.0.2 443 [SYN] Seq=1498 ← NEW connection\r\n (expected: SYN-ACK)\r\n (received: ACK Seq=1498 Ack=1025 — belongs to PREVIOUS stream)\r\n09:13:44 10.0.13.77 53426 → 10.117.0.2 443 [SYN] Retransmit...\r\n09:13:44 10.0.13.77 53426 → 10.117.0.2 443 [SYN] Retransmit...\r\n → eventually: i/o timeout'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbd617250>)])]>

The pattern was clear. Something was causing the KCP VM to respond to a fresh SYN with an ACK that belonged to a different connection that had ended minutes ago. The old connection had used the same source port (53426 in the example above), and the KCP VM had never fully let go of it.

"The KCP server sends FIN,ACK; client responds with FIN,ACK and then ACK. But KCP receives ACK before FIN,ACK. KCP server never ACKs the FIN,ACK — continuous retransmission of ACK. RST does not clear state."

The "Invalid ACK" spike

While one thread of the investigation was analyzing packet captures, a.k.a., pcaps, another team was digging into internal metrics for the Private Service Connect infrastructure — specifically the user space packet switch (USPS), Google's software switching layer that handles NAT and proxy protocol header injection for Private Service Connect connections.

They found something alarming in the metrics for Andromeda, Google's networking fabric. A counter called "Proxy protocol drop: invalid ACK packet" had been flat for months — and then, in February 2026, it spiked sharply upward to several drops per second across the affected europe-west regions.

That date matched a USPS version rollout. This version included a change which was a fix for a different bug but something about it had changed the behavior of how Private Service Connect handled a specific class of packet.

Challenge ACKs: The RFC 5961 mechanism

To understand the next piece of the puzzle, we need to talk about a TCP security mechanism called the challenge ACK, defined in RFC 5961.

Imagine a TCP connection has closed — gone through the FIN/ACK dance and terminated. The server-side (here, the KCP VM) may still briefly remember the connection's sequence numbers in its TCP state machine. Now imagine that shortly afterward, a new client connection arrives on the same source port and destination port (the same 5-tuple). From the kernel's perspective, this could be a stale packet from the old connection, or it could be an attacker trying to inject data into what the kernel thinks is still an open session.

RFC 5961 says: when you receive a SYN that matches an existing or recently-closed connection's remembered state, don't accept it blindly. Send a challenge ACK — an acknowledgment that carries the last known sequence number of the old connection. If the new client is legitimate, it will see this unexpected ACK, recognize the connection is stale, and send a RST to clear the state. The server then accepts the next SYN normally.

The challenge ACK flow — how it's supposed to work

code_block: <ListValue: [StructValue([('code', '# Step 1: New client sends SYN on a reused port\r\nClient → Server: SYN (seq=X, same 5-tuple as an old connection)\r\n\r\n# Step 2: Server is in FIN_WAIT_2, sends challenge ACK\r\nServer → Client: ACK (ack=old_seq — "is this really you?")\r\n\r\n# Step 3: Client recognizes the stale state, resets it\r\nClient → Server: RST\r\n\r\n# Step 4: State is cleared. Server accepts the next SYN normally.\r\nClient → Server: SYN (new connection)\r\nServer → Client: SYN-ACK ← handshake succeeds'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbd616590>)])]>

This mechanism works perfectly — when the challenge ACK is allowed through. But that was exactly what the new USPS change was blocking.

The culprit: A well-intentioned fix

The change in USPS had been designed to fix a legitimate security issue in Private Service Connect's proxy protocol handling. Part of its effect was to drop packets it classified as "invalid ACKs" during the connection handshake phase — specifically, ACKs that arrived before the initial SYN-ACK had been sent, which in normal circumstances would be anomalous.

Unfortunately, challenge ACKs look exactly like this: They arrive from the server before the new handshake has completed, in response to a SYN. From the USPS's new perspective, they looked like invalid packets. They were dropped silently.

Why this broke GKE specifically: Private Service Connect uses NAT to share a single public endpoint across many client pods. This means many different pods — running on different nodes, from different namespaces — can all end up using the same source port after NAT, creating a high rate of 5-tuple collisions. The challenge ACK mechanism was being triggered far more often than it would be in a normal direct-connection scenario. The USPS fix hadn't introduced challenge ACKs — they had always been there. It had introduced the dropping of them, which broke a flow that GKE's TCP stack had silently depended on to clear stale state.

Mitigation: Rolling back the change

In February, we rolled back the USPS change, restoring the old behavior where challenge ACKs were passed through instead of dropped. The rate of "Proxy protocol drop: invalid ACK packet" dropped back toward baseline almost immediately.

By early March , 2026, the customer confirmed that all affected clusters had recovered. No more timeout errors.

But the rollback wasn't a fix. It was a band-aid. It restored the behavior the system had accidentally depended on. The deeper question — why was the KCP VM in FIN_WAIT_2 for over two minutes in the first place? — was still completely unanswered.

That question led to Part 3. And it's where things got genuinely strange.

Part three: Three bugs, one crime scene

The mitigation bought us time, but it left a question hanging in the air: Why did the KCP VM keep connections in FIN_WAIT_2 for so long? A well-behaved TCP stack processes a FIN, acknowledges it, and closes the connection. The KCP VM was failing to do this, leaving half-open connections that persisted for more than two minutes — long enough for the port to be reused by Private Service Connect's NAT layer, long enough to cause the challenge ACK scenario we'd uncovered.

To answer this, we needed to understand not just one bug, but three distinct failure modes that had intersected at the worst possible moment.

Bug #1: Packet reordering at the PIGv

The first piece was discovered through careful analysis of simultaneous captures at two points: the KCP VM's host NIC (inside the physical host running the VM) versus the PIGv — the virtual NIC as seen by the VM.

At the host NIC level, packets arrived in perfect order. But by the time they crossed through USPS and appeared at the PIGv, they had been subtly reordered.

This happens because of how Private Service Connect's miss-thread processing works. When Private Service Connect intercepts a new flow it hasn't seen before (a "miss"), it processes that packet through a slower path that adds the proxy protocol header and sets up NAT state. Subsequent packets in the same flow take a fast path. The consequence: a FIN or RST packet belonging to a flow in mid-close might arrive slightly out of order relative to the ACKs that followed it.

What the KCP VM actually received (reordered sequence)

code_block: <ListValue: [StructValue([('code', "# What should have happened:\r\nClient → Server: FIN, ACK (client initiates close)\r\nServer → Client: ACK (server acknowledges client FIN)\r\nServer → Client: FIN, ACK (server initiates its own close)\r\nClient → Server: ACK (client acknowledges server FIN)\r\n → Connection fully closed. Both sides free.\r\n\r\n# What the KCP VM actually received (reordered by PIGv/USPS):\r\nClient → Server: ACK (arrives first — out of order)\r\nClient → Server: FIN, ACK (arrives second, but kernel sees ACK first)\r\nServer → Client: FIN, ACK (server sends close, enters FIN_WAIT_1)\r\nClient → Server: FIN, ACK (old, now-reordered FIN from client)\r\n → Server should ACK this. But it doesn't..."), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbd6158d0>)])]>

The reordering put the client's FIN after the server had already sent its own FIN. The server entered FIN_WAIT_1, then FIN_WAIT_2 after seeing the client's ACK of the server's FIN. But then the client's own FIN arrived — and the server's kernel ignored it.

Bug #2: Envoy's half-close and the frozen timer

Why couldn't the operating system clean up on its own? There is a Linux kernel parameter called tcp_fin_timeout that governs exactly this: how long a socket in FIN_WAIT_2 waits before being forcibly closed. On the KCP VM, this was set to 60 seconds.

But tcp_fin_timeout only applies to orphaned sockets — sockets that no process is currently holding open. Envoy, the proxy that sits in front of kube-apiserver on the KCP VM, was performing a half-close: it called shutdown(SHUT_WR), which closes the write side of the connection but keeps the socket open so it can still receive data. This is perfectly valid TCP behavior and is how graceful half-duplex termination is supposed to work.

The side effect: As long as Envoy held the socket with SHUT_WR, the kernel did not consider it orphaned. tcp_fin_timeout never started. The socket sat in FIN_WAIT_2 indefinitely, waiting for a client FIN it would never be able to process correctly.

Kernel socket state — captured via netstat just before a new connection

code_block: <ListValue: [StructValue([('code', 'Proto Local Address Foreign Address State\r\ntcp 10.128.0.5:443 10.0.13.77:53426 FIN_WAIT_2\r\n# This socket has been in FIN_WAIT_2 for > 120 seconds.\r\n# tcp_fin_timeout=60 has not fired because the socket is not orphaned.\r\n# Envoy holds it open with SHUT_WR.\r\n\r\n# 2 minutes later, Private Service Connect NAT reuses port 53426 for a new pod connection.\r\n# The kernel sees the new SYN on this 5-tuple and sends a challenge ACK.\r\n# USPS drops the challenge ACK. → i/o timeout.'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbd617490>)])]>

Bug #3: The Linux kernel's silent failure

But there was still a missing piece. Even with the reordered FIN, a healthy kernel should still process a received FIN in FIN_WAIT_2 and move to TIME_WAIT. The socket should eventually close. Why wasn't it?

This led us deep into the Linux TCP state-machine implementation. When the reordered client FIN arrived at the KCP VM, the kernel's tcp_ack() function was called to process it. And here, in kernel version 6.1 (which underpinned the Container-Optimized OS (COS) running on the KCP VMs), there was a bug.

RFC 5961 added stricter validation for acceptable ACK numbers — specifically, an ACK could be considered valid even if it was slightly "old" (within a sequence number window), to handle network reordering. But in kernel 6.1, there was a code path where tcp_ack() returned zero — indicating "this packet was not fully processed" — when it encountered an ACK that was old but within the RFC 5961 acceptable window. The caller of tcp_ack() in the FIN processing path checked this return value, saw zero, and did… nothing. The FIN was received and silently discarded.

The Linux kernel commit that fixed it (upstream)

code_block: <ListValue: [StructValue([('code', 'commit 795a7dfbc3d9\r\nAuthor: Eric Dumazet <edumazet@google.com>\r\nnet: tcp: accept old ack during closing\r\n\r\ntcp_ack() returns 0 when ack is old but acceptable wrt rfc5961.\r\nIn FIN_WAIT_2, this caused tcp_fin() to never be called for a\r\nvalid received FIN, leaving the connection stuck indefinitely.\r\n\r\nFixed: properly handle old-but-acceptable ACKs during connection close.\r\nIntroduced in 6.1, fixed in 6.9.7. LTS backports: 6.6.y and 6.1.y.'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbd6159d0>)])]>

The bug had existed since kernel 6.1 and was already fixed in kernel 6.9.7 — but GKE's KCP VMs were running COS builds based on 6.1 and 6.6. The fix needed to be backported and shipped as a COS image update.

How the three bugs combined

Individually, none of these three issues would have caused an outage of this magnitude. It took all three acting together to create the failure:

Bug #1 alone: Packet reordering is mildly disruptive, but the kernel would normally still eventually process the FIN and clean up, just slower than ideal.
Bug #2 alone: Envoy's half-close is perfectly correct behavior. It only becomes a problem if the FIN is never properly processed.
Bug #3 alone: The kernel's failure to process old-but-acceptable ACKs would cause some connections to linger, but the impact would be limited without the port-reuse pressure from Private Service Connect NAT.
All three together: Reordering causes the FIN to arrive out of sequence; the kernel bug prevents it from being processed; Envoy's SHUT_WR prevents the timeout from firing; the socket stays in FIN_WAIT_2 indefinitely. Private Service Connect NAT reuses the port. challenge ACK is sent. The USPS fix drops it. Connection fails.

The resolution: A three-pronged fix

Resolving this required action at three different levels, coordinated across multiple teams over several weeks:

Immediate mitigation (Feb, 2026): Rollback of change in USPS. Restored the previous behavior where challenge ACKs were forwarded rather than dropped. Customer clusters recovered by early March.
Correctness fix: A replacement change to handle the challenge ACK scenario properly in the Private Service Connect proxy protocol layer, rather than simply restoring the old drop behavior. A PacketDrill test was added to the test suite to prevent regression.
Root cause fix — Linux kernel patch: Upstream patch ("net: tcp: accept old ack during closing"), authored by Eric Dumazet. Merged into Linux 6.9.7. Backported to 6.6.y and 6.1.y LTS branches. — shipping in GKE 1.33.5 and later. The bug is fully resolved at the OS level for all GKE versions running these COS images.

What made this case different

Most networking cases resolve at one layer. You find the misconfigured firewall rule, the exhausted connection pool, the miscalculated MTU. You fix it and move on. This one was different because the symptom — i/o timeout on a connection that never completed — was the same whether you were looking at the Private Service Connect layer, the kernel's TCP state machine, or Envoy's connection management. Every layer had a plausible explanation, and every explanation was incomplete.

What finally cracked it was the discipline of simultaneous packet capture at multiple points in the path. Without captures at the host NIC, the PIGv, and inside the container at the same time, we would never have seen the reordering. Without correlating those pcaps to the USPS metric spike on February 18, we would never have found the dropped challenge ACKs. And without reproducing the issue in a lab environment, we would never have found the kernel bug that made the FIN_WAIT_2 state permanent.

The other thing that made this work was the team. Over the course of two weeks, engineers from seven or eight different specialties contributed: GKE networking, Private Service Connect infrastructure, Hostnet, Andromeda, KCP, and ultimately the kernel community. No single person could have held all of this in their head. The case notes, the shared bug tracker, the meticulously collected pcaps — they were the connective tissue that let a distributed team work as one.

In the end, the customer got back to production. The kernel got a better bug fix. And the rest of us got a very good story about why packet reordering, half-closed sockets, and a well-intentioned security patch should never be allowed in the same room at the same time.

_{This blog is based on real case data from an anonymized customer support engagement and internal bugs. The Linux kernel fix referenced is commit 795a7dfbc3d9, authored by Eric Dumazet. Special thanks to the engineers across GKE, Private Service Connect, Hostnet, Andromeda, COS, and the Linux networking community whose work made this resolution possible.}

What’s new in AI infrastructure and orchestration in October

Fri, 30 Oct 2026 16:00:00 +0000

The Google Cloud AI Infrastructure team had another busy month in October, filled with new product launches and updates to power today’s AI and agentic workloads scalably and efficiently. We were also honored to share our progress with customers, partners, and the broader community at leading industry events, including NVIDIA GTC, PyTorch Conference, and AGNTCon + MCPCon. Let’s dive in!

Product, technology, and tools updates

New compute service: Agent Compute Engine is a purpose-built compute service providing high-performance, hardware-isolated microVM sandboxes alongside expanded agent governance capabilities like Agent Gateway and Agent Registry, all in the service of securing and scaling autonomous AI agents. Its primary benefits include rapid sub-second activation, extreme workload density through copy-on-write memory sharing, enterprise-grade security and identity management, and cross-cloud observability.
AI Hypercomputer updates: At NVIDIA GTC Berlin 2026 we announced updates to our agentic infrastructure, including to A5X instances powered by the NVIDIA Vera Rubin NVL72 platform, Virgo Network scale-out fabric, and Managed Lustre storage. These co-engineered advancements provide enterprises with accelerated performance, enhanced sovereign AI controls, while advanced Confidential Computing capabilities help you train and deploy complex multi-step models securely and efficiently.
GKE product update: CPU startup boost for GKE, in preview, uses Kubernetes In-place Pod Resize to dynamically elevate container CPU resources during initialization and scale them back down to steady-state levels without having to restart the containers. This feature removes startup throttling and accelerates cold starts by up to 2x for demanding workloads, helping to lower cloud infrastructure costs.
GKE product update: The preview of the Wiz Runtime Sensor add-on for Google Kubernetes Engine (GKE) brings low-touch deployment, automated lifecycle management, enterprise credential security, and fleet-wide governance to Wiz's eBPF runtime threat detection for both Standard and Autopilot GKE clusters.
Modernization tools: Together at last. Google Cloud Modernize brings Google Cloud’s migration and modernization tools, including Migration Center, Google Cloud VMware Engine, Google Cloud Mainframe Modernization and our new EKS-to-GKE Migration Agent, into a single portfolio. Enterprise teams now have purpose-built agentic capabilities across infrastructure assessment, platform modernization, and application modernization.
Cloud Run product update: Temporal Serverless Workers on Cloud Run automatically scale worker pools based on Temporal task queue demands and scale down to zero when idle. This integration eliminates cluster management, lowers costs, using Temporal Worker Versioning to safely handle stateful, long-running workflows and AI agent workloads.
New open-source product: llm-d 1.0 is an open-source, Kubernetes-native serving stack that turns a cluster of model servers into a coordinated serving pool, unifying distributed LLM inference across mixed Google Cloud TPU and GPU fleets. Co-founded by Google and partners, it provides production-ready benefits such as cache-aware routing, tiered KV cache offloading, and priority-based flow control to dramatically improve context reuse, reduce time-to-first-token latency, and efficiently scale multi-tenant agent workloads.
New open-source product: Working with Meta, we open-sourced TorchTPU, a native PyTorch stack that integrates Google TPUs directly into the PyTorch ecosystem alongside GPUs and CPUs. Now developers can train, serve, and deploy models natively on TPUs using familiar frameworks like Hugging Face, vLLM, and TorchTitan.
Google Distributed Cloud updates: Organizations that operate under strict sovereignty mandates would love to use AI to help them process data, but can’t send it to the cloud. The latest version of Google Distributed Cloud (GDC) air-gapped brings frontier AI models and multimodal enterprise search with Gemini into these isolated environments. In addition, GDC air-gapped supports NVIDIA HGX B300 systems based on the NVIDIA Blackwell Ultra architecture, and features enhanced networking and security capabilities.
Edge inference product: Google open-sourced ML Drift, a cross-platform on-device GPU compute engine built for LiteRT that abstracts hardware complexities across OpenGL ES, OpenCL, Metal, and WebGPU. The framework delivers peak performance, enhanced 5D tensor support, and memory efficiency for both classical machine learning workloads and advanced generative AI models at the edge.
Open-weight model support: Reflection's Beam is an open-weight, 501-parameter model released under an Apache 2.0 license designed specifically for frontier software engineering and long-horizon agentic tasks. We announced day 0 support and ready-to-run deployment recipes for service Beam on AI Hypercomputer and NVIDIA AI Infrastructure.

Practitioner guides and how-tos

AI network reference architectures: In Networking for AI inference model serving - GKE only and for all other backends, learn about reference architectures for AI inference model serving that provide centralized governance and private IP routing using Private Service Connect, Apigee API Management, and Model Armor. These architectures simplify multi-model scaling and prompt security across GKE, Cloud Run, and hybrid clouds.
How-to guide: A Tale of TPU Observability on GKE: Part 2 - Dynamic Slicing Enhancements breaks down GKE's new Dynamic Slicing observability tools and metrics for next-generation v7 TPUs through an engaging, story-driven observability guide featuring Alex, a lead AI platform engineer. Explore the step-by-step diagnostic workflows with real-world examples and handy command-line snippets. Check out Part 1 here.

Customer and partner updates

AI21 achieves an 83% reduction in time-to-start for AI workloads with AI Hypercomputer: Frontier lab AI21 needed a new stack on which to train its Jamba family, and chose AI Hypercomputer configured with GKE on A3 and A3 Ultra instances. By doing so, AI21 says it cut high-priority job wait times from 72 hours to 12, and manual scheduling interventions from 20 per week to zero.
Visko streams real-time world models on Google Cloud TPUs: Visko, an AI startup developing interactive world models, deployed an end-to-end live inference pipeline that combines video generation, super-resolution upscaling, and synchronized audio onto a single Cloud TPU slice. In this blog, the team discusses how it engineered the pipeline, the architectural optimizations it made, and practical lessons for other startups building ultra-low-latency generative AI.
ZenteiQ scales physical AI inference 4x with GKE and llm-d: In order to pre-train models that understand physical laws, governing equations, and numerical solvers, Indian AI startup ZenteiQ designed and built a large-scale batch synthetic data-generation pipeline that runs across multiple model families and feeds verified datasets into training runs. GKE and GKE Inference Gateway gave ZenteiQ the inference-routing layer within its pipeline to scale aggregate inference throughput 4.07x and grow daily output from 1.47 billion to 5.97 billion generated tokens for every 1,000 accelerators.
Palo Alto Networks modernizes cloud-delivered security service: Palo Alto has worked with Google Cloud since 2018 to deliver its Prisma Access security service, and it was time to update it. In order not to compromise its security guarantees, Palo Alto separated the platform into two independently scalable layers: an ingress layer that handles traffic routing and forwarding running on GKE, with Axion C4A instances; and a security layer that handles threat inspection, running on dedicated single-tenant VMs fronted by an Internal Load Balancer. Each layer can now scale, update, and operate independently.
Oracle is the first hyperscaler to provide TPUs outside Google Cloud: By deploying Google TPUs, Oracle is broadening the range of AI chips available through its Oracle Cloud Infrastructure (OCI), and giving customers more options for running AI workloads. Google TPUs on OCI will be available in 2027.

Research, reports and deep-dives

New resource: As enterprise organizations shift from isolated AI experiments to production-scale systems, different departments — from engineering and finance to executive leadership — often speak different technical dialects; The Encyclopedia of AI Infrastructure bridges that communication gap by demystifying complex infrastructure jargon.
Analyst report: We are thrilled to announce that Google was named a Leader in the inaugural 2026 IDC MarketScape for Agentic AI Platforms.
Deep dive: A companion technical guide describing the architecture of Agent Compute Engine and how Google’s Gemini team needed both the isolation of a VM and the speed and density of a container to deliver personal agents for every Gemini user.

September 2026

Product, technology, and tools updates

Google Kubernetes Engine updates: The GKE team is all about improving the scalability of the platform, and in September, those improvements came in many shapes and sizes:

New feature: Need an execution runtime with higher density for your agentic workloads? We engineered the new open-source GKE Agent Substrate to run millions of sandboxes with 10x higher density than standard container runtimes. Agent Substrate also delivers sub-500ms resume operations at over 500 suspend/resume activations per second with a native zero-trust kernel and network isolation.
Product update: GKE now has scale-to-zero capabilities built-in. No need to configure complex components to scale your workloads down, thanks to the HPA with the Autoscaling Metric and support for KEP-2021, which do the job for you, out of the box. Read the blog to learn more.
Product update: Further, the GKE HPA (with the above-mentioned Autoscaling Metric) now lets you scale up and down based on custom PromQL metrics, in addition to standard metrics, allowing you to trigger workloads according to conditions that are meaningful and unique to your business. Read more here.
New feature: Yet another scalability feature is GKE Pod snapshots, which lets you save the running state of your workload, including CPU and GPU memory, and restore it on demand. According to internal tests, GKE Pod snapshots can reduce AI inference start-up by as much as 89%. Learn more here.
New migration tool: Finally, if you’ve always wanted to migrate your container workloads from AWS EKS to GKE but feared a daunting, high-friction engineering endeavor, we’ve just launched GKE agentic migration, a purpose-built agent plugin that replaces brittle, ad-hoc prompting with an AI-assisted migration pipeline protected by deterministic guardrails. Designed as a compilation of agent skills and a local Model Context Protocol (MCP) server, it uses AI to translate complex AWS EKS IaC and Kubernetes manifests directly into GKE landing zones. Get started with the onboarding guide.
Feature updates: Reinforcement learning (RL) and evaluation workloads are a beast: In a standard agentic RL loop, an LLM policy generates actions like code snippets on GPUs and executes them inside isolated CPU sandboxes to observe a reward signal. However, when scaling up this loop to support tens of thousands of parallel rollouts, infrastructure bottlenecks emerge, for instance idle accelerators, image cardinality, and a saturated control plane. To help, we developed GKE Agent Sandbox optimized for RL, plus an Agent Sandbox RL orchestration SDK and native integrations for popular RL gyms and harnesses. All are now generally available, and you can learn more here.

Storage updates: AI trains and creates lots of data, and that data has to live somewhere — in block storage systems, file systems, object stores and databases. We announced enhancements to our storage portfolio to help this critical layer roll with the agentic punches:

Product update: Filestore agent volumes offer high-performance, elastic, persistent file storage for agentic workloads. Thanks to its tight integration with GKE Agent Substrate and GKE Agent Sandbox, Filestore agent volumes automatically allocates and attaches a dedicated, isolated file workspace to GKE agent sandboxes in milliseconds. Request access to the preview here.
New product: If you run generative AI and RAG data layers — think Milvus, Pinecone, Qdrant, Vespa, Redis, and in-memory context caching — you may want to take a look at the M4N family of VMs, now GA, which offers the highest per-core IOPS and throughput of leading hyperscalers. Paired with Google Cloud's custom Titanium offload architecture and paired with Hyperdisk Extreme, M4N instances deliver up to 25,000 MiB/s (25 GiB/s) of aggregate host storage performance and up to 1 million IOPS.
New product: Another new Compute Engine product, Z4D, is GA, and a strong storage solution for AI/ML training and inference workloads. When configured as a bare metal instance, Z4D provides both the high local SSD (LSSD) capacity and low latency required by agentic microVMs, so you can run thousands of isolated sandboxes per host with native performance and efficiency. Learn about Z4D machines here.
Product update: Today’s AI training and inference pipelines create data faster than most storage management systems can keep up, creating challenges for teams trying to understand their storage estates. A new version of Storage Intelligence advisor makes it easier to answer the question: "What’s in my buckets?" and quickly identify unexpected changes. Then, enhanced batch operations let you automate bulk changes across your buckets — say, move storage classes, mass-delete stale or temporary data, or apply metadata, tagging, retention, or encryption changes. Learn about the latest in Storage Intelligence advisor here.
New product: Last but not least, a new version of AlloyDB for PostgreSQL brings together pioneering Google infrastructure — Colossus distributed file system, and Jupiter network — to power a new, no-compromises database architecture for the agentic era.

Practitioner guides and how-tos

How-to: Wish you could make GPUs and TPUs scattered around the globe behave as a single pool behind a single entry point? In this blog, we show you how to do just that. At the edge, the multi-cluster GKE Inference Gateway focuses on global, multi-region traffic distribution and high availability. Beneath that, the LLM-d router handles complex, memory-aware scheduling algorithms to keep utilization high. This architecture is deliberately runtime-, model-, and accelerator-agnostic, and in tests, routing traffic through the multi-cluster GKE Inference Gateway added less than 1% overhead.
Guide: Using or planning to use GKE on TPUs for AI model training or inference? Training massive Large Language Models (LLMs) or running high-throughput inference serving represents a significant investment in specialized AI hardware, such as Cloud TPUs and GPUs. To get the most out of every dollar spent, you need to understand workload lifecycle metrics in GKE. This detailed guide explains how to turn opaque cluster behaviors into actionable telemetry.

Customer and partner updates

GKE customer: Learn why gaming startup SeaVerse relies on GKE Agent Sandbox for its multi-tenant workloads, and how the platform helped it decrease its infrastructure costs by 60%.

Research, reports and deep-dives

In case you missed it, we’re also thrilled to share that Google has been named a leader, including achieving the highest score on either product or strategy, in three key analyst reports from Gartner and Forrester.

Google is a leader in The Forrester Wave™: Public Cloud Platforms, Q3 2026: Highest overall score of any cloud provider!
Google named a Leader in 2026 Gartner® Magic Quadrant™ for Strategic Cloud Platform Services: Positioned furthest for “Completeness of Vision” of all vendors evaluated.
Google is a Leader in the 2026 Gartner Magic Quadrant for Container Management: Positioned highest in “Ability to Execute” of all vendors evaluated.
Google Cloud achieved a Gold rating in the latest SemiAnalysis ClusterMax 3.0 report, which evaluates the reliability, performance, support, pricing, and security of GPU providers globally. See the full report for more.

August 2026

Product, technology, and tools updates

Product update: Filestore, Google Cloud’s first-party, secure, scalable NFS file service, has emerged as a popular storage platform for AI and agentic workflows, and now, it’s even better suited to the task, with a new backend storage layer built directly on Colossus, Google’s foundational distributed storage system. This new backend lets you provision IOPS independently from storage capacity, and is deeply integrated with GKE. In AI environments, this can help you service so-called agentic swarms — large groups of agents that need to read and write to a common dataset — without a drop off in performance. For more, check out the blog post.
New feature: gVisor sandboxes are now available in distributed Ray clusters on GKE. In partnership with Anyscale, we introduced an experimental library for Ray that brings gVisor, Google’s open-source application kernel, directly into distributed Ray clusters. gVisor provides lightweight environments with stronger isolation than ordinary containers, plus fast startup times and low memory overhead. To try out these sandboxing capabilities on GKE, head over to the Ray sandboxing User Guide.
Product update: Looking for high-performance, easy-to-use infrastructure on which to run a personal AI agent, but don’t want to spend a lot of money? New Cloud Run instances are dedicated, singleton compute runtimes on Cloud Run that won’t shut down when the agent is idle. Better yet, the cost to run a Cloud Run instance with 1 vCPU and 1 GiB of memory continuously for 30 days is just $5.70.

Practitioner guides, documentation and how-tos

How-to guide: Big news in Model Context Protocol (MCP) land: As of the 2026-07-28 specification, the protocol core is “completely stateless. The handshake is gone. The initialize / initialized handshake (SEP-2575) and the logical Mcp-Session-Id header (SEP-2567) have been removed entirely. Instead, every request is now self-describing and independent.” Whoa. Learn more about the changes that the latest MCP specification brings, and more importantly, how to implement them, in this Google Developers blog.
Guide: Real-time AI systems make a mess of traditional network load balancing techniques. “Instead of handling isolated requests, the backend has to manage a continuous, live bidirectional stream. You’re dealing with a constant stream of audio chunks, transcripts, model outputs, and synthesized speech flowing back and forth simultaneously.” Things only get worse when the user gets involved. “The server has to immediately halt its current speech generation, pivot to update the context, maybe trigger a new tool, and start drafting a different response; this must be done without dropping the connection.” For a new approach to managing load in the AI era, read Scaling real-time AI agents with session-aware load balancing.
How-to: Learn how to build an elastic, scalable LLM inference platform on GKE, even with a mix of different GPU accelerators. The proposed architecture combines Capacity Advisor and Compute Advisor, plus high-performance storage like RunAI:model streamer or GCPFuse with parallel downloads. Get all the details here.
Documentation: The thing about hosts with GPUs or TPUs is that you can’t use live migration to update them, setting up a maintenance challenge. In this new docs page, learn how to update accelerator-equipped hosts according to your tolerance for downtime for your training and inference workloads.
Documentation: Advanced Compute Images, or ACIs, are standardized image stacks for AI/ML and HPC infrastructure, so you don’t need to manually build your own custom images. In this new docs page, learn how to create an ACI image using the Google Cloud CLI, console, or SchedMD's Slurm workload manager.
Guide: AI workloads are notoriously difficult to architect, resource-intensive, and bursty, which can also lead to scaling bottlenecks and large pools of underutilized — or misutilized — compute resources. A new blog outlines the three main ways to achieve dynamic capacity management in Google Cloud: 1) scheduling capacity for planned downtime; 2) maintaining automated fallback capacity for unplanned downtime; and 3) relying on GKE’s core orchestration capabilities to automate resource allocation.

Customer and partner updates

Business orchestration software provider UiPath was dealing with spiky workloads, and wanted more predictable costs. To get there, it re-architected its infrastructure, moving from isolated clusters to a shared Google Cloud GPU fleet that included both A3 VM instances (NVIDIA H100 GPUs) for training with G4 VM instances (NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs) for inference. You can read more about their architecture here.
Mirendil, an frontier AI lab focused on accelerating AI development, announced that it is using AI Hypercomputer with both TPUs and NVIDIA GPUs to support its model pre-training and post-training applications.
Replenit, a retail CRM provider, built its AI decision engine in Google Cloud, using BigQuery, Gemini Enterprise Agent Platform, and open-source Gemma models that it runs on Cloud TPUs. This latter combination provided Replenit with 90% lower pipeline costs than their previous cloud provider, the company reports. Read the full case study for more.
Malachyte architected its AI-powered e-commerce recommendation platform on top of Bigtable, Managed Service for Apache Kafka, Pub/Sub, Compute Engine, and last but not least, GKE. See how it all comes together in this blog.

July 2026

Product, technology, and tools updates

Product update: Google Cloud Managed Lustre is now GA, and available in four distinct performance tiers that deliver throughput ranging from 125 MB/s, 250 MB/s, 500 MB/s, to 1000 MB/s per TiB of capacity — with the ability to scale up to 8 PB of storage capacity. The Managed Lustre solution is powered by DDN’s EXAScaler, combining DDN's decades of leadership in high-performance storage with Google Cloud's expertise in cloud infrastructure.
Product update: C4N network and storage optimized VMs are now GA. C4N is our first network- and block-storage-optimized VM series built to eliminate data-transfer bottlenecks. Powered by 5th Gen Intel Xeon Scalable processors and built on Google's Titanium offloading hardware, it achieves 400 Gbps network bandwidth, 95 million packets per second (MPPS), and up to 25 GiB/s of block storage throughput when paired with Hyperdisk Extreme.
New feature: GKE Dataplane V2 up to 15K Nodes with Network Policies (GA). This capability enables standard GKE clusters to scale up to 15,000 nodes while maintaining full active Network Policy enforcement, supporting the massive infrastructure needs of large enterprise and AI/ML customers.
New feature: Co-operative time-slicing in llm-d. If you’re running reinforcement learning (RL) workloads, you can now interleave independent RL jobs onto shared physical hardware, increasing aggregate accelerator duty cycles from a ~40% baseline up to 70% without impacting model convergence or accuracy.
New AI security tool: Looking to secure your AI supply chain on GKE, deploy AI workloads safely, and cut down on shadow AI? We open-sourced k8s-aibom, a lightweight, unprivileged Kubernetes controller that continuously monitors container clusters to automatically detect running AI runtimes (like vLLM and Triton) and generate standard CycloneDX Machine Learning Bill of Materials (ML-BOMs). Check out the k8s-aibom project and get involved.

Practitioner guides and how-tos

How-to guide: On July 27, Google announced Day 0 support for Moonshot AI’s Kimi K3 2.8-trillion-parameter open-weight model, the day weights were released. Whichever your preferred deployment path — via Model Garden, custom orchestration, or GKE with llm-d recipes — this guide offers detailed step-by-step instructions to help you evaluate and pilot Kimi K3 in Google Cloud.
How-to guide: Google Kubernetes Engine (GKE) managed DRANET supports both GPUs and TPUs. There are several configurations to use this implementation, including standard cluster (where you have full control) and autopilot cluster (where Google does the heavy configs for you). Take a deeper dive in the hands-on lab, GKE Autopilot clusters with TPUs, GKE managed DRANET and Gemma 4.
How-to guide: Learn to run Ray on TPUs, not GPUs. In Part 1 of this two-part series, we discuss TPU slices (hint: Ray thinks of them as just another accelerator on which to schedule), then walk through Ray’s various AI libraries (Part 2).
How-to guide: Evaluate TPUs for sample workloads using a new microbenchmark suite that helps you accurately assess whether a device is achieving its theoretical performance specifications, and to identify specific performance gaps or architecture-specific bottlenecks. Dive in here.
How-to guide: Scale your agents without killing your budget. Learn how GKE orchestration can help you safely pack more agents onto a fixed compute footprint with GKE Agent Sandbox and Pod snapshots. Whether your goal is performance or cost optimization, we teach you how to turn the right dials for optimal agent efficiency.
Technical blueprint: Inside the optimization of Mistral 3 large inference on Ironwood. This blog outlines how one Google team optimized Mistral 3 large MoE model inference on Google’s Ironwood (TPU v7x), achieving a 1.5x performance gain. They did so with hybrid sharding, replacing linear VPU summations with tree reductions, optimizing GMM/MLA kernels, and adopting asynchronous scheduling. As a result, they boosted throughput by up to 48% while maintaining benchmark accuracy neutrality. Read the full blog here.

Research, reports and deep-dives

Report: Google was named a Leader in the inaugural GartnerⓇ Magic Quadrant™ for AI Infrastructure, positioned highest for ‘Ability to Execute’ and furthest for ‘Completeness of Vision’. Gartner called out Google’s proprietary scalable compute, integrated AI Hypercomputer architecture, and the scale of our AI compute capacity as key strengths. Download a copy here.
Report: We recently surveyed more than 1,400 senior IT leaders for our State of AI Infrastructure report, and a resounding pattern emerged: The gap between AI ambition and infrastructure reality is widening. In fact, 83% of organizations say they require infrastructure upgrades to support production-grade agentic AI. Read the accompanying blog to understand how adapting your infrastructure to meet the demands that agentic applications place on your systems will help you move from pilot to production.

June 2026

Product, technology and tool updates

Product update: Protecting sensitive data used with AI is a critical part of advanced and secure cloud infrastructure. Confidential Computing cryptographically protects data in use in hardware-based Trusted Execution Environments (TEEs) with verifiable data integrity, and is now available on the accelerator-optimized G4 machine series, featuring NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs. Get started with Confidential G4 VMs and Confidential G4 GKE Nodes.
Developer resource: The new TPU Developer Hub is the place to go for model builders, optimizers, and developers to learn to unlock the full performance of Google Cloud TPUs. Read more in this blog.
New product: Scale your AI workloads with the new OpenTelemetry-Based TPU AI Telemetry Collector Agent. For the first time, you can route high-fidelity TPU hardware telemetry to Google Cloud Monitoring, Google Managed Prometheus, or your own self-hosted Grafana stack.

Practitioner guides and how-tos

How-to guide: Learn how to build high availability into an AI inference workload running on GKE Inference Gateway with TPUs, Cloud Storage FUSE and Dynamic Resource Allocation (DRA). This blog provides an overview, or you can get all the technical details in the hands-on codelab.
How-to guide: Did you know you can connect your AI agents to unstructured data in Cloud Storage via Model Context Protocol (MCP)? In this blog, learn about why would want to do that from three customer examples, then how to do it, choosing either a fully managed service, or a self-managed local server for more customization and control.

Research, reports and deep-dives

Report: According to an independent benchmark report, GKE Inference Gateway outperforms the next leading managed Kubernetes service with 15.7% higher throughput, 92.8% shorter wait times, and 62.6% lower inter-token latency. This performance can be attributed to its use of prefix caching, which optimizes LLM performance by storing the KV cache (activation states) of long, repetitive prompt prefixes. Learn more in the blog.
Architecture deep dive: A closer look at the cold start problem, this time for TPUs and GKE, and how the Run:ai Model Streamer can help change the dynamic.

Customer and partner updates

Customer win: Leveraging GKE, BigQuery, Cloud SQL, and Gemini Enterprise Agent Platform, Pager Health is eliminating operational fragmentation to deliver a simplified, personalized U.S. healthcare experience that transforms lives.
Customer win: Trustpilot, the customer review platform, built a high-volume streaming pipeline using fine-tuned Gemma models with Dataflow and Gemini Enterprise Agent Platform running on cost-optimized A2 VMs using A100 GPUs, as well as optimized version of vLLM maintained by Gemini Enterprise Agent Platform.

May 2026

Product, technology and tool updates

Product update: GKE Agent Sandbox is now generally available.
New open-source project: Agent Substrate is a new open-source project aimed at continuing to push the limits of agentic infrastructure density
New feature: Google AI Edge Portal, a solution for testing and benchmarking on-device machine learning (ML) at scale, now supports benchmarking and debugging on-device LLMs. Read more here.
Product deep dive: We went into depth about Cloud Storage Rapid, a new family of high-performance storage offerings for AI workloads. At launch, offerings include Rapid Bucket (formerly Rapid Storage), a high-performance zonal object storage offering, and Rapid Cache (formerly Anywhere Cache), which accelerates reads on-demand and colocates compute and data for workloads in existing buckets.

Research, reports and deep dives

Architecture deep dive: Google Global Infrastructure VP Bikash Koley and Engineering Fellow Arjun Singh provide a high-level overview of the challenges that AI workloads pose to network infrastructure, and discuss the deep enhancements we’ve made to our data center fabrics, WAN, and global networks to better support them.
Architecture deep dive: We unveiled a new cluster-level reliability model for developing frontier AI models on TPUs, ditching instance-level reliability

Customer and partner updates

Customer win: Visual media provider Imgix serves more than 8 billion images and videos from AI Hypercomputer equipped with G4 VMs powered by NVIDIA RTX PRO 6000 Blackwell GPUs.

How GKE networking scales to support high densities of concurrent AI agents

Wed, 28 Oct 2026 16:00:00 +0000

What do you need to orchestrate high-density concurrent AI agents? As we transition from static requests toward autonomous agentic workflows, the need for secure, high-density sandboxes is pushing conventional infrastructure far beyond its architectural breaking point. This transition is driving top-tier research labs to demand infrastructure at unprecedented scale — often up to one million concurrent code-execution sandboxes!

At this scale, the efficiency of your Kubernetes networking infrastructure directly impacts the economic viability of an agentic initiative. In advanced workflows like reinforcement learning (RL), extreme pod churn can mean thousands of isolated sandboxes spinning up and down every minute. To make sure that valuable GPUs and TPUs are spending their time executing models rather than waiting on sandboxes to initialize, you need sub-second provisioning and extreme compute density.

By providing secure, lightweight isolation, Agent Sandbox on GKE is uniquely positioned to power these highly dynamic execution environments. Furthermore, for next-generation workloads with millions of concurrent idle sessions, Agent Substrate on GKE builds on these primitives to deliver even greater density and sub-second state rehydration.

In this post, you will learn how GKE Dataplane V2 optimizes network identity management, solves identity scaling limits, and enables sub-second provisioning to power high-density, large-scale AI agent workloads.

A common scale challenge: network identity management

Operating with massive concurrency introduces fascinating new engineering challenges, particularly in how you manage network identities. Traditionally, custom controllers and workload orchestrators often assign unique, dynamic labels to each pod (such as agent-id, timestamps, or claim-uid). GKE Dataplane V2, powered by eBPF technology, helps enforce robust network security policies by mapping pod labels to a numeric identity. Because agentic workflows generate highly unique labels for every pod, the system proactively provisions a distinct identity for each sandbox. In a rapidly changing environment, this serial allocation process can briefly pause pod creation as the system securely updates eBPF maps, occasionally leading to slower-than-desired initialization. Additionally, sophisticated workloads can approach the cluster-wide limit of 65,536 network identities, either through this rapid generation or overall scale of the workload itself.

GKE network dataplane innovations for an agentic world

Providing a reliable and highly secure platform is fundamental to everything we do. As such, we needed to find a way to provide the same underlying identity and security controls while providing higher scalability and performance. That's where the architectural innovations in GKE Dataplane V2 come in. A customer has testified to the success of Agent Substrate on GKE to orchestrate hundreds of thousands of secure environments across clusters and handle spikes of over 30,000 sandboxes on a single cluster.

To achieve this, GKE relies on a two-pronged networking approach:

Migrating high-cardinality data

Google's GKE Dataplane V2 separates "identifying" data from "functional" metadata. Ensure your fields that do not actively participate in service selector matching or core resource grouping are migrated from the Pod's Labels set to the Annotations set. GKE Dataplane V2 intelligently calculates network identities based on labels rather than annotations, moving this high-cardinality metadata out of the labels completely bypasses the need for per-Pod identity generation. This elegant shift natively resolves identity scaling limits without sacrificing visibility.

Advanced Cilium label filtering

In addition, applying precise label exclusion filters (e.g., using the cilium-config-emergency-override ConfigMap in kube-system) by configuring rules like !agents.x-k8s.io/sandbox-name-hash or !timestamp, these explicitly instruct GKE Dataplane V2 to gracefully ignore these dynamic labels during security identity calculation.

This means that even if a developer framework inadvertently introduces high-cardinality labels, configuring appropriate exclusion filters allows Google’s network dataplane to drop them automatically.

As a result, identical agent types continue sharing a single Cilium pod identity, maintaining the ultra-low-latency startup times critical for dynamic AI workloads. By preventing linear identity growth and avoiding Cilium’s 65,536 identity limit, GKE provides leading speed and efficiency for fully isolated agents with 300 sandboxes per second at sub-second latency.

Looking ahead

For workloads that require extreme performance, scale, and density, Agent Substrate natively integrates these GKE Dataplane V2 recommendations while bypassing Kubernetes scheduling limits, pushing GKE’s performance and scale even further. In benchmark validation running 200,000 agents, Agent Substrate sustained 500 activations per second at less than 500 ms suspend and resume latency. You can deploy it on your existing GKE clusters today to minimize idle compute costs for your most demanding AI workflows.

By optimizing network identity calculation and filtering high-cardinality metadata, GKE Dataplane V2 addresses critical infrastructure bottlenecks in large-scale AI workloads. These architectural improvements reduce sandbox provisioning latency to sub-second levels, minimize expensive GPU idle times, and help maintain high compute density. Ultimately, these efficiencies let you scale your agentic AI operations reliably while controlling operational costs.

To learn more, explore Agent Sandbox and Agent Substrate on GKE.

How to troubleshoot GKE workloads faster with VPC Flow Logs

Wed, 28 Oct 2026 16:00:00 +0000

Having visibility into your network traffic is essential to operating Kubernetes environments effectively. When a service slows down or loses connectivity, knowing who is talking to whom lets you narrow the problem down quickly. In a standard cloud network, flow logs provide that view by recording connections between IP addresses, ports, and protocols. In Kubernetes, however, pods are temporary. Their IP addresses change constantly as pods scale, crash, or move between nodes, so a raw log entry showing 10.4.2.18 talking to 10.4.8.44 tells you little about which workload (and pod) owned that address, what namespace it was in, or what application it was trying to reach. By the time you search your cluster, the pod may already be gone!

In a Google Cloud environment, VPC Flow Logs closes this gap at the network layer. Google Cloud's virtual network already sees every connection a pod makes, so each sampled flow is recorded with the Google Kubernetes Engine (GKE) metadata behind the IP address: pod, workload, namespace, and service. You get workload-level attribution for L3/L4 traffic without deploying extra agents or sidecars, and it complements in-cluster Container Network Interface (CNI) and dataplane observability tools you may already run when you need deeper detail. Let’s take a deeper look at how VPC Flow Logs and GKE work together, and how to get started with it.

How GKE metadata enriches VPC Flow Logs

VPC Flow Logs samples traffic directly inside Google Cloud's virtual network. When flow logs are enabled for a subnet that hosts a GKE cluster, every record of pod traffic that passes through the VPC network is annotated with the Kubernetes context that owned the IP address at the time of sampling. This helps ensure the context is preserved even after the pod is gone:

Cluster: The cluster name and location.
Workload: The pod name and namespace, along with the controller that owns the pod, such as a deployment, StatefulSet, or DaemonSet.
Service name: The Kubernetes service that received the traffic.
Node and zone: The standard VPC fields, including the node VM and its zone on both ends of the flow.

If you register your GKE workloads in App Hub, each record also carries application-level labels (apphubSource and apphubDestination) that identify the business application, environment, and criticality of each endpoint. This is especially useful when GKE microservices talk to App Hub-registered resources outside the cluster, such as Cloud Run services.

Before you start

Upgrade your log bucket to Observability Analytics (formerly Log Analytics): Flow Analyzer reads only from an upgraded bucket, typically _Default. Note: This upgrade is permanent.
gcloud logging buckets update _Default \ --location=global \ --enable-analytics

Enable VPC Flow Logs organization-wide: One organization-level configuration covers every subnet in every VPC network, including future clusters (requires Network Management Admin role). Keep include-all-metadata on so every record carries namespace and pod details. To balance visibility against cost, you can add a secondary sampling rate (--flow-sampling), an aggregation interval (--aggregation-interval), or a filter expression (--filter-expr) to view only the traffic of interest.
gcloud network-management vpc-flow-logs-configs create CONFIG_NAME \ --location=global \ --organization=ORGANIZATION_ID \ --metadata=include-all-metadata

Turn on intranode visibility: Autopilot clusters already have it, but you need to enable it on Standard clusters. Note: Enabling it restarts control plane and node components and adds some per-node overhead, so test it on a non-production cluster first.
gcloud container clusters update CLUSTER_NAME \ --location=LOCATION \ --enable-intra-node-visibility

Now you’re ready to get started.

Analyze traffic visually in Flow Analyzer

VPC Flow Logs write every record to Cloud Logging, you can search and query it in Logs Explorer or Log Analytics. For day-to-day work, Flow Analyzer, part of Network Intelligence Center, turns the same records into an interactive chart and table, so you can group and filter traffic by cluster, namespace, workload, pod, or service in a few clicks.

The walkthroughs below use Online Boutique, an open-source microservices demo, on a GKE Autopilot cluster in us-central1. A loadgenerator deployment sends synthetic shopper traffic to the frontend, which calls productcatalogservice, cartservice, currencyservice, recommendationservice, shippingservice, adservice, and checkoutservice.

Case 1: Turn IP addresses into workload names
Whether you’re investigating an incident or assessing an architecture change, the first question is always the same: Which endpoints are communicating, and at what volume?

Rather than presenting ephemeral IP address pairs, GKE metadata allows Flow Analyzer to answer this question directly using Kubernetes workload names.

Steps in Flow Analyzer:

Open Network Intelligence Center > Flow Analyzer and set the time range to last 1 hour.
Under Source and Destination, set Organize flows by to GKE workload.
In the filter bar, add the GKE workload is known filter so only pod-to-pod traffic is shown.
Click Run new query.

Grouping traffic by Source GKE workload and Destination GKE workload in Flow Analyzer to map microservice dependencies without raw IP lookups.

Key insights: Sorted by volume, the table is a ready-made application dependency map: frontend handles incoming requests from loadgenerator (representing user traffic), while productcatalogservice answers calls from both recommendationservice and frontend. Because each row shows workload names instead of raw IP addresses, any unauthorized or unexpected workload-to-workload communication immediately becomes apparent.

Case 2: Drill into a single service
When a service is slow or about to change, you need to know who depends on it and how heavily.

Steps in Flow Analyzer:

Change Traffic aggregation to Client ↔ server (bidirectional), so requests and responses between a pair are counted together.
Under Server, add the filter GKE workload = productcatalogservice.
Click Run new query, then click Details on the top row.

Filtering on Server GKE workload (productcatalogservice) with Client ↔ server (bidirectional) aggregation to view caller share and the Details timeline.

Key insights: You get every caller of the catalog service, each caller's share of its traffic, and a timeline for every pair, so if one caller's line climbs while the others stay flat, you know where to look first. When the question is latency rather than volume, switch the display mode to latency to rank the same callers by round-trip time.

Case 3: Find the pods that talk across zones
In a regional GKE cluster, chatty services that land in different zones send every request across a zone boundary, adding inter-zone data transfer charges. Because VPC Flow Logs record the zone at both ends of every flow, the pod pairs driving that traffic are easy to spot.

Steps in Flow Analyzer:

Switch from Basic filters to SQL filters, which let you compare two fields of the same flow.
In the WHERE box, enter src_gcp_zone <> dest_gcp_zone.
Set Organize flows by to Source GCP zone, Source GKE pod, Destination GCP zone, and Destination GKE pod.
Click Run new query.

Using SQL filters with src_gcp_zone <> dest_gcp_zone and multi-attribute grouping to identify pod pairs driving cross-zone network traffic.

Key insights: The table displays cross-zone traffic ranked by volume, identifying the exact pods and zones on each end. In our cluster, the top flows occur between a frontend pod in us-central1-b and a currencyservice pod in us-central1-c, followed by checkoutservice, shippingservice, and productcatalogservice pairs. To investigate further, click Explore in Log Analytics to open the underlying SQL query for additional customization, such as adding joins or time buckets.

GKE has a built-in answer: set trafficDistribution: PreferSameZone (PreferClose on older versions) on the Service to prefer healthy endpoints in the caller's zone, and make sure each zone has enough replicas to serve its own traffic.

What to keep in mind

Pod-to-internet traffic: VPC Flow Logs carry no pod annotations, because the masquerade agent replaces the pod IP with the node IP before the flow is sampled. Keeping internal destinations in the non-masquerade list preserves pod names for those flows, and if you run ip-masq-agent with intranode visibility, that list must also include the cluster's node and pod IP ranges to avoid connectivity problems.
Blocked traffic: Traffic blocked by a network policy never reaches the VPC network, so use NetworkPolicy logging to investigate it.
Sampling: Flow logs are sampled, which makes byte and packet counts well suited for ranking and trending rather than per-connection accounting.

Next steps

If your clusters already have VPC Flow Logs turned on, upgrading the _Default bucket and opening Flow Analyzer is all it takes to reproduce the three walkthroughs above. To learn more, see the documentation:

Introducing Agent Compute Engine and expanded agent governance

Thu, 22 Oct 2026 04:00:00 +0000

Blog authors: Nirav Mehta, VP Product Management; Michael Bachman, VP/GM; Kevin Hutchins, VP Product Management

Scaling Gemini to planet scale taught us firsthand that securely running autonomous agents is fundamentally an infrastructure challenge. The shift from single-turn prompting to interactive agents requires a new paradigm of execution and control. To support massive concurrent agent sessions, we engineered new compute and governance solutions for Gemini to help solve these challenges.

Today at AGNTCon + MCPCon, we are delivering these capabilities to customers by launching Agent Compute Engine, a purpose-built compute service that already powers Gemini as an agent for work tasks. It will be available in preview in the coming months, providing hardware-isolated runtime sandbox security. Additionally, we are extending Gemini for business’ agent governance capabilities to support agents wherever they operate. Together, these capabilities deliver the control you need to run, govern, and scale your agents.

Under the hood of Agent Compute Engine

Agent Compute Engine arose from the need to handle Google’s massive agentic workloads that write and execute code, run tools, drive browsers, orchestrate enterprise workflows, and iterate rapidly and at scale. While traditional compute primitives are excellent for standard workloads, they weren't optimized for the unique, highly iterative nature of AI agents.

Purpose-built for agents, Agent Compute Engine gives every Gemini agent its own high-performance, secure computer to act upon, powering Gemini's dynamic, high-churn sessions. Agent Compute Engine is optimized specifically for the unique lifecycle patterns of AI agents, and you will soon be able to use this service to power your own custom agents. Highlights include:

Speed with efficiency: Agents start, pause, and resume constantly and require an architecture that delivers speed, at scale, all with extreme efficiency. Agent Compute Engine delivers microVMs that boot rapidly, so an agent loop or an eval sweep doesn’t wait on infrastructure. Further, supporting Google users with multiple agents requires a cost-effective solution that can get the most out of a physical machine. Agent Compute Engine achieves this by using copy-on-write memory sharing to run multiple agents on a single operating system template, and pausing idle microVMs via fast checkpoint and restore. When an agent finishes executing a tool call and enters the “thinking” phase of the agentic loop, the microVM signals that it's idle, and the host moves it from an active to an idle state. In other words, it can pause a running microVM while an agent waits on a human or an external system, and quickly resume it from snapshots.
Enhanced security, isolation and governance: As agents take on more complex tasks, the risk of untrusted or adversarial code executing inside a sandbox is an ever-present reality. By running Agent Compute Engine with Titanium, Google’s offload infrastructure processing unit (IPU), we deliver silicon-rooted defense-in-depth built for untrusted AI agents. Agent Compute Engine delivers strong isolation via a hardware-isolated Cloud Hypervisor boundary with its own guest kernel and root access. (For a closer look at Agent Compute Engine’s microVM architecture and security, read the technical deep dive.) Each microVM runs its agent’s own identity, using short-lived credentials. Every microVM also gets its own cryptographically verified ID, and service credentials are injected in egress outside the microVM. Integrated with Agent Gateway, Agent Compute Engine delivers comprehensive observability, policy and security evaluation.
Persistence and enterprise integration: Agents require the ability to persist agent data across multiple, long-running sessions. To solve this, each Agent Compute Engine microVM has its own Hyperdisk, enabling persistence of execution workspaces and agent context. In addition, in order to do meaningful work, they need the ability to use enterprise data without moving it. By weaving Agent Compute Engine directly into Google’s cloud and identity fabric, enterprise connectivity is governed by the exact same security and identity model that protects the microVM itself, enabling a unified identity model as well as auditability.

"Scaling the Gemini agent to planet scale requires compute that spins up rapidly and operates with extreme efficiency. Agent Compute Engine microVMs give us the responsiveness, hardware isolation, and workload density needed to deliver interactive agent sandboxes to our users around the world." - Gautham Thambidorai, VP & Google Fellow, Gemini

Building on GKE with Agent Substrate

For platform teams looking for customization and multi-cloud portability, or who have already standardized on Kubernetes, Agent Substrate on Google Kubernetes Engine (GKE) is an open-source execution runtime that brings the same agent-first design to Kubernetes. It’s built on a zero-trust architecture that cleanly decouples agent execution from machine management. When an agent becomes idle, Agent Substrate snapshots its active memory and returns the machine to the pool, then resumes the agent the moment it’s needed again. Because it’s open source and Kubernetes-native, you can customize it, extend it, and run it alongside your existing workloads. Some Agent Substrate highlights include:

High-density: Agent Substrate delivers as much as 10x higher compute density compared to traditional compute while maintaining full kernel isolation and identity-based network security for every agent.
Sub-second activation: In benchmark validation running 200,000 agents, Agent Substrate sustained 500 activations per second at less than 500 ms suspend and resume latency.
Security by default: Think of Agent Substrate as zero-trust for agents. Each sandbox has strong hardware isolation built in using your choice of microVM or gVisor and every agent gets its own cryptographically secure identity paired with egress proxies that enforce granular network policies and inject credentials outside the reach of the agents themselves. Looking ahead, we will expand the Agent Substrate runtime architecture to integrate with our broader Gemini for business governance framework.
Open source portability: Runs on any Kubernetes cluster in any compute environment and works with any agent framework or harness, including Claude Code, OpenClaw, and Hermes.
Dynamic storage: Integrates with Filestore agent volumes with sub-second resume latency to scale alongside active compute, providing a single shared workspace for multi-agent workflows.

Leading AI teams like Nous Research are already using this reliable architecture to deploy multi-tenant agents directly into existing customer infrastructure efficiently.

“We built Hermes Enterprise to enable customers to deploy into their existing infrastructure, while handling per-agent isolation and extensible access control. Agent Substrate addresses both at the platform layer in a way that also preserves valuable compute resources. Our experience with Agent Substrate gives us confidence the architecture can scale efficiently as agent workloads grow.” - Hervé Bizira, Chief Business Officer, Nous Research

Expanding governance capabilities for agents everywhere

As organizations transition autonomous agents from isolated pilots to multi-runtime production fleets, traditional security frameworks must be hardened and evolve. Customers face a triple challenge: securing dynamic non-deterministic interaction paths across agents, tools, models, skills and data; maintaining a default deny posture with least-privilege-access distributed runtimes; and ensuring that asynchronous agent actions can be audited. Without strong agent governance, agent sprawl quickly introduces risk — including shadow AI, data exfiltration, and agent actions that are hard to track and audit.

To solve these challenges, we built robust agent governance capabilities into Gemini platform, including:

Agent Gateway: Serves as the air traffic controller for client-to-agent and agents-to-anywhere traffic. It integrates a broad set of governance and security policies including sensitive data protection, agent attack protection, semantic governance, and the ability to bring your own policy.
Agent Registry: Provides a central repository that serves as the source of truth for all approved running agents, tools, MCP servers, skills, models and endpoints, governing runtime access via applied policies.
Agent Identity: Assigns a unique secure ID using open standard based SPIFFE to each agent, acting as the digital signature for authentication, access control, and auditing.
Agent observability: Provides OpenTelemetry (OTel)-compliant telemetry adopting agentic semantic conventions, emitting standardized metrics, logs, and distributed traces for every model call, tool invocation, and agent hop across Agent Gateway.

Now as agent fleets continue to grow dramatically across diverse environments, we are extending the reach of our Gemini for business agent governance capabilities across Google Cloud runtimes, cross-cloud, on-prem and client devices to manage and secure agents everywhere they run. This includes the following capabilities:

Broad Google Cloud runtime coverage: Extends comprehensive agent governance across Agent Runtime, Cloud Run, Agent Sandboxes in Gemini platform, GKE, and Agent Compute Engine (preview).
Consistent cross-cloud reach: Deploys lightweight, Envoy proxy-based policy enforcement points to extend governance to multicloud and hybrid environments (in preview soon).
Governance for local agents on endpoints: Extends governance to coding agents like Claude Code and Antigravity CLI on managed devices with an OS-level sandbox and local governance services (in preview soon).

Additionally we are announcing new agent governance capabilities to Agent Gateway to enable granular cost controls, intelligent model routing, and simplified customizations and integrations. Highlights coming this quarter in preview include:

Granular token and cost governance: Token management provides control over token consumption across agents and models, allowing you to attribute spend, enforce budget guardrails, and provide a single source of truth for token metrics.
Route requests to the right model: Model routing policies route user and agent requests to the right model, controlling costs and improving performance.
Extensible policies: Service Extensions bindings add streamlined support for inline programmability and extensibility. Allows tightly integrating Google services, customers’ bespoke functions, and partner services directly into the datapath.
Expanded security partner ecosystem: Provides delegated user authorization with third-party identity providers Ping Identity, SailPoint, and Saviynt; in addition to secure Oauth using Agent Identity auth manager. Our extensive list of integration partners is here.
Simplified and secure agent-to-tool connectivity: Express Link for Cloud Run allows agents to declare what they need to reach, and it establishes the secure, authenticated connection between components, including identity token injection.

“At L’Oréal, our ambition is to put agentic AI in the hands of our teams, empowering them to bring new ideas to life and shape the future of beauty. Through our Beauty Tech Agentic Platform, AI agents are becoming part of everyday work. Google Cloud’s agent governance provides us the foundation to scale this ambition with confidence, unifying identity, policy and observability across runtimes such as Cloud Run. Our developers can innovate faster while we maintain clear control over what each agent can access.” - Alexandre Frizzo, AI Lead Engineer, L’Oreal

Build on our resilient agent infrastructure

The agentic era requires an evolution in how we scale infrastructure. By combining the high-performance execution of Agent Compute Engine and Agent Substrate with agent governance capabilities from Gemini for business — now expanded with cross-cloud reach — we are establishing a resilient foundation for protected, high-density AI agents.

Learn more about Agent Compute Engine and indicate your interest in Agent Compute Engine.
Help shape Agent Substrate built in the open with the community. Try it on GKE today by visiting its open-source repository.
Learn more about agent governance and indicate your interest.
Come visit us at AGNTCon+MCPCon to speak with experts and see our demos.

Introducing Google Cloud’s U4 compute: Enabling ultra-low latency trading

Wed, 07 Oct 2026 16:00:00 +0000

In capital markets, operational success is built on predictable performance and low latency, where even tiny variations in packet timing alter queue priority and trade execution. At the same time, both exchange venues and their participants face mounting constraints in traditional on-premises data centers — from power and physical rack space limits to lengthy hardware procurement cycles. Increasingly, global capital markets seek the speed and determinism of physical co-location combined with the dynamic scalability and automation of the cloud.

At Google Cloud Next ‘26, we announced the Ultra Low Latency (ULL) Solution, now generally available, providing high-frequency trading workflows that run in the cloud with an ultra-low latency network and agility. The ULL Solution includes the new U4 machine family, also generally available today.

The ULL Solution is based on three infrastructure pillars:

Hardware-level networking: Scaleable, hardware-based multicast data distribution for reliable market-data feeds
Advanced networking observability: Dynamic network traffic capture with hardware-level timing accuracy, facilitating consistent auditing, market replays, and absolute trade validation without impacting primary traffic performance
Deterministic high-performance compute: Processing of latency-critical execution tiers in a highly predictable, consistent amount of time, every single time

Taken together, the ULL Solution’s compute, storage, networking, and observability provide financial exchanges and market participants with a number of technical capabilities:

Bare metal performance: Dedicated bare metal compute provides direct access to physical host resources, minimizing jitter and delivering predictable, low-latency execution for market-data feeds and order routing.
High-performance storage options: Local Titanium SSDs handle real-time transaction and tick logging on the host, complemented by scalable Google Cloud Hyperdisk for persistent market data archives and analytics.
Physical traffic isolation: The ULL trading fabric’s redundant A/B multicast market feeds are accessed through two independent and dedicated Titanium adapters, while an independent third Titanium adapter offloads telemetry, management, and provides access to Google Cloud services.
Hardware-accelerated multicast distribution: The ULL network architecture supports hardware-level multicast feed ingestion. This allows participants to stream high-throughput market data directly to low-latency trading applications, bypassing traditional hypervisor-level virtual switches.
Accelerated packet processing: Support for OpenOnload and DPDK enables Linux user-space networking to deliver predictable unicast and multicast packet handling while minimizing application code changes.
Precision timing and UTC synchronization: Integration with Google Cloud’s Firefly clock synchronization system allows the solution to consistently achieve sub-10 nanosecond network-level timestamping and better synchronization to UTC than the sub-100 microsecond regulatory requirement for financial exchanges.
Built-in telemetry: 24/7 low-latency packet capture and seamless out-of-band packet brokering for regulatory compliance and real-time analytics helps ensure deep visibility without impacting primary traffic performance.
24-7 market ready: The solution is designed for continuous, round-the-clock trading readiness by isolating production workloads in a dedicated primary zone for live trading, while routine cloud maintenance and qualification testing occur in a secondary zone for updates and testing.

Compute in the ULL Solution is delivered by the new U4 machine family, which brings predictable performance and ultra-low latency compute in three specialized machine series: dual-socket bare metal instances with three physical NICs — U4P for exchange operators and U4C for market participants — alongside U4S high-performance VMs for operators, participants, and service providers.

Machine Series	CPU Platform	CPU Cores	Memory	NICs	Storage Options
U4P (Bare Metal)	Intel 5th Gen Xeon (Emerald Rapids)	120 Physical Cores	512 GB or 768 GB	3 Physical (1 Standard + 2 ULL)	Local Titanium SSD (12 TiB), Hyperdisk
U4C (Bare Metal)	Intel 5th Gen Xeon (Emerald Rapids)	120 Physical Cores	512 GB or 768 GB	3 Physical (1 Standard + 2 ULL)	Local Titanium SSD (12 TiB), Hyperdisk
U4S (VM)	Intel 6th Gen Xeon (Granite Rapids)	2 to 288 vCPUs	Up to 2,232 GB	Multi-vNIC (Up to 200 Gbps)	Local Titanium SSD (18 TiB), Hyperdisk

Machine Series

CPU Platform

CPU Cores

Memory

NICs

Storage Options

U4P

(Bare Metal)

Intel 5th Gen Xeon (Emerald Rapids)

120 Physical Cores

512 GB or 768 GB

3 Physical (1 Standard + 2 ULL)

Local Titanium SSD (12 TiB), Hyperdisk

U4C

(Bare Metal)

Intel 5th Gen Xeon (Emerald Rapids)

120 Physical Cores

512 GB or

768 GB

3 Physical (1 Standard + 2 ULL)

Local Titanium SSD (12 TiB), Hyperdisk

U4S

(VM)

Intel 6th Gen Xeon (Granite Rapids)

2 to 288 vCPUs

Up to 2,232 GB

Multi-vNIC (Up to 200 Gbps)

Local Titanium SSD (18 TiB), Hyperdisk

We developed the U4 machine family to enable the world’s most technically demanding markets to run within Google Cloud and benefit from cloud services and scale. An example of this is our ongoing collaboration with CME Group, through which we are migrating listed derivatives markets to Google Cloud. Here, the U4C and U4P bare metal instances deliver direct physical co-location latency parity, providing predictable low-latency clock precision, and native hardware-multicast feed ingestion. This architecture demonstrates that core exchange systems and trading strategies can run in the cloud with the speed, consistency, and control that financial markets require.

"Migrating the world’s leading derivatives marketplace to the cloud requires uncompromising performance, latency, and reliability. Google Cloud’s Ultra Low Latency Solution and the U4 instance families represent a breakthrough for financial market infrastructure. By combining deterministic, ultra-low latency networking and dedicated bare metal compute with the benefits of the cloud, market participants can execute and ingest market data and execute with the speed of traditional co-location and the agility of the cloud." - Pearce Peck-Walden, Managing Director Markets Engineering, CME Group

Powered by Google Cloud-native infrastructure

The ULL Solution is available in select private Google Cloud regions, where trading teams can leverage core cloud infrastructure benefits such as rapid automated resource provisioning, on-demand capacity scaling, and dynamic fleet management. Trading teams can execute ultra-low latency trades on a dedicated, isolated ULL network, and offload data streams over an independent network interface to Google Cloud services such as BigQuery and Gemini Enterprise.

For exchange participants managing tick-to-trade workflows or processing real-time market data, microsecond variations are critical. As bare metal instances, U4P and U4C machine series don’t add any virtualization overhead for latency-sensitive execution tiers. Leveraging Google Cloud's Titanium offload system architecture, these bare metal instances provide direct access to the host resources. Compute Engine instances enable users to provision dedicated hardware through standard Google Cloud APIs, orchestrate deployments with infrastructure as code, apply Cloud Next Generation Firewall (NGFW) policies, and collect rich telemetry.

The U4S VM series complements these bare metal offerings for applications such as pre-trade risk validation and real-time analytics. Deployed alongside U4P and U4C instances, these VMs shorten transit hops across the trading architecture while offering elastic scaling from 2 to 288 vCPUs and networking up to 100 Gbps.

“Our testing of the U4 instances validated that Google Cloud delivers the low latency and determinism required for our demanding exchange trading workloads. Having both bare metal and VM options allows us to evaluate the value of that performance relative to its cost, helping us align the right infrastructure to each of our trading strategies.” - Corbin Kidd, EVP/CTO, Prime Trading

"In our tick-to-trade performance testing, the U4C bare metal instance demonstrated exceptional speed, showing that cloud compute and ultra-low latency networking can comfortably satisfy microsecond trading demands. But, what sets the U4 series apart isn't just raw processing speed, it’s structural determinism. When we stress-tested Google Cloud's U4 bare metal family, the performance curve remained virtually flat, which confirms that these cloud compute layouts can now offer the strict determinism institutional market participants require." - Christopher Wilson, Global Head of Experience Modernization, 28Stone

Looking to the future, this low-latency architecture serves as the foundation for algorithmic trading, allowing firms to feed real-time markets directly into AI-driven trading models.

Storage and placement for trading workloads

The U4 family supports Google Cloud storage options and placement policies, helping trading teams record trades at speed and minimize network latency between machines:

Local Titanium SSD for fast logging: For real-time trade logging and data caching without slowing down execution, U4P and U4C provide up to 12 TiB of direct-attached NVMe SSDs, and U4S VMs scale up to 18 TiB.
Hyperdisk for data archives: For historical market data, backtesting models, and regulatory archives, you can attach Hyperdisk Balanced and Hyperdisk Extreme volumes across all U4 series, scaling up to 32 volumes and 512 TiB per instance.
Placement policies: To reduce physical network hops, compact placement policies keep related machines close together in the data center. Alternatively, spread placement policies place machines on separate racks to improve resiliency.

Getting started with the ULL Solution

With the ULL Solution, financial exchange operators, exchange participants, and trading service providers can execute trades with the speed and reliability they need today while positioning their businesses to leverage continuously evolving AI and infrastructure capabilities for the trading of tomorrow. The ULL Solution is available in select Google Cloud regions. To learn more, check out the public documentation, and reach out to our team to request access.

Networking for AI inference model serving - GKE only and for all other backends

Tue, 06 Oct 2026 12:00:00 +0000

Enterprises and individual developers frequently run multiple AI inference models. The right architecture can simplify how the models are called while also providing centralized governance. In this post, we'll look at two reference architectures focused on networking AI inference model serving: one for Google Kubernetes Engine (GKE) and one all other backend types. First, we'll explore the commonalities between the reference architectures that you'll see later. Then we'll explore unique components of the architecture for GKE backends and finally, we'll go over the elements of the architecture for all backend types.

The entry point

You can expose your model deployment behind a stable, secure, and reliable entry point that acts as the front end for inference calls. This entry point also acts as a control zone where policy, security, and logic can be enforced. Both the Cloud Load balancer and the Inference Gateway provide entry point capability. These types of endpoints can terminate secure connections with TLS, integrate with API management components, extend functionally with service extensions, and capitalize on capabilities of Model Armor for added security.

Common services in the designs

Both reference designs use these services:

Private Service Connect inference endpoint: Anchors the entry point inside your consumer Virtual Private Cloud (VPC) network. Traffic hits a private internal IP address, keeping inference calls in your private network.
Apigee API Management (Optional): Integrates via an Apigee Extension Processor callout to handle client identity verification, rate limits, and quota enforcement before requests ever reach compute resources.
Model Armor: Serves as an inline AI safety checkpoint, screening prompts and output completions against prompt injection and sensitive data leakage.

Design pattern serving on GKE only

This section focuses on a GKE-only backend design. To understand the full end-to-end concept, please read the entire architecture document Networking for AI inference model serving on GKE. The design pattern is based on this diagram:

In addition to the common services identified in the previous section, the design for GKE uses these components:

GKE Inference Gateway: Deployed as an internal Application Load Balancer (gke-l7-rilb). It acts as a specialized ingress engine that parses incoming request payloads, evaluates HTTPRoute rules, and steers queries to appropriate model-serving targets.
Inference pools: A logical group containing replicas of the same model. When the Gateway receives a prompt, it evaluates HTTPRoute rules to select the appropriate inference pool based on the model identifier. Pools have an initial size and can be configured to autoscale dynamically.
Model replica sets: Individual model replicas (inference server instances) deployed across single-node or multi-node GPU or TPU node pools. A replica set represents a uniform group of these model replicas.

The traffic flow GKE example

A client application that uses this GKE-based architecture to call a backend model would go through a flow like this:

Ingress: A client application in the consumer VPC issues an OpenAI-compatible API call to the local Private Service Connect endpoint, routing directly to the GKE Inference gateway using a regional internal Application Load Balancer.
Payload inspection: The Gateway reads the target model parameter specified in the request body and adds it to the HTTP headers.
Control plane validation: If Apigee is used, it checks client credentials and quotas. Model Armor screens the prompt for policy violations or data leakage.
Backend selection: The Gateway evaluates HTTPRoute mappings to identify the target pool, matches shared prefix cache context, and routes to the lowest-load GPU or TPU replica based on real-time Prometheus data.
Egress: The replica runs the inference workload. Output tokens pass through Model Armor for final response verification before streaming back over the private connection.

Design pattern serving on all backends

This section focuses on multiple backend types which can be used for inference, and it provides an overview of the architecture in the following diagram. To understand the full end-to-end concept, please read the entire architecture document Networking for AI inference model serving on all backends.

For architectures spanning mixed environments such as GKE, Cloud Run, Agent Platform, on-premises data centers, or external clouds, these additional components are used:

Regional internal Application Load Balancer: Serves as the central Layer 7 routing proxy that manages routing logic, SSL termination, and Service Extensions callouts.
Inference Payload Processor (Service Extensions): This is similar to body-based routing as used in the GKE Inference Gateway, but to enable the functionality on an Application Load Balancer a service extension is needed. A lightweight Cloud Run callout inspects the JSON body of incoming OpenAI API requests, extracts the target model identifier, and writes an X-Gateway-Model-Name header to drive URL map routing.
Network Endpoint Group (NEG): Delivers flexible routing to heterogeneous backends based on the injected model header.

All-backends traffic flow example

A client application that uses this architecture to call a backend model would go through a flow like this:

Private ingress: The client application targets the Private Service Connect endpoint over private IP address space. The regional internal Application Load Balancer receives the request.
Model name extraction: The load balancer sends the payload to the Cloud Run body-based router callout, which inspects the JSON payload and injects the X-Gateway-Model-Name header.
Policy and safety enforcement: The request passes to Apigee for identity and quota validation, then to Model Armor to scrub sensitive data and block malicious prompts.
NEG routing: The load balancer URL map inspects the model header and forwards the request to the matching backend NEG (Agent Platform, GKE, Cloud Run, Hybrid, or Internet).
Private delivery: The target backend executes the model prompt, Model Armor screens the completion, and the result returns privately along the ingress path.

What's next

Take a deeper dive into building AI workloads on Google Cloud:

Document set: Agentic AI architecture guides
Architecture Center: Multi-agent private networking patterns in Google Cloud
Document: Gemini Enterprise Agent Platform networking access overview

Want to ask a question, find out more, or share a thought? Please connect with me on Linkedin.

How Google Cloud Networking Supports Your Fluid Compute Choices for AI Workloads

Thu, 24 Sep 2026 13:00:00 +0000

The availability of resources for AI workloads can be challenging across the industry, especially accelerators. This can slow your AI workload deployment if it’s built around a specific type of accelerator. The concept of fluid compute allows you to design your AI deployment with several options based on available resources that can fit your use case.

In this blog, we will explore how Google Cloud networking supports your AI workloads and considerations that are relevant to your choice of accelerator (GPU or TPU), as the backend networking component configuration is not exactly the same.

The resource options

After deciding the type of work you want to achieve with your AI deployment, another important component is the actual hardware to get this done. In this case, we want to run inference for a private LLM, and the target is the NVIDIA B200 GPU family which is available in the A4 VMs (a4-highgpu-8g).

Now we have identified what we want to get done and a possible compute option, but the challenge is: is this available?

To get access to resources, there are several options which include:

Dynamic Workload Scheduler (Flex-start VM): Queues workloads until all required accelerator nodes are available at the same time, provisioning them together and running non-preemptibly for up to seven days.
Dynamic Workload Scheduler (calendar mode): Enables reserving accelerator capacity 1 to 90 days in advance with guaranteed start and end times, ideal for scheduled pre-training runs and benchmarking.
Future reservations: Guarantees access to committed hardware in a specified zone beginning at a specific future date.
Flex reservations: Offers short-term commitment windows to secure scarce accelerator nodes without multi-year lock-in.
Dynamic node auto-provisioning and ComputeClasses: In Google Kubernetes Engine (GKE), defining multi-family fallback lists within ComputeClasses allows the cluster to automatically attempt provisioning alternative accelerator types if primary pools face regional constraints.
Spot VMs: Delivers surplus compute at substantial discounts for fault-tolerant, checkpointed batch jobs.

Networking your choices

The networking component of the accelerator varies based on your choice, so let's explore four configurations: standard networking, accelerated GPU networking (TCPX/TCPXO and RoCEv2), TPU networking, and Cloud Run.

Standard networking

Supported accelerators: NVIDIA T4 (N1 series), NVIDIA L4 (G2 series), NVIDIA A100 (A2 machine series single-node and multi-node), Cloud TPU v3, and Cloud TPU v5e (single-host/standalone slices).
Architecture: Nodes communicate over the primary Virtual Private Cloud (VPC) network using the Google Virtual NIC (gVNIC) over standard TCP/IP.
Workload fit: Provides straightforward portability across Google Cloud compute environments, supporting distributed data preprocessing, decoupled pipeline stages, independent inference replicas, and computer vision workloads using standard VPC routing and network policies.

Accelerated GPU Networking (TCPX/TCPXO and RoCEv2)

Distributed training and multi-node inference require specialized multi-rail network fabrics to handle massive parameter exchanges and collective communications.

GPUDirect-TCPX and TCPXO Fabrics

Supported accelerators: NVIDIA H100 (A3 High VMs with 4 rails) and NVIDIA H100 Mega (A3 Mega VMs with 8 rails).
Architecture: Uses custom GPUDirect-TCPX (4 dedicated VPCs) and GPUDirect-TCPXO (8 dedicated VPCs) offload engines to achieve high-throughput multi-rail GPU communication over standard Ethernet infrastructure without requiring native RDMA hardware.
Deployment blueprints: These multi-VPC topologies can be deployed in many ways including using pre-built blueprints from the Cluster Toolkit.

RoCEv2 Fabrics (VM and Bare Metal)

Supported accelerators: NVIDIA H200 (A3 Ultra VMs), NVIDIA B200 (A4 VMs), NVIDIA GB200 NVL72 (A4X VMs), and NVIDIA GB300 (A4X Max Bare Metal).
Zonal network profiles: RoCEv2 operates over a dedicated RDMA VPC attached to a specialized zonal network profile: VM instances (A3 Ultra, A4, A4X) use the ZONE-vpc-roce profile, while Bare Metal instances (such as A4X Max) utilize the dedicated ZONE-vpc-roce-metal bare-metal profile.
Rail-aligned fabrics: This dedicated VPC is isolated strictly for GPU communication and contains subnets mapped directly to the accelerator NICs. The backend is rail-aligned, with support for Jumbo Frames (MTU 8896), delivering non-blocking multi-terabit bandwidth with minimal cross-rail interference.
Automated plumbing with GKE Dynamic Resource Allocation Network (DRANET): When deploying these GPUs on GKE, the GKE managed DRANET can be used to automatically provision additional networks and assign drivers that map the RDMA network interfaces to the GPU. These can then be assigned and consumed directly in your workload pods using standard Kubernetes resource claims.
Turnkey deployment: You can deploy this entire end-to-end stack—including RDMA VPCs, MTU tuning, and DRA drivers—using automated blueprints from the Cluster Toolkit.

TPU Networking

Supported accelerators: Cloud TPU v4, Cloud TPU v5p, Cloud TPU v5e (multi-host Pod slices), Cloud TPU v6e (Trillium), and TPU7x (Ironwood).
Inter-chip interconnect (ICI): Inside a TPU Pod or slice, chips communicate directly over dedicated, ultra-low-latency optical links organized in 2D or 3D torus meshes, bypassing traditional network stacks entirely.
Optical circuit switches (OCS): In TPU v4 and TPU v5p SuperPods, software-reconfigurable OCS units dynamically change physical network topologies, route around faulty trays, and provision custom-sized accelerator slices without manual recabling.
Multi-NIC architecture (TPU v6e and Higher): While earlier TPU generations relied on ICI within a slice and single-NIC for host traffic, Cloud TPU v6e (Trillium) and TPU7x introduce a native multi-NIC architecture where worker nodes isolate standard Kubernetes management traffic onto a primary VPC while using secondary dedicated VPCs configured for high-throughput TPU data and cross-slice communication.
DRANET for TPU deployments: When deploying these TPUs on GKE, the GKE managed DRANET can be used to automatically provision additional networks and assign drivers for TPU communication. These can then be assigned and consumed directly in your workload pods using standard Kubernetes resource claims.
Data-center network (DCN) Multislice: For models scaling beyond an individual TPU slice, Cloud TPU Multislice connects multiple independent ICI meshes over Google's high-speed Jupiter Data Center Network utilizing these dedicated multi-NIC paths.

Cloud Run

Supported accelerators: NVIDIA L4 (G2 series) and NVIDIA RTX PRO 6000 (Blackwell) on Cloud Run GPU services.
Direct VPC egress: Binds serverless containers directly to your private VPC network using sub-minute IP allocation via Direct VPC Egress, enabling secure, low-latency access to internal data lakes, databases, and private APIs without traversing the public internet or requiring legacy connector VMs.

Summary

Google Cloud networking options support various accelerator types. When using fluid compute you can adjust your network setup to support the best design to optimise your workloads performance.

Next Steps

Take a deeper dive into Google Cloud AI infrastructure and networking architectures with these resources:

Want to ask a question, find out more, or share a thought? Please connect with me on LinkedIn.

Simplify your resilience testing strategy with Fault Injection Testing

Wed, 26 Aug 2026 16:00:00 +0000

When databases fail and network paths falter, you still need your mission-critical cloud services to stay online. Yet guaranteeing high availability has become increasingly difficult because of the complexity of modern distributed systems.

To help you maintain availability and reliability during adverse events, we’re announcing Fault Injection Testing in preview. Fault Injection Testing is designed to help developers and architects automate failure testing to ensure predictable behavior during disruptions.

By deliberately introducing faults into your environment, you can verify your safety mechanisms before an actual outage impacts your customers.

Why native resilience testing matters

Unlike in self-hosted data centers, cloud applications offer less direct access to underlying infrastructure to facilitate failover testing.

Without native tools to prove your application can survive a failure, you risk a critical gap in your reliability strategy that exposes you to several risks:

Damaged trust and reputation: Frequent failures or poor performance lead to customer dissatisfaction and long-term damage to your brand's image.
Compliance and regulatory penalties: For many industries, particularly financial institutions, failing to prove disaster recovery capabilities can lead to non-compliance, audits, and fines.
Migration delays: Large-scale migrations often stop when teams cannot verify that critical applications will remain stable during a zone failure.

How Fault Injection Testing works

Fault Injection Testing allows you to run experiments by creating experiment templates. These templates act as blueprints, defining the specific fault to be injected and the resources that will be targeted for the experiment.

In this public preview, you can test two primary failure scenarios:

Failover Cloud SQL: This fault triggers a failover of a high availability Cloud SQL instance from the primary zone to a standby zone.
Degrade application traffic: This allows you to selectively add latency and HTTP error codes through an Application Load Balancer.

Before any fault is injected, Fault Injection Testing performs an automated dry run. This read-only simulation checks your permissions and provides an up-to-date list of every resource that will be affected.

Once you verify the scope, you can manually start the injection. The duration you defined in the template will run its course, and the faults will be reverted at the expiration of the timer.

During the experiment, you can verify that your application is behaving as you planned. If things do not go as planned, you can use the stop and revert capability to immediately halt the experiment and begin restoring resources to their normal state.

During preview, we recommend as a best practice to use Fault Injection Testing (FIT) in a non-production environment. Preview is an opportunity to get early access to learn how the service fits and complements your existing testing practices, and to provide us with your feedback to improve the product as well!

Built for the enterprise

Partners like KeyBank and Servier are already using Fault Injection Testing to validate their deployments. By using native fault injection, these organizations can approximate demanding failure scenarios — such as zonal outages — to help ensure their services remain stable.

Get started with Fault Injection Testing

Fault Injection Testing is available through the Google Cloud console, the gcloud CLI, and REST APIs.

Request preview access: Talk to your Google Cloud Account Team to add your project to the preview.
Enable the API: Search for "Fault Testing API" in your Google Cloud console and select enable.
Assign roles: Ensure your team has the roles/faulttesting.operator role to configure and run experiments.
Run your first dry run: Create a template for a Cloud SQL or load balancer resource in a non-production environment and execute a dry run to see the potential impact.

For more details on implementation, talk to your account team, or view the User Guide for Fault Injection Testing.

How Uber improves network reliability while unblocking cloud migration

Wed, 26 Aug 2026 16:00:00 +0000

Uber has a lot in common with the cities it serves. Both are always changing and growing, both must carefully manage the resulting traffic to prevent congestion and sprawl.

Uber has continuously evolved its technical strategies to manage its expanding network, and this careful planning and constant evolution helps ensure that application traffic across its entire platform runs smoothly. Ultimately, maintaining a reliable, high-scale platform that operates seamlessly at any given time is key to preserving user trust.

One important solution in this effort has been application awareness on Cloud Interconnect. An industry-first tool for application prioritization across hybrid networks, application awareness on Cloud Interconnect has helped Uber prioritize critical traffic to ensure business continuity during potential network congestion events.

Uber acted as an early design partner for application awareness on Cloud Interconnect, helping ensure that this capability met the demands of Uber’s global-scale operations. It not only improved Uber’s daily operations, it also gave Uber the confidence to move forward with a Google Cloud migration, with confidence that there would be less risk of service interruptions during switchovers.

In this post, we’ll explain the features Uber most sought and why, the inner workings of application awareness on Cloud Interconnect, and how it can help other organizations as well.

Prioritizing critical traffic

When migrating distributed, hybrid, or multicloud applications at a global scale, network reliability becomes a primary concern. Even the most worthwhile migrations may not seem worth it if such migrations interrupt ongoing service. For organizations like Uber, moving vast amounts of data to support large data analytics workload — including emerging AI use cases — can saturate network links, resulting in increased reliability risk for their critical application traffic.

With standard cloud interconnect approaches, enterprises typically apply simple bandwidth overprovisioning to meet extreme infrastructure needs. But with today's hybrid cloud demands, and given the size of an organization like Uber, overprovisioning network capacity for peak usage is often too costly and unreliable.

The shortcomings of overprovisioning only become magnified with the integration of cutting-edge AI innovations. Uber needs systems in place that can take on massive data transfers without congesting its network and protecting the performance of business-critical applications.

With the benefit of application awareness on Cloud Interconnect, including the four major features of application awareness — traffic handling, congestion response, latency management, and cost efficiency — Uber was able to achieve the networking optimization its modern tech stack requires.

Starting with a private preview, Uber deployed this feature across its infrastructure, beginning with Google Cloud Interconnect deployments in Phoenix, Arizona, and Ashburn, Virginia. Application awareness on Cloud Interconnect allows Uber to classify and prioritize end-user application traffic over less time-sensitive data using DSCP marking and configured queuing profiles.

In the following chart, we look at the four key features of application awareness on Cloud Interconnect, how they differ from legacy approaches, and how they help provide better operational continuity for organizations like Uber.

Feature	Standard interconnect solutions	Application awareness on Cloud Interconnect
Traffic handling	All traffic treated equally (first-in, first-out)	Traffic classified into six distinct traffic classes
Congestion response	High-priority application traffic may be dropped during bursts	Business-critical traffic is protected via strict priority or bandwidth sharing policies
Latency management	Unpredictable latency for high priority applications	Predictable and consistent low-latency for time-sensitive workloads
Cost efficiency	Requires expensive overprovisioning to absorb peaks	Efficient bandwidth utilization and lower TCO

Uber's key takeaways

For Uber, the business value of being able to prioritize business-critical traffic on its networks by deploying application awareness on Cloud Interconnect was immediate. And in doing so, Uber has also created a blueprint that other enterprises with similar hybrid cloud challenges can replicate. The core elements of that blueprint include:

Ensuring business continuity: Uber can decide in real time which application traffic to prioritize during major, high-traffic events. This means that mission critical applications stay up and running during even extreme events (both planned and unplanned). Uber leadership has called application awareness on Cloud Interconnect important for its global operations.
Efficient bandwidth utilization: Instead of blindly overprovisioning bandwidth to prevent congestion, application awareness allows Uber to better utilize their existing Cloud Interconnect capacity aligned with their expected network bandwidth needs. The result is lower total cost of ownership for network infrastructure.
Unblocked workload migration: By protecting critical applications from network congestion, Uber was able to migrate significant workloads to Google Cloud and, in the process, dramatically reduce operational overhead.

"Application awareness on Cloud Interconnect was the key that unlocked our ability to migrate more strategic workloads to Google Cloud and is critical for maintaining service reliability during peak global demand. By allowing us to intelligently prioritize traffic, it helps us ensure that we can protect our higher priority services and make our infrastructure more efficient, lowering our total cost of ownership. This wasn't just a feature deployment; it was a deep engineering partnership that delivered a solution critical to our business." – Harry Liu, Director of Engineering, Uber

Securing network reliability for AI and beyond

As more enterprises integrate cloud-based AI models, distributed applications, and data analytics, it's becoming a business imperative to be ready to handle the massive data transfers that follow. But in doing so, they also have to ensure they never compromise the reliability of their critical applications.

With application awareness on Cloud Interconnect, Uber demonstrated that moving beyond simple bandwidth overprovisioning to protect business-critical traffic was an essential step to building the stability required to embrace modern hybrid and multicloud strategies.

You can read our blog about the potential of Cloud Interconnect across industries to learn more about what the service can bring to your organization, and if you’re ready to explore more, our team of networking and industry experts are ready to help.

ClusterNetworkPolicy in GKE: Balancing control and autonomy for your microservices

Mon, 10 Aug 2026 16:00:00 +0000

Managing network security in a multi-tenant Kubernetes environment typically requires balancing two distinct needs: developers need their microservices to communicate effectively, while platform and security teams must maintain compliance, prevent lateral movement, and establish cluster-wide guardrails.

Historically, the standard Kubernetes NetworkPolicy has been the primary tool for this. While effective for single-namespace isolation, standard NetworkPolicy is scoped strictly to individual namespaces and designed around developer self-service. When cluster administrators attempt to use it for global security enforcement, it can lead to policy conflicts and operational challenges.

To address this, we introduced ClusterNetworkPolicy (CNP), an open-source standard developed by the Kubernetes SIG-Policy Working Group (WG), to Google Kubernetes Engine (GKE). Designed for scale, CNP is a cluster-wide resource that allows administrators to manage network security centrally, providing a mechanism for those responsible for global security to implement consistent, non-bypassable policies.

Read on for technical details about CNP, some common use cases, an example policy, and how to get started.

Structuring policies with tiers

A core capability of ClusterNetworkPolicy is its hierarchical tier system. Rather than attempting to reconcile flat, conflicting peer rules simultaneously, CNP establishes a deterministic, top-to-bottom evaluation hierarchy:

The admin tier: The highest precedence level. Rules here are enforced before any other policies.
The network policy tier: The standard namespace level, where developers manage their specific application policies.
The baseline tier: The lowest precedence, establishing the cluster’s default behavior when no other policies apply. This can be overridden using namespace scoped policies.

This tiered structure helps align network security with organizational roles. Using standard role-based access control (RBAC), you can manage the admin tier to enforce compliance mandates, while platform teams can use the baseline tier to set a default "deny-all" zero-trust posture across the cluster. At the same time, developers can write standard network policies for their applications without overriding core security mandates.

This deterministic, top-to-bottom evaluation method resolves conflicts between different teams' policies. The admin tier introduces an explicit Pass action. This allows security teams to inspect traffic against global rules and then delegate the final Accept or Deny decision down to the developer's namespace policy, facilitating both central oversight and distributed management.

Common network security scenarios

This tiered architecture translates complex security requirements into centralized rules. Here are common scenarios where ClusterNetworkPolicy provides a practical solution:

Isolating sensitive workloads: You can apply an admin-tier global deny rule to isolate specific namespaces — such as those used for payment processing or compliance data — from the rest of the cluster. This action overrides any permissive developer policies that might otherwise expose these environments.
Protecting core services: To prevent configurations that might disrupt internal operations, administrators can create an admin-tier global allow rule for critical services like kube-dns. This allows these services to remain accessible regardless of any misconfigured namespace policies.
Managing external egress: By utilizing IP address range matching, egress traffic can be controlled at the cluster level. This functionality allows you to explicitly restrict or permit access to corporate intranets or external IP ranges, serving as a safeguard against unauthorized data exfiltration.

Example scenario

Consider a common enterprise requirement: Application workloads across all namespaces must be permitted to reach central platform infrastructure (such as shared authentication and telemetry services), while access to sensitive environments — like a restricted vault namespace — is strictly prohibited. Meanwhile, routine microservice traffic is delegated to developer-managed, namespace-scoped policies.

ClusterNetworkPolicy makes this straightforward. A platform administrator simply defines an admin-tier guardrail centrally:

code_block: <ListValue: [StructValue([('code', 'apiVersion: policy.networking.k8s.io/v1alpha2\r\nkind: ClusterNetworkPolicy\r\nmetadata:\r\n name: platform-isolation-guardrail\r\nspec:\r\n tier: Admin\r\n priority: 10\r\n subject:\r\n # Target all application tenant namespaces, excluding system and core infrastructure\r\n namespaces:\r\n matchExpressions:\r\n - key: kubernetes.io/metadata.name\r\n operator: NotIn\r\n values: ["kube-system", "shared-services", "restricted-vault"]\r\n egress:\r\n # 1. Mandate access to central shared platform services\r\n - name: allow-shared-services\r\n action: Accept\r\n to:\r\n - namespaces:\r\n matchLabels:\r\n kubernetes.io/metadata.name: shared-services\r\n\r\n # 2. Enforce strict block on accessing the restricted vault namespace\r\n - name: block-restricted-vault\r\n action: Deny\r\n to:\r\n - namespaces:\r\n matchLabels:\r\n kubernetes.io/metadata.name: restricted-vault\r\n\r\n # 3. Explicitly delegate all remaining traffic to developer namespace policies\r\n - name: delegate-remaining-egress\r\n action: Pass\r\n to:\r\n - namespaces: {}\r\n - networks:\r\n - 0.0.0.0/0\r\n - ::/0'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbd7ecf10>)])]>

Extending open-source foundations

Instead of building this functionality as proprietary extensions, we worked with the Kubernetes community to design the ClusterNetworkPolicy API (policy.networking.k8s.io), distinguishing it from the namespace-scoped NetworkPolicy API (networking.k8s.io). Furthermore, we collaborated closely with the Cilium community to build its implementation of the API.

Because it is built on open-source standards, GKE helps ensure that security configurations remain portable across different environments. The ClusterNetworkPolicy API natively supports tier selection, enabling clear and deterministic policy evaluation. This approach lets administrators enforce robust security guardrails while maintaining the operational flexibility that development teams depend on.

ClusterNetworkPolicy on GKE elevates workload network security — shifting operations from namespace-scoped rules to unified, cluster-wide governance. It is currently in preview in version 1.36 and later. To learn more and get started, check out:

GOL! How TelevisaUnivision streamed the FIFA World Cup to millions with Google Cloud

Fri, 07 Aug 2026 16:00:00 +0000

Live sports broadcasting represents the ultimate stress test for digital media infrastructure, where operational success or failure is measured in milliseconds and observed live by millions of viewers simultaneously. During the 2026 FIFA World Cup, the stakes reached a high for TelevisaUnivision, the leading Spanish-language media conglomerate. With Mexico serving as both a primary host nation and a core contender on home soil, fan engagement created unprecedented demand across Latin America and TelevisaUnivision's ViX streaming platform.

For a marquee broadcaster like TelevisaUnivision, high-stakes events carry direct, long-term brand and reputational implications. Audiences demand uninterrupted, pristine access to every critical moment of play. Playback interruptions during a key goal, login latency at kickoff, or degraded stream resolutions immediately impact customer satisfaction, risking subscriber churn and brand dilution. When streaming tier-1 global sports events, technical execution directly impacts consumer trust, requiring an absolute commitment to zero-downtime availability and flawless performance on the part of the provider.

Why TelevisaUnivision selected Google Cloud

Navigating Latin America's complex networking ecosystem, which is marked by heavy ISP fragmentation and cross-border transit bottlenecks, required more than a standard vendor relationship. TelevisaUnivision needed a strategic partner willing to make joint investments in network capacity, infrastructure resiliency, and custom feature development. TelevisaUnivision selected Google Cloud's Media CDN based on two foundational differentiators: its architecture, and Google Cloud’s customer focus.

Platform architecture

Media CDN provided a resilient, globally distributed infrastructure built specifically to absorb massive live-stream traffic spikes while protecting origin infrastructure. Core architectural advantages included:

In-ISP deep edge caching: Media CDN embedded cache nodes deep within local ISP networks across Mexico and Central and South America, placing video segments within a single network hop of viewers.
Direct ISP peering: By establishing direct peering connections with major regional telecommunications operators such as América Móvil and Telefônica, the architecture completely bypassed congested international transit routes.
Dedicated capacity reservations: TelevisaUnivision reserved live event capacity with dedicated allocated headroom in-region. This isolated livestream traffic from "noisy neighbor" risks and comfortably absorbed peak traffic surges.
Sub-millisecond sessions with Valkey 9.0: To handle massive traffic spikes during the World Cup, TelevisaUnivision migrated its session store to Memorystore for Valkey 9.0, operating as serverless microservices on edge compute. This architecture delivered sub-millisecond response times for critical authentication and entitlement checks, while providing automatic scaling to process peak API traffic without the need for manual capacity reservations.

Obsession for customer success

Beyond technical capabilities, TelevisaUnivision chose Google Cloud for its joint co-engineering model and deep operational alignment.

Joint 24/7 war rooms: For all 104 matches, TelevisaUnivision engineers and Google Cloud specialists operated side-by-side in unified command centers.
Proactive monitoring as a service: Google Cloud’s Customer Reliability Engineering teams provided round-the-clock proactive monitoring and automated alerting.
Joint operational authority: Combined teams performed extensive pre-tournament stress tests and simulated failovers. During live matches, unified telemetry empowered joint leads to dynamically route traffic and adjust CDN configurations instantly as regional ISP congestion emerged.

Summary

The strategic partnership between TelevisaUnivision and Google Cloud during the 2026 FIFA World Cup established a new benchmark for global sports broadcasting. Across 39 consecutive days of tournament execution, TelevisaUnivision reported that the joint infrastructure delivered:

Total matches broadcast	104 live matches
Platform availability	100% platform availability (0 downtime)
Cumulative viewership	675 million views across TelevisaUnivision & ViX

By uniting localized edge delivery, sub-millisecond serverless compute, and dedicated operational co-engineering, TelevisaUnivision and Google Cloud solidified a battle-tested blueprint for executing marquee live streaming events at record global scale.

IDC: Why the right networking approach is foundational to agentic AI

Wed, 15 Jul 2026 16:00:00 +0000

Editor’s note: Today we hear from IDC on the results of its 2026 AI in Networking Special Report Survey exploring the enterprises' concerns about networking infrastructure to support the rise of agentic AI in their organizations. The survey was sponsored by Google Cloud.

Enterprises are moving quickly on AI pilots, but the move from pilot to production remains uneven. While AI models remain important, IDC research indicates that the pilot-to-production bottleneck is primarily infrastructure-centric, with core networking concerns emerging as one of the leading drivers of AI project delays and abandonment. In IDC's 2026 AI in Networking Special Report Survey:

32.6% of respondents cite security concerns: As AI workflows become more distributed and autonomous, enforcing consistent security and governance becomes more difficult.
26.8% of respondents cite challenges in automation: Manual operations and fragmented controls can slow deployment and make AI environments harder to scale.
24.7% of respondents cite staff time and talent restrictions: Limited skills and operational bandwidth can constrain an organization's ability to move AI initiatives into production.

Agentic AI specifically heightens these concerns by introducing more distributed and dynamic interactions across applications, services, APIs, tools, and data sources. In production environments, these interactions often span different agent frameworks, model providers, clouds, open-source tools, SaaS APIs, and internal applications, expanding both the operational scope and the security and governance surface area.

Networking for operational control, security, and governance at scale

Networking is the primary enabler of agentic interactions and plays a foundational role for intracloud and intercloud network- and services-layer connectivity, end-to-end security, and consistent governance. In agentic systems, networking increasingly extends into tighter service-centric controls that govern how distributed services identify one another, communicate, and exchange data securely. While AI workloads in general are increasing east-west traffic demands, agentic AI adds an additional layer of complexity by creating dynamic interactions that require tighter policy, visibility, and control closer to the application workflow.

From an infrastructure perspective, networking is much more than just a connectivity function. It is part of the infrastructure platform control plane that applies policy-based controls, supports observability, and helps maintain consistent security and governance across an AI agent's activity. This is significant because framework-level controls alone become insufficient in environments where agents and services span different runtimes, clouds, deployment models, and operating domains.

That is why an infrastructure-level approach becomes key. It does not replace application frameworks or orchestration environments, but it provides broader and more consistent policy implementation across a complex architectural landscape. As agentic AI becomes more autonomous and distributed, organizations need these controls built in as part of the infrastructure to reduce fragmented observability, inconsistent policy application, and unmanaged shadow agent activities. From a cloud infrastructure standpoint, this is where cloud network services become strategically important.

Balancing act: A platform vs. best-of-breed approach

Agentic AI systems are inherently fragmented because of underlying distributed workflows. Enterprises are already navigating a rapidly evolving landscape of business requirements, open-source components, emerging protocol standards, and new architecture patterns. In this context, choices between best-of-breed point solutions and platform-based approaches should be strategic rather than ideological.

Best-of-breed capabilities may be necessary to address specific technical requirements. But it is also true that point solutions introduced across a distributed agentic AI landscape can create inconsistent policies, operational complexity, and governance gaps. IDC research reflects this tension. In IDC’s 2026 AI in Networking Special Report Survey, organizations remained divided between platform and best-of-breed preferences for AI workloads; among respondents who favored platforms, the main reasons cited were stronger security (32.9%), reduced complexity (27.7%), and faster deployment (24.2%).

In IDC's view, a balance is important. Platforms can provide a consistent operational and policy foundation for AI deployments, but at the same time, they need to be modular and extensible to allow the inclusion of best-of-breed functionality as part of the platform toolset. The right platform for agentic AI should be open, flexible, and able to evolve. It should support integration with third-party and open-source tools, allow insertions of needed security and observability functions, and adapt without complete architectural rework.

This is a period of technology disruption. Businesses must meet their AI objectives while carefully managing dynamic agentic AI systems. In this environment, networking not only remains a connectivity piece of the AI infrastructure but becomes foundational to how organizations establish operational control, apply policy consistently, and maintain end-to-end trust across agentic workflows.

As agentic AI systems continue to evolve, the demands they place are unlikely to be addressed through best-of-breed point solutions alone. Operationalizing agentic AI at scale will require organizations to leverage the right networking approach, supported by infrastructure platforms that are open, flexible, and extensible, enabling a cohesive and adaptable security and governance framework.

Message from the sponsor
The autonomous and non-deterministic communications of agentic applications pose challenges for which the infrastructure and governance models of the cloud-native era are not prepared. In the agent-native era, an infrastructure-led approach is required to enable agentic applications at scale in production with effective governance and observability. An extensible platform based on open standards is critical in enabling the agentic journey today and through its maturity. Learn about the infrastructure imperatives and open standards that make a viable agentic infrastructure here.

Autopilot Clusters with GKE managed DRANET: GPUs and TPUs

Thu, 09 Jul 2026 07:00:00 +0000

Google Kubernetes Engine (GKE) managed DRANET supports both GPUs and TPUs. There are several configurations to use this implementation, including standard cluster (where you have full control) and autopilot cluster (where Google does the heavy configs for you). I've been exploring the capabilities and in this blog we will explore setting up for autopilot clusters.

Autopilot and managed DRANET

GKE autopilot is a managed version of GKE that handles nodes, scaling, security, and other preconfigured settings. GKE managed DRANET lets you request and allocate networking resources for your Pods, including network interfaces that support TPUs and Remote Direct Memory Access (RDMA).

Setup flow

To deploy your GKE autopilot cluster and enable managed DRANET, you need to create a Virtual Private Cloud (VPC). Let's walk through the setup:

Deploy an Autopilot cluster.
Create a custom ComputeClass which supports the accelerator type (TPU or GPU)
Create a ResourceClaimTemplate for GPUs (RDMA) or non-GPU (TPU)
Deploy workload and reference the ComputeClass and ResourceClaimTemplate to get the correct networking set up.

Now let's explore the configs for both TPU and GPU.

Configure variables:

code_block: <ListValue: [StructValue([('code', 'export PROJECT_ID=$(gcloud config get project) #automatically sets your Project_ID\r\nexport REGION="REGION"\r\nexport CLUSTER_NAME="CLUSTER_NAME"\r\nexport NETWORK="NETWORK"\r\nexport SUBNETWORK="SUBNETWORK"\r\nexport RESERVATION_URL="RESERVATION_URL"\r\nexport HF_TOKEN="HUGGING_FACE_TOKEN"'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbdb09650>)])]>

Replace the following:

REGION: The region where you want to create your cluster, such as us-east1. You can only create the cluster in the region where your reservation or resources exists.
CLUSTER_NAME: A name for your cluster, such as dranet-cluster.
NETWORK: The name of the VPC network.
SUBNETWORK: The name of the subnet in the VPC.
RESERVATION_URL: The URL of the reservation that you want to use to create your resources.
HUGGING_FACE_TOKEN: The Hugging Face access token to download your model.

1. Deploy an Autopilot cluster

Deploy an Autopilot cluster.

code_block: <ListValue: [StructValue([('code', 'gcloud container clusters create-auto $CLUSTER_NAME \\\r\n --project=$PROJECT_ID \\\r\n --region=$REGION \\\r\n --release-channel=rapid \\\r\n --network=$NETWORK \\\r\n --subnetwork=$SUBNETWORK'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbdb08f90>)])]>

2. Create a custom ComputeClass

Example: GPU B200 custom ComputeClass with managed DRANET support and a reservation.

code_block: <ListValue: [StructValue([('code', 'apiVersion: cloud.google.com/v1\r\nkind: ComputeClass\r\nmetadata:\r\n name: dranet-a4-computeclass\r\nspec:\r\n nodePoolAutoCreation:\r\n enabled: true\r\n nodePoolConfig:\r\n dra:\r\n networking:\r\n enabled: true\r\n priorities:\r\n - machineType: a4-highgpu-8g\r\n gpu:\r\n count: 8\r\n type: nvidia-b200\r\n acceleratorNetworkProfile: auto\r\n reservations:\r\n affinity: Specific\r\n specific:\r\n - name: ${RESERVATION_URL}\r\n project: ${PROJECT_ID}'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbdb08910>)])]>

Replace the following:

${RESERVATION} : With the URL of the reservation that you want to use to create your resources.
${PROJECT_ID}: With the ID of the project you are using.

Alternatively you can set the variables in your terminal and use the following command to pass the variables at creation envsubst < filename.yaml | kubectl apply -f -

Example: TPU v6e custom ComputeClass using on-demand example.

code_block: <ListValue: [StructValue([('code', 'apiVersion: cloud.google.com/v1\r\nkind: ComputeClass\r\nmetadata:\r\n name: dra-gke-auto\r\nspec:\r\n nodePoolAutoCreation:\r\n enabled: true\r\n nodePoolConfig:\r\n dra:\r\n networking:\r\n enabled: true\r\n priorities:\r\n - tpu:\r\n type: tpu-v6e-slice\r\n count: 8\r\n topology: "2x4" \r\n acceleratorNetworkProfile: auto\r\n location:\r\n zones: \r\n - us-east5-b'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbdb0a090>)])]>

3. Create a ResourceClaimTemplate

RDMA support deviceClassName: mrdma.google.com ResourceClaimTemplate example for GPUs:

code_block: <ListValue: [StructValue([('code', 'apiVersion: resource.k8s.io/v1\r\nkind: ResourceClaimTemplate\r\nmetadata:\r\n name: all-mrdma\r\nspec:\r\n spec:\r\n devices:\r\n requests:\r\n - name: req-mrdma\r\n exactly:\r\n deviceClassName: mrdma.google.com\r\n allocationMode: All'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbdb0a990>)])]>

Non-RDMA deviceClassName: netdev.google.com ResourceClaimTemplate example for TPUs.

code_block: <ListValue: [StructValue([('code', 'apiVersion: resource.k8s.io/v1\r\nkind: ResourceClaimTemplate\r\nmetadata:\r\n name: all-netdev\r\nspec:\r\n spec:\r\n devices:\r\n requests:\r\n - name: req-netdev\r\n exactly:\r\n deviceClassName: netdev.google.com\r\n allocationMode: All'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbdb0aad0>)])]>

4. Deploy workload and reference ComputeClass and ResourceClaim

Create a secret in your cluster

code_block: <ListValue: [StructValue([('code', 'kubectl create secret generic hf-secret \\\r\n --from-literal=hf_token=${HF_TOKEN}'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbdb08890>)])]>

Example deploying GPUs

code_block: <ListValue: [StructValue([('code', 'apiVersion: apps/v1\r\nkind: Deployment\r\nmetadata:\r\n name: gemma-4-31-deploy\r\nspec:\r\n replicas: 2\r\n selector:\r\n matchLabels:\r\n app: gemma4\r\n template:\r\n metadata:\r\n labels:\r\n app: gemma4\r\n ai.gke.io/model: gemma-4-31b\r\n ai.gke.io/inference-server: vllm\r\n spec:\r\n resourceClaims:\r\n - name: rdma-claim \r\n resourceClaimTemplateName: all-mrdma\r\n containers:\r\n - name: vllm-inference\r\n image: us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/pytorch-vllm-serve:gemma4\r\n resources:\r\n requests:\r\n cpu: "10"\r\n memory: "1000Gi"\r\n ephemeral-storage: "1Ti"\r\n nvidia.com/gpu: "8"\r\n limits:\r\n cpu: "10"\r\n memory: "1000Gi"\r\n ephemeral-storage: "1Ti"\r\n nvidia.com/gpu: "8"\r\n claims:\r\n - name: rdma-claim\r\n command: ["python3", "-m", "vllm.entrypoints.openai.api_server"]\r\n args:\r\n - --model=$(MODEL_ID)\r\n - --tensor-parallel-size=8\r\n - --host=0.0.0.0\r\n - --port=8000\r\n - --max-model-len=131072\r\n - --max-num-seqs=16\r\n - --enable-chunked-prefill\r\n - --gpu-memory-utilization=0.90\r\n env:\r\n - name: MODEL_ID\r\n value: google/gemma-4-31B\r\n - name: HUGGING_FACE_HUB_TOKEN\r\n valueFrom:\r\n secretKeyRef:\r\n name: hf-secret\r\n key: hf_token\r\n volumeMounts:\r\n - mountPath: /dev/shm\r\n name: dshm\r\n startupProbe:\r\n httpGet:\r\n path: /health\r\n port: 8000\r\n failureThreshold: 240\r\n periodSeconds: 10\r\n livenessProbe:\r\n httpGet:\r\n path: /health\r\n port: 8000\r\n periodSeconds: 10\r\n readinessProbe:\r\n httpGet:\r\n path: /health\r\n port: 8000\r\n periodSeconds: 5\r\n volumes:\r\n - name: dshm\r\n emptyDir:\r\n medium: Memory\r\n nodeSelector:\r\n cloud.google.com/compute-class: dranet-a4-computeclass'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbdb09b50>)])]>

Notice how the deployment references the ResourceClaimTemplate and ComputeClass. When this kicks off, it triggers a scale-up operation. GKE Autopilot reads the ComputeClass to provision the specific node type and to configure managed DRANET networking. Meanwhile, the resource claim acts as the bridge, binding your Pods directly to the accelerators on those nodes. This process works exactly the same for TPUs.

Next Steps

Take a deeper dive into GKE managed DRANET and autopilot with these resources:

Hands-on Lab: GKE Autopilot clusters with TPUs, GKE managed DRANET and Gemma 4
Document set: DRANET
Documentation: AI Hypercomputer

Want to ask a question, find out more, or share a thought? Please connect with me on Linkedin.

C4N, now GA: Delivering cloud’s highest per vCPU network and block storage I/O for x86 workloads

Wed, 08 Jul 2026 20:00:00 +0000

September 15, 2026: C4N Instances with Titanium Local SSD (c4n-*-lssd) are now available in preview.

As organizations scale modern workloads — from high-throughput databases and network/security appliances to real-time analytics and AI/ML inference — network and block storage performance can quickly become a bottleneck. Standard virtual machines often struggle to balance compute efficiency with the high-volume data-transfer demands of these applications.

At Google Cloud Next ‘26, we announced C4N in preview, our first network- and block-storage-optimized Google Compute Engine instance that’s purpose-built to eliminate I/O bottlenecks for demanding enterprise applications, and today, it is generally available. Built on Google's custom-designed Titanium offload architecture, C4N instances offload network and storage tasks to dedicated hardware to unlock incredible performance and compute efficiency. C4N offers up to 400 Gbps of network bandwidth and a market-leading 95 million packets per second (MPPS) — nearly 33% higher network bandwidth per vCPU and 224% faster packet processing performance than comparable Intel-based offerings at other hyperscalers. This performance makes C4N a great fit for network-intensive applications such as virtual appliances (e.g., next-gen firewalls, virtual routers, load balancers, DDoS mitigation), large-scale data analytics, telco applications (5G UPF), distributed compute and CPU-based AI/ML workloads.

Paired with Hyperdisk Extreme, our high-performance block storage, C4N also delivers Compute Engine’s highest block storage performance, scaling up to 25 GiB/s of storage bandwidth and 1M IOPS — nearly 33% higher storage bandwidth and 39% more IOPS per vCPU versus comparable Intel-based offerings, making them a strong choice for large-scale databases, high-performance file systems, in-memory databases, and other workloads that benefit from high block storage performance.

Engineered specifically to deliver predictable, high-throughput I/O performance for networking, packets-per-second-bound and storage-optimized applications, C4N allows customers to scale network, storage, and compute resources more precisely to meet specific workload requirements, delivering significant TCO benefits by eliminating the need to over-provision resources just to meet I/O demands.

C4N is powered by 5th Gen Intel® Xeon® Scalable processors (code-named Emerald Rapids).

“Google Cloud’s introduction of C4N highlights how infrastructure innovation and a strong silicon foundation can help customers address increasingly data-intensive workloads. With Intel Xeon and Custom Infrastructure Processing Unit (IPU), C4N delivers the performance and efficiency needed for demanding network optimized environments.” – Srini Krishna, Intel Fellow, Data Center products, Intel

What’s new: Scaling massive data layers with C4N

Our network-optimized C4N instances are designed to deliver predictable, high-performance I/O at scale. By providing consistent bandwidth, packet-processing performance (PPS), and IOPS scaling across all VM shapes and sizes, C4N helps ensure your most demanding data workloads run reliably. To achieve this, we have built deep resiliency into every layer of our infrastructure — from the host and fabric layers to redundant top-of-rack (ToR) switches — delivering continuous performance for your applications.

Compared to general-purpose C4 VMs, the network-optimized C4N delivers significant performance gains across both network and block storage vectors.

Next-generation network performance

Superior VM-to-VM network bandwidth: Achieves up to 400 Gbps of VM-to-VM network bandwidth (an almost 4x increase in BW-per-vCPU over standard C4) and supports up to 50 Gbps single-flow bandwidth between C4N instances routed within the same VPC network. This provides non-blocking data delivery for high-throughput single-stream and multi-stream applications.
Enhanced VM-to-internet performance: Benefits from an 8x increase in internet egress network bandwidth, reaching up to 200 Gbps. It also features a nearly 32x increase in internet egress packet processing performance, scaling up to 48 MPPS.
Optimized I/O for smaller shapes: Keeps your cloud bill lean by delivering up to 25–50 Gbps of network bandwidth specifically for 2–16 vCPU shapes, great for accelerating I/O-bound tasks without needing to over-provision compute. Furthermore, these smaller shapes introduce predictable, steady-state baseline bandwidth limits to provide consistent performance at a lower cost.
Enhanced out-of-the-box networking: gVNIC interfaces on C4N now start with more Tx/Rx queues by default, scaling with vCPUs up to a maximum of 64 (in comparison to 16 queues on C4/C4D).
Shorter Google Cloud Storage transfer times: C4N VMs now offer up to a 2x increase in bandwidth to retrieve and store large volumes of data from Cloud Storage, boosting performance for analytics, AI/ML, and backup workloads.

Better yet, this performance is available out of the box, with no add-ons. Designed for high performance from the get-go, C4N offers maximum performance without needing to purchase or configure premium add-ons like Tier_1 networking.

Dynamic storage performance with Hyperdisk

The C4N instance family, when combined with Hyperdisk, can help dynamically tune storage performance, latency, and throughput independently of your compute instance sizing to deliver high block storage performance for your applications. C4N supports the complete Hyperdisk portfolio, including Hyperdisk Balanced, Balanced High Availability, Extreme, Throughput, and ML block storage options.

Hyperdisk Extreme: C4N with Hyperdisk Extreme provides low-latency, high-speed data access for modern databases and enterprise AI applications, with up to 25 GiB/s of block storage throughput and nearly 1M IOPS, a 2x increase in storage performance over C4. Also, exclusive to network optimized machine series such as C4N, we now offer Hyperdisk Extreme across all machine sizes — even down to the smallest 2 vCPU sizes.
Hyperdisk Balanced: Delivering the highest throughput and IOPS for general-purpose block storage in the Compute Engine portfolio, Hyperdisk Balanced on C4N scales up to 20 GiB/s of block storage throughput and nearly 640K IOPS. This makes it a highly cost-effective option for running storage-intensive applications at scale.

Together, C4N’s network and storage optimizations combine for tremendous impact in real-world applications:

Web serving: Up to 1.5x additional Nginx requests per second compared to C4 for typical web request sizes (100–300Kb), significantly boosting capacity for network-bound web applications
Databases: Up to 45% better queries per second (QPS) for MySQL when data resides primarily on disk than equivalent C4 VMs

What customers are saying

Industry leaders are already proving that workload-optimized infrastructure is the engine for transformation. Here is how our customers are leveraging the network-optimized power of C4N:

“5G Core workloads are inherently network-heavy, demanding high-throughput packet processing and deterministic latency that standard public cloud instances often struggle to maintain at scale. By leveraging the Google Cloud C4N compute family, we’ve found the ideal engine for Ericsson On-Demand. The C4N’s architectural focus on network-optimized compute allows our 5G Core-as-a-Service to reach unprecedented throughput levels — like our recent 1 Tbps milestone — while maintaining the carrier-grade reliability our customers expect. It’s no longer just about cloud-native; with C4N, we are delivering network-native performance in a public cloud environment.” - Eric Parsons, VP, Head of Ericsson On-Demand, Ericsson

“Teradata's Autonomous Knowledge Platform unifies production-grade AI, analytics, and data into a single integrated system — providing the context, governance, and performance backbone autonomous AI demands at scale. Customers rely on Teradata to run mission-critical, highly I/O-intensive workloads where performance and cost control directly determine value.

Google Cloud C4N instances are well suited for these demanding workloads, delivering strong price-performance and supporting more efficient, optimized deployments. By leveraging C4N on Google Cloud, Teradata Cloud can help customers accelerate from insight to action — scaling enterprise intelligence with confidence and driving greater impact from their data and AI investments” - Kevin Dougherty, Senior Director of Product Management, Core Platform, Teradata

“With the next-generation network and storage bandwidth of C4N VMs, Google Cloud NetApp Volumes will unlock new levels of performance to support our customers’ most demanding AI workloads. By collaborating to extend Google Cloud NetApp Volumes support for the C4N VM family, Google and NetApp are deepening our partnership to address real customer challenges. Together, we’re delivering data-in-place AI and analytics solutions that simplify architectures, maximize performance, and turn data into impact.” - Pravjit Tiwana, Senior Vice President and General Manager of Cloud Storage and Services, NetApp

"Most Compute Engine instances ship with a single high-speed network interface. The new C4N doubles the bandwidth potential with two 200 GbE interfaces. That architectural shift is significant. It means we can dedicate both networks entirely to storage traffic, doubling the available bandwidth for data-intensive workloads, and achieving 2x storage performance over the previous generation. The C4N was announced just weeks ago and is already active in Sycomp's test environment, ensuring our customers can evaluate the latest GCP capabilities without delay. Google Cloud’s published maximum hyperdisk balanced performance for the C4N is 20 GiB/s. In our tests, with three storage servers Sycomp achieved 58.5 GiB/s on read and 58.6 GiB/s on write, with ten C4N storage servers we achieved 195 GiB/s read and write — 97% of the theoretical ceiling with zero platform-specific tuning. That's a strong starting point, and there's measurable room to close the remaining gap through configuration work we can finetune. The C4N isn't just faster — it changes the price-performance equation for storage workloads on Google Cloud." - Scott Fadden, Senior HPC Solutions Architect, Sycomp

“At ClipperDB Technologies, our mission is to drive down the cost and drive up the performance of large-scale Spark analytics. Google Cloud’s C4N instances are the perfect compute engine for our fully native architecture. C4N’s substantial increase in network bandwidth per vCPU combined with large memory configurations and 5th Generation Intel Xeon processors align with ClipperDB’s precise parallel cloud-store prefetching and caching, concurrent dataflow native batch pipelines, streaming no-copy exchange, and cloud store checkpoint fault tolerance to radically accelerate and cost reduce Spark workloads with disaggregated Cloud Storage datalakes.

The results speak for themselves: across industry-standard TPC-DS benchmarks, ClipperDB+C4N delivered over 3x lower cost per query and up to 11x faster analytics, all while maintaining 100% Spark compatibility. We can’t wait to see customers dramatically improve their Spark workload price-performance with C4N coupled with Clipper DB Accelerator." - John Busch, CEO, ClipperDB Technologies

A deeper look at C4N shapes and specs

C4N instances are available in nine different sizes ranging from 2-192 vCPUs and up to 1.5 TB of DDR5 memory, offering predefined shapes in high-cpu, standard, and high-mem configurations. For applications that benefit from caching and high-speed, low-latency local storage, C4N VM instances are equipped with up to 12 TiB of latest Titanium SSDs. C4N with Local-SSD are in preview. For workloads that require direct access to the machine's resources (e.g., hypervisors, container platforms), where nested virtualization does not meet the workload’s performance requirements, or have special performance monitoring or licensing needs, we are introducing C4N bare metal shapes. Coming soon, these native bare metal shapes will offer the same network and storage I/O performance as their virtual machine counterparts. Google Cloud customers can use C4N instances with Compute Engine and Google Kubernetes Engine (GKE), with support for other services coming soon.

Name	vCPUs	Memory (GB)	Local Storage (GiB)	Network Bandwidth		Hyperdisk Extreme Bandwidth (MiB/s)	Hyperdisk Extreme IOPS
Name	vCPUs	Memory (GB)	Local Storage (GiB)	VM-VM (Gbps)	VM-Internet (Gbps)	Hyperdisk Extreme Bandwidth (MiB/s)	Hyperdisk Extreme IOPS
C4n-highcpu	2 - 192	4 - 384	N/A	25 - 400	7 - 200	1.000 - 25,000	80,000 - 1M
C4n-standard	2 - 192	7 - 720	N/A	25 - 400	7 - 200	1.000 - 25,000	80,000 - 1M
C4n-standard-lssd	4 - 192	15 - 720	375 - 12,000	30 - 400	7 - 200	1.000 - 25,000	100,000 - 1M
C4n-highmem	2 - 192	15 - 1,488	N/A	25 - 400	7 - 200	1.000 - 25,000	80,000 - 1M
C4n-highmem-lssd	4 - 192	31 - 1,488	375 - 12,000	30 - 400	7 - 200	1.000 - 25,000	100,000 - 1M

C4N machine series performance and specifications

How to get started

Whether you’re hosting heavy-duty distributed databases, running network virtualization appliances, or orchestrating large-scale data pipelines for AI, C4N is engineered to provide the throughput, scale, and efficiency your business demands. C4N instances are now generally available via on-demand, as Spot VMs, and via reservations. You can also take advantage of further cost savings by purchasing Committed Use Discounts (CUDs) or FlexCUDs in one- and three-year terms in the us-central1 (Iowa), us-east1 (South Carolina), us-east5 (Ohio), us-west1 (Oregon) and europe-west2 (London). For more information visit Network Optimized Machine Type.

Ready to establish a high-performance launchpad for innovation? Head straight to the Google Cloud console to spin up a C4N VM under the “Network Optimized” machine family. Stay up-to-date on regional availability by visiting our regions and zones page or contact your Google Cloud sales representative for more information.

BGP route policies: Top 3 use cases by customer demand

Tue, 07 Jul 2026 16:00:00 +0000

When we first made BGP route policies for Cloud Router generally available over a year ago, our goal was to give network administrators deep, programmable control over how network paths are evaluated and propagated. Since then, we’ve been watching closely how our customers have adopted this feature. We've seen network engineering teams build incredibly sophisticated, resilient routing architectures that were previously difficult to achieve without third-party virtual appliances.

This year, we launched policy named sets for Cloud Router. As routing environments grow more complex, managing individual prefixes or communities within these policies can become cumbersome.

Policy named sets solve this by allowing you to group lists of IPv4/IPv6 prefixes or BGP communities into a single, reusable entity. This significantly simplifies your configurations, making it easier to scale, manage, and update your routing rules across multiple Cloud Routers.

Powered by the Common Expression Language (CEL), BGP route policies allow you to define fine-grained, ordered rules to filter BGP routes and modify route attributes directly within Cloud Router.

To celebrate the launch of policy named sets, we want to highlight three of the most impactful ways we've seen customers use BGP route policies over the past year, along with resources on how you can build them yourself.

1. The foundation: Route filtering and network protection

Before manipulating traffic paths, network stability requires strict control over which routes are allowed into and out of your network. We've seen customers extensively use BGP route policies to filter out unwanted learned routes from peers or prevent specific subnet prefixes from being advertised out of their Virtual Private Cloud (VPC).

Operating on a "fail open" model by default, many security-conscious organizations have adapted BGP route policies to create a "fail closed" environment — appending a "drop all" policy as the final term in their evaluation list. This helps enable absolute certainty over accepted network routes, preventing routing loops and ensuring traffic isn't BGP hijacked or inadvertently blackholed.

Dive deeper: For a foundational look at how to set up CEL expressions for route filtering, check out our deep-dive guide: Introduction to BGP policies.

2. Influencing traffic paths for active/standby architectures

Achieving optimal traffic distribution often requires forcing traffic down a specific path, whether for cost optimization or managing active/standby interconnects. Customers have used BGP route policies to influence the preferred BGP route without touching their on-premises hardware.

By dynamically modifying the BGP multi-exit discriminator (MED) attribute, network teams can make a specific peer preferred for incoming traffic. Conversely, if they want to steer traffic away from a congested or backup link, they are using AS-PATH prepending — adding one or more values to the route's AS-PATH to deprioritize it across the broader network.

Dive deeper: To see the configuration steps for managing MED and AS-Path prepending, read: Using BGP policies to influence traffic paths.

3. Solving asymmetric routing with BGP communities

One of the most advanced and highly requested use cases we’ve seen over the last year is achieving traffic symmetry. When enterprises use stateful firewalls or specific network appliances on-premises, return traffic must flow back through the exact same appliance it originated from. If it doesn't, the traffic is dropped.

Customers are successfully solving this by using BGP route policies to match against specific standard BGP communities. By tagging routes with specific communities on-premises, Cloud Router can read those tags via inbound policies and adjust the route preference by manipulating the MED accordingly. This helps ensure that Google Cloud inherently understands the stateful topology of the on-premises network and routes the return traffic symmetrically.

Dive deeper: To learn how to architect stateful traffic symmetry using BGP community tags, explore: Using BGP communities to create traffic symmetry.

Get started today

Taking control of your dynamic routing is now easier and more robust than ever. Using BGP route policies, it's a great time to optimize and secure your hybrid cloud connectivity.

We recommend testing your BGP route policies in a staging environment to verify your CEL expressions and routing logic before rolling them out to production. To explore the technical documentation, check out the BGP route policies overview.

Cloud Network Insights: end-to-end observability for the Cross-Cloud Network

Wed, 17 Jun 2026 19:30:00 +0000

In today’s digital landscape, the network is no longer confined to a single data center or even a single cloud provider. Enterprises are increasingly adopting cross-cloud strategies, connecting Google Cloud workloads to on-premises environments, other clouds like AWS and Azure, and a vast array of internet-facing applications. While this flexibility drives innovation, it can also introduce significant operational complexity. When a user experiences degradation in application performance, the critical question remains: Is it the network, the application, or something else?

We are excited to announce the general availability of Cloud Network Insights, an out-of-the-box, Google Cloud-native solution that provides comprehensive visibility into network and digital experience performance across complex multi-cloud, and hybrid environments.

Closing the visibility gap with active monitoring

Cloud Network Insights, offered in partnership with Broadcom AppNeta, expands your observability beyond Google Cloud to your entire global deployment. By utilizing active synthetic probing, the solution monitors network routes even when no user traffic is present, allowing teams to be proactive rather than reactive.

Whether the source of degradation is in the cloud, on-premises data centers, internet applications, ISPs, or last-mile connectivity, Cloud Network Insights helps you pinpoint the exact location of the bottleneck.

Cloud Network Insights integrates directly into the Google Cloud Observability suite, bringing sophisticated network intelligence into the tools you already use. With Cloud Network Insights, you get:

End-to-end network path visibility: Gain a hop-by-hop visualization of the network path between your sources and destinations. Monitor critical metrics like round-trip time (RTT), packet loss, and jitter across networks you don’t directly manage.
Digital experience insights: Go beyond the network layer to monitor digital experience for web applications. Measure DNS resolution times, HTTP response codes, and full browser page-load times to identify whether an application's degradation is due to the network or the application itself.
Proactive detection and alerting: Use synthetic testing to identify performance dips before they impact your customers. Alarms are integrated with Cloud Monitoring and Cloud Logging, enabling alerting via email, Slack, or PagerDuty.
SLA validation: Arm your team with the data needed to verify if ISPs and service providers are meeting their performance commitments.
Rapid root-cause analysis: Quickly differentiate between network problems, application-level issues, or browser performance impacts.
Integrated monitoring: Access metrics and logs directly within Google Cloud, leveraging Cloud Monitoring and Cloud Logging for dashboards and alerting. Utilize the open partner ecosystem of Google Cloud as well as support for the OpenTelemetry protocol for metrics and logs, allowing direct ingestion by OTel SDKs and collectors.
Agentic workload monitoring: Use synthetic testing to monitor connectivity and network performance to help ensure optimal connectivity to your agents and tools.

Network performance and multi-path routes to/from Google Cloud, AWS, and Azure in one view

How it works: active synthetic probing

Cloud Network Insights uses active synthetic probing technology that consists of three main components:

Monitoring Points: You deploy lightweight software agents, called Monitoring Points, into critical network segments, such as a central VPC, a remote branch, or an on-premises data center. These can be deployed as containers or virtual machines.
Synthetic probes: These Monitoring Points send small, frequent bursts of synthetic traffic (simulating a user or application) to a target destination. This allows you to monitor performance 24/7, even when no real users are on the network.
Data synchronization: The Monitoring Points send real-time performance telemetry to a central backend service. This data is then synchronized back to Google Cloud, with metrics exported to Cloud Monitoring, and alarms and events sent to Cloud Logging.

Core capabilities

Cloud Network Insights supports two primary types of monitoring to give you a full picture of your infrastructure:

1. Network performance monitoring (Layers 3 and 4)

This provides a hop-by-hop visualization of the network between a source and a destination, including.

Metrics captured: Round-trip time (RTT), packet loss, jitter, and path changes.
Single-ended mode: The agent probes an external target (like a URL, IP address or an API endpoint) that doesn't have a Monitoring Point installed.
Dual-ended mode: The Monitoring Point probes another Monitoring Point. This provides richer data, including precise one-way latency and the ability to detect asymmetric routing (when data takes a different path going out than it does coming back).

Network path metrics in Google Cloud console

2. Digital experience monitoring (Layer 7)

With digital experience monitoring, you can track the end-to-end experience of a web application. Here, you can choose from:

Browser mode: Uses a real browser engine (Selenium) to load full web pages, execute JavaScript, and render content. It measures complete page-load times to validate the actual user experience.
HTTP mode: Sends synthetic HTTP/S requests to a URL or API endpoint. This is a lightweight check for server availability, response time, and DNS/TLS performance.

Intelligence and automation

Cloud Network Insights also offers a variety of monitoring and troubleshooting capabilities.

Proactive alarms: Cloud Network Insights leverages auto-baselining to establish dynamic performance thresholds based on your historical metric data. If a metric deviates from your defined parameters, the system instantly triggers an event in Google Cloud, routing alerts directly to your team via email, Slack, or PagerDuty.
Monitoring policies: You can automate monitoring setups across large-scale environments by defining policies that dynamically create or remove paths based on custom tags. For instance, you can automatically track a core web application's performance from specific geographic regions.
Root-cause analysis: Because Cloud Network Insights extends visibility into traditionally "unwatched" areas like ISPs and transit networks, it instantly pinpoints whether a slowdown is occurring within Google Cloud, at the ISP level, or inside another cloud environment like AWS or Azure.
AI-driven insights: With integration to Gemini Cloud Assist, you can use natural language to interrogate Cloud Network Insights telemetry alongside your broader infrastructure data. Rather than manually pivoting between dashboards, ask Gemini to cross-reference specific Cloud Network Insights metrics against other Google Cloud metrics, reducing mean time to resolution (MTTR).

What customers are saying

We are already seeing strong interest from customers looking to simplify their cross-cloud operations. Organizations like Sabre and Pexip are already using Cloud Network Insights to gain clarity in their hybrid environments.

"In an environment as complex and high-scale as Sabre’s, total visibility isn't just a luxury — it's a requirement for operational resilience. Cloud Network Insights will enable us to further shift our posture towards proactive optimization. By providing granular, real-time telemetry across our global cloud footprint, it helps eliminate the traditional 'black box' of the network, allowing our teams to resolve bottlenecks before they impact the traveler experience." - Alfredo Rodriguez, VP of Cloud and Infrastructure, Sabre

“Cloud Network Insights closes the 'visibility gap' between the private corporate network and the public cloud, empowering our joint customers to pinpoint performance bottlenecks in seconds rather than hours.” - Alan Davidson, CIO, Broadcom

Get started today

Navigating complex digital ecosystems shouldn't mean sacrificing visibility. Cloud Network Insights bridges the gap across multi-cloud and hybrid environments by combining deep network performance metrics with digital experience monitoring. Coupled with direct integrations into Google Cloud Observability and Gemini Cloud Assist, your teams are empowered with intelligent alerting, robust SLA validation, and rapid root-cause analysis. We look forward to helping you gain a clearer, unified view of your Cross-Cloud Network.

You can get started in the Google Cloud console today. To learn more:

Explore our product documentation for deep dives into deploying Monitoring Points and configuring policies.
Check out the latest release notes to stay updated on new features.
Watch the overview video
Hear more about the partnership between Google Cloud and Broadcom:

Report: GKE Inference Gateway delivers up to 92% faster AI responses

Tue, 09 Jun 2026 16:00:00 +0000

As generative AI moves from experimental pilots to massive production environments, the efficiency of your infrastructure becomes the ultimate differentiator. One way to get the most out of it and minimize costly accelerator idle time is to leverage the Google Kubernetes Engine (GKE) Inference Gateway, which intelligently routes generative AI workloads based on real-time model server metrics.

Instead of relying on traditional, naive round-robin load balancing — which frequently triggers expensive accelerator recomputation and spikes user latency — this native extension of the GKE Gateway utilizes advanced capabilities like prefix caching and model-aware routing. By ensuring requests land on the exact accelerator that is primed to process them right away, GKE transforms how you can serve your large language models (LLMs), with excellent hardware utilization and ultra-fast response times.

In fact, according to an independent benchmark report, GKE Inference Gateway outperforms the next leading managed Kubernetes service with 15.7% higher throughput, 92.8% shorter wait times, and 62.6% lower inter-token latency. This performance takes LLM-based applications from sluggish and expensive to fast and production-grade.

That performance tracks with Snap’s experience using GKE Inference Gateway.

“At Snap, we are integrating llm-d into our production AI infrastructure to facilitate high-performance inference at scale. By employing prefix-cache-aware routing, we have achieved prefix cache hit rates ranging up to 75-80%. We appreciate the open-source nature of llm-d, as it enables seamless integration with our Envoy-based Service Mesh.” - Vinay Kola, Senior Manager, Software Engineering, Snap Inc.

In this blog, we take a closer look at GKE Inference Gateway’s prefix caching, complete with examples. We also provide more details about its benchmark results. Let’s jump in.

The secret to low-latency AI: Prefix caching

Prefix caching optimizes LLM performance by storing the KV cache (activation states) of long, repetitive prompt prefixes. When consecutive user requests share the same system instructions, context, or documentation, the model entirely skips reprocessing those tokens. GKE Inference Gateway reads incoming request prefixes and matches them to the specific pods that already hold that data in memory. This eliminates the "thinking" tax on your GPUs and TPUs, turning heavy reasoning loops into near-instant answers.

Use case 1: Documentation and codebase Q&A with retrieval-augmented generation (RAG)

When querying massive enterprise repositories, you can ground your LLMs’ responses without any added latency by pinning entire documentation sets as static cached prefixes, using RAG.

Instead of forcing an LLM to re-read thousands of lines of API references or corporate wikis for every single user question, GKE Inference Gateway routes the query to a pod that already has that specific context warmed up in its KV cache. The LLM only has to compute the user's brief, dynamic question, completely bypassing expensive document re-evaluation.

code_block: <ListValue: [StructValue([('code', '[STATIC PREFIX - STAYS IN CACHE] You are an expert AI assistant specializing in technical documentation. Below is the complete API documentation for our software platform. Use this context to answer the user\'s questions accurately. If the answer cannot be found in the documentation, say "I cannot find that in the provided context." \r\n\r\n<documentation> [10,000+ words of API reference documentation, endpoints, error codes, etc.] </documentation> \r\n\r\n[DYNAMIC SUFFIX - CHANGES PER REQUEST] User Question: How do I handle a 429 rate limit error using the Python SDK?'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbd616dd0>)])]>

Use case 2: Multi-turn chat

You can also use prefix caching to maintain customer service interactions across thousands of simultaneous sessions without compounding compute costs. You can do so by caching permanent system personas and core business rules directly on the LLM server.

In enterprise chat architectures, the base system prompt and reference tables remain completely identical across millions of customer interactions. GKE Inference Gateway handles these multi-turn conversations using context-aware routing to bypass repetitive token processing, so that your chatbot stays ultra-responsive even under peak traffic.

code_block: <ListValue: [StructValue([('code', '[STATIC PREFIX - STAYS IN CACHE] \r\n-System Persona: You are "FinBot", a helpful, empathetic, and compliant virtual assistant for ABC Banking Solutions. You must strictly adhere to the following rules: 1. Never provide concrete investment advice. 2. Always verify if the user is asking about checking or savings. 3. Keep your answers under 3 sentences. 4. If a user is angry, offer to connect them to a human manager. \r\n\r\nHere is the current interest rate table for May 2026: \r\n- Savings: 4.2% APR \r\n- Checking: 0.5% APR \r\n- CD (12-month): 5.1% APR \r\n\r\n[DYNAMIC SUFFIX - CHANGES PER REQUEST] User: Hi, I\'m trying to figure out how much I\'d make if I locked away $10,000 for a year?'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbedf3a90>)])]>

GKE outperforms alternative managed Kubernetes solutions

To validate these architectural advantages, Principled Technologies recently released an independent benchmark report comparing GKE (equipped with the GKE Inference Gateway) against a standard third-party managed Kubernetes service utilizing conventional round-robin HTTP load balancing.

Tested on a Llama 3.1 8B Instruct shared prefix workload using identical hardware (eight NVIDIA A100 40GB GPUs) the results reveal a massive performance gap between the two Kubernetes services. GKE didn't just win; it completely redefined inference efficiency across three critical metrics:

Higher throughput: 15.7% more tokens processed per second, enabling higher request capacity or reduced hardware needs for the same workload
Much faster time to first token (TTFT): 92.8% shorter wait times, producing dramatically quicker perceived response starts for interactive scenarios
Lower inter-token latency (ITL): 62.6% reduction, resulting in smoother and faster token streaming after the first token

Figure 3: Mean latency (normalized time per output token) of GKE with GKE Inference Gateway and third-party managed Kubernetes service on the Llama 3.1-8B Instruct LLM on the Shared prefix use case. Both solutions used the same hardware. Source: Principled Technologies

	GKE	3rd party ManagedKubernetes Service	GKE Advantage
Mean outputtoken throughput	7,169.21 outputtokens per second	6,042.05 outputtokens per second	15.7% more outputtoken throughput
Mean time tofirst token (TTFT)	188.36 ms	2624.73 ms	92.8% less TTFT
Mean inter-tokenlatency (ITL)	30.20 ms	81.03 ms	62.6% lower ITL

Figure 4: GKE with GKE Inference Gateway delivered superior AI inference compared to a third-party managed Kubernetes service using standard HTTP LB.

Ready to accelerate your gen AI inference workloads?

Whether you’re deploying inference workloads such as real-time customer support agents, dynamic coding assistants, or sub-second fraud detection models, infrastructure latency dictates your user experience. By ensuring shared prompt prefixes hit the active cache nearly 100% of the time, GKE Inference Gateway transforms your LLMs from sluggish, expensive reasoning engines into rapid, capital-efficient, production-grade powerhouses.

Ready to explore the performance advantage that GKE Inference Gateway can bring to your gen AI workloads? Access the full benchmark report here and watch this explainer video to learn more.

^{A special thanks to Dan Sullivan, Senior Performance Architect, Principled Technologies.}

Experimenting with TPUs, GKE Managed DRANET, and Multi-cluster Inference Gateway

Tue, 02 Jun 2026 07:00:00 +0000

What happens when your workload fails in one region but you need access to service? This is a common case for availability and uptime. With recent enhancement to the Kubernetes ecosystem and capabilities like Dynamic Resource Allocation (DRA) and Inference Gateway. I decided to experiment with these capabilities in Google Cloud for a simple test using an AI inference workload.

In this blog, we will explore this setup and you can also jump straight into the detailed configs in this codelab Build multi-cluster GKE Inference Gateway, with TPUs , Cloud Storage FUSE and managed DRANET.

Building blocks

To build out this experiment, use the following products, features, and tools:

Google Kubernetes Engine (GKE) managed DRANET: This is a managed feature that lets you request and share resources among Pods. This supports GPUs, and TPUs. In this test TPUs were used in two different regions with networking assigned using managed DRANET.
Multi-cluster GKE Inference gateway: Load balances your AI/ML inference workloads across multiple GKE clusters. This works in a failover situation which is what my experiment intended to test. The type which supports this is the Multi-cluster Cross-region internal Application Load Balancer gke-l7-cross-regional-internal-managed-mc
Cloud Storage FUSE: Provides a way to store data, models, checkpoints, and logs directly in Cloud Storage. To speed up the deployment, an open source gemma model was downloaded to this storage for retrieval.
Virtual private Cloud (VPC): The foundational global network providing isolated, secure communication for the internal load balancers and compute nodes
GKE Fleets: Fleets group the separate regional clusters under a unified management control plane
TPU v6e: Google's custom AI accelerators that provide the high-performance compute required to serve the model. The VM family type used was the ct6e-standard-4t in a 2x2 Slice

Design pattern example

The aim is to deploy a LLM model (Gemma 3) onto 2 GKE clusters in different regions. Each cluster will use 4 TPU v6e chips. The model should be stored in Cloud Storage. The workload is served using GKE Inference Gateway which supports multi-clusters. The traffic should be routed to the region closest to the user and failover to the other region if one region fails.

Putting it together

To get access to the TPUs for your project in two regions you have to ensure you have the necessary quota in those regions.

Begin: Set up the environment.

Create a standard VPC, with firewall rules and subnet in the same zone as the reservation.
Create a proxy-only subnet this will be used with the Internal regional application load balancer attached to the GKE inference gateway
Set up firewall rules allowing traffic and health checks.
Reserve static internal IP addresses in both regions for the Gateway.
Provision a Cloud Storage FUSE bucket and configure a dedicated IAM Service Account. Bind this to a Kubernetes Workload Identity so your pods can securely mount the bucket and read the model weights directly.

Next: Create standard GKE clusters and node pools.

Deploy two separate GKE clusters in your chosen regions configured.
Enable the Gateway API (--gateway-api=standard) and the Cloud Storage FUSE CSI driver (--addons GcsFuseCsiDriver) during cluster creation.
Create dedicated TPU v6e node pools (ct6e-standard-4t) for both clusters.
Enable managed DRANET on these TPU node pools by setting the flags ---accelerator-network-profile=auto, and --node-labels=cloud.google.com/gke-networking-dra-driver=true

Next: Establish the global mesh via Fleet Registration.

Register both GKE clusters to a unified GKE Fleet by following the fleet creation and registration setup.
Enable Multi-Cluster Service Discovery and Multi-Cluster Ingress on your fleet.
Designate your primary region as the configuration hub to act as the control plane for routing rules across both regions.

Next: Deploy the AI workload.

Use a temporary Kubernetes job to download the Gemma 3 (gemma-3-27b-it) model weights directly into your Cloud Storage bucket.
Define a ResourceClaimTemplate that explicitly requests the managed DRANET device class (deviceClassName: netdev.google.com ) with the allocation mode set to "All".

code_block: <ListValue: [StructValue([('code', 'apiVersion: resource.k8s.io/v1\r\nkind: ResourceClaimTemplate\r\nmetadata:\r\n name: all-netdev\r\n namespace: default\r\nspec:\r\n spec:\r\n devices:\r\n requests:\r\n - name: req-netdev\r\n exactly:\r\n deviceClassName: netdev.google.com\r\n allocationMode: All'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbeb76650>)])]>

Deploy your inference server (e.g. vLLM) on the TPU nodes in both regions. Ensure the pod spec utilizes node selectors for the 2x2 TPU topology, requests exactly 4 TPUs, and mounts the netdev claim. This guarantees your pods utilize the dedicated accelerator networking alongside standard Ethernet.

Next: Configure the Multi-Cluster Inference Gateway.

Install the necessary Custom Resource Definitions (CRDs) so Kubernetes can process specialized routing objects like the InferenceObjective.
Deploy an AutoscalingMetric to track hardware utilization, such as KV cache usage.
Use Helm to group the independent AI deployments from both regions into a single, logical InferencePool.
Deploy the Cross-Region Gateway and its associated HTTPRoute to manage incoming global traffic.
Apply health checks and backend policies to the pool to ensure load balancing relies on your custom hardware metrics.

Configure an InferenceObjective to instruct the gateway to route prompts to the region with the highest availability, avoiding overloaded TPUs.

code_block: <ListValue: [StructValue([('code', 'apiVersion: gateway.networking.k8s.io/v1\r\nkind: Gateway\r\nmetadata:\r\n name: cross-region-gateway\r\n namespace: default\r\nspec:\r\n gatewayClassName: gke-l7-cross-regional-internal-managed-mc\r\n addresses:\r\n - type: networking.gke.io/named-address-with-region\r\n value: "regions/europe-west4/addresses/gemma-gateway-ip-europe-west4"\r\n - type: networking.gke.io/named-address-with-region\r\n value: "regions/us-east5/addresses/gemma-gateway-ip-us-east5"\r\n listeners:\r\n - name: http\r\n protocol: HTTP\r\n port: 80\r\n---\r\napiVersion: gateway.networking.k8s.io/v1\r\nkind: HTTPRoute\r\nmetadata:\r\n name: gemma-route\r\n namespace: default\r\nspec:\r\n parentRefs:\r\n - name: cross-region-gateway\r\n kind: Gateway\r\n rules:\r\n - backendRefs:\r\n - group: networking.gke.io\r\n kind: GCPInferencePoolImport\r\n name: gemma-pool\r\n port: 8000\r\n---\r\napiVersion: networking.gke.io/v1\r\nkind: HealthCheckPolicy\r\nmetadata:\r\n name: gemma-health-check\r\n namespace: default\r\nspec:\r\n targetRef:\r\n group: networking.gke.io\r\n kind: GCPInferencePoolImport\r\n name: gemma-pool\r\n default:\r\n config:\r\n type: HTTP\r\n httpHealthCheck:\r\n requestPath: /health\r\n port: 8000\r\n---\r\napiVersion: networking.gke.io/v1\r\nkind: GCPBackendPolicy\r\nmetadata:\r\n name: gemma-backend-policy\r\n namespace: default\r\nspec:\r\n targetRef:\r\n group: networking.gke.io\r\n kind: GCPInferencePoolImport\r\n name: gemma-pool\r\n default:\r\n timeoutSec: 100\r\n balancingMode: CUSTOM_METRICS\r\n trafficDuration: LONG\r\n customMetrics:\r\n - name: gke.named_metrics.tpu-cache\r\n dryRun: false\r\n maxUtilizationPercent: 60\r\n---\r\napiVersion: autoscaling.gke.io/v1beta1\r\nkind: AutoscalingMetric\r\nmetadata:\r\n name: tpu-cache\r\n namespace: default\r\nspec:\r\n selector:\r\n matchLabels:\r\n app: gemma-server\r\n endpoints:\r\n - port: 8000\r\n path: /metrics\r\n metrics:\r\n - name: vllm:kv_cache_usage_perc\r\n exportName: tpu-cache\r\n---\r\napiVersion: inference.networking.x-k8s.io/v1alpha2\r\nkind: InferenceObjective\r\nmetadata:\r\n name: gemma-objective\r\n namespace: default\r\nspec:\r\n priority: 10\r\n poolRef:\r\n name: gemma-pool\r\n group: "inference.networking.k8s.io"'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f4dbdc6b7d0>)])]>

Testing the Failover

Verify the highly available architecture by simulating a primary region outage. Once the primary deployment is taken offline, the Gateway automatically detects the failure and seamlessly reroutes all subsequent user requests to the active secondary cluster, ensuring continuous availability without dropping traffic.

Next Steps

Take a deeper dive into a hands-on codelab and more information on these features review the following.

Hands-on Codelab: Build multi-cluster GKE Inference Gateway, with TPUs , Cloud Storage FUSE and managed DRANET
Document set: DRANET
Documentation: AI Hypercomputer

Want to ask a question, find out more or share a thought? Please connect with me on Linkedin.

How we evolved Google’s global and data center networks for the AI era

Tue, 26 May 2026 16:00:00 +0000

Over the last 25 years of building Google’s global network, we’ve navigated major architectural eras — from the Internet, to streaming, and the cloud. Today, we are squarely in the midst of a fourth: the AI era. The applications in the AI era are fundamentally different from the consumer and enterprise applications of the previous eras and impose a set of novel and demanding requirements — on compute resources, of course, but also on the network.

Consider the fundamental physical challenge, which is that it is far more difficult to move electrons (electrical power) than it is to move photons (data over fiber). Because the demand for AI compute frequently outpaces the space and power capacities of individual facilities, we strategically locate data centers near sustainable energy sources, or in locations with pathways to add clean energy sources to the local grid. Then, by utilizing the network to distribute AI workloads across campuses, we create a massive-scale, pooled hypercomputing resource that overcomes the power limitations of any single site.

To deliver this, we created an end-to-end, vertically integrated AI technology stack that comprises everything from chips to systems, to platforms and application and agentic ecosystems. This stack includes a portfolio of pre-built agents and applications; our Gemini Enterprise Agent Platform for you to build, scale, govern, and optimize your AI-enabled applications; world-class AI models; as well as our unified data platform. All this is anchored by our AI Hypercomputer, a unified infrastructure that combines purpose-built hardware and open software, and that comes with flexible consumption options. Our network, forged through decades of innovation, is the essential fabric of the AI Hypercomputer.

The network supporting this stack must meet the stringent bandwidth, scale, and performance needs of AI workloads. This applies not only within the campus, where the network must scale up and out, but also across the wide area network (WAN) along with high-bandwidth interconnects, to bring AI training data from its source to AI compute resources.

To address these challenges, we’ve reimagined three key pillars of our network infrastructure: the fabric inside the AI Hypercomputer, the fabric across the AI Hypercomputer, and our global network. Let’s take a closer look at each of these.

1. The fabric inside AI Hypercomputer

The massive scale of today’s AI models, fueled by the explosive growth of foundational AI model parameters, makes AI training very compute- and network-intensive.

This necessitates an exponential increase in required network bandwidth, with strict bounds on delay (e.g., tail latency) to accommodate AI workloads’ peculiar traffic patterns, which are characterized by sensitivity to performance variation and synchronized bursts, i.e., intense, coordinated, millisecond-level traffic spikes. Furthermore, since large-scale training jobs are uniquely vulnerable to failures and performance stragglers, maintaining high reliability and predictable performance is absolutely essential.

To address the scale, low latency, and high predictability that modern AI workloads require — as well as protection from extreme bursts — we’ve adopted a "campus as a computer" philosophy, decoupling our network into three distinct domains:

a scale-up domain for intra-pod connectivity
a dedicated east-west scale-out accelerator fabric
the Jupiter frontend network for north-south compute and storage access

This decoupled architecture provides three strategic advantages: it allows domains to evolve independently for faster innovation; provides a non-blocking scale-out network with massive training bandwidth; and helps ensure the network can be co-designed in lockstep with new ML accelerators, for superior hardware support.

Recently, we unveiled Virgo Network, our scale-out data center fabric specifically engineered for modern AI. Virgo utilizes high-radix switches and a flat, two-layer non-blocking topology to provide massive bisection bandwidth, while minimizing latency by reducing network tiers. Its multi-planar design, featuring independent control domains for each plane, provides hardware-level resilience and fault isolation. Furthermore, Virgo can expand across multiple data centers, removing physical building limitations and enabling flexible AI compute scaling.

The effectiveness of our network and accelerator codesign is perfectly illustrated by the recently debuted eighth generation TPUs. Within this architecture, Virgo Network can link 134,000 TPU 8t chips with up to 47 petabits/sec of non-blocking bi-sectional bandwidth in a single fabric. Virgo Network delivers up to 4x the bandwidth per TPU 8t accelerator over the previous generation, and 40% lower unloaded fabric latency for TPU 8t compared to the previous generation network for TPUs. In this setup, Virgo Network manages the raw accelerator traffic, while Jupiter provides reliable and rapid access to the global WAN and storage. When integrated with Pathways and JAX, this AI Hypercomputer networking engine facilitates near-linear scaling for up to a million TPU 8t chips in a single logical cluster.

Autonomous reliability: protecting workload goodput

Building a resilient megascale fabric represents only part of the challenge. In a cluster of hundreds of thousands of chips, hardware failures are a statistical certainty. A single stalled instance can stop an entire synchronous training job, wasting valuable compute cycles. As such, efficient fault localization is critical.

We engineered Virgo Network with autonomous reliability capabilities to maximize workload efficiency at scale, a.k.a., goodput. Expanding on our existing straggler detection, Virgo Network now also features automated hang detection. The moment a fail-stop event occurs, our specialized agents immediately localize the fault, isolate the faulty instance, and enable you to restore the training job from a checkpoint — getting your training timeline back on track, with minimal manual intervention. Learn more by watching this demo:

To complement these capabilities, we also use high-resolution, sub-millisecond telemetry to identify elusive network micro-bursts that are usually missed by conventional 30-second monitoring intervals. These high-resolution telemetry advancements enable more efficient network operations, better provisioning, and a lower mean time to recovery.

2. The fabric across AI Hypercomputer

The exponential growth of modern AI workloads requires us to scale and distribute AI workloads across multiple campuses over a WAN. At the same time, traditional networks weren’t built for the high bandwidth and extreme burstiness of AI traffic, and often fail to detect microbursts that can lead to severe performance degradation. We have developed a suite of innovations to optimize WAN performance for cross-site AI deployments, including:

A multi-shard global network that enables horizontal scaling. Our global network sustained a 10X WAN traffic growth from 2020 to 2025.
Tuning the fabric for essential availability, latency, and quality of service (QoS) attributes. Real-time microburst management helps ensure fair bandwidth allocation and infrastructure isolation across our multi-tenant infrastructure.
Multi-shard isolation to ensure each network shard operates with its own control, data, and management planes.

Combined with regional isolation and Protective Reroute, this architecture minimizes failure impact and shortens user-visible outages — delivering the beyond-nines reliability essential for AI workloads.

Providing high-speed, flexible, and cost-effective interconnectivity is also a priority. AI training relies on vast datasets that are often located on-premises or across various clouds. Given the high cost of AI compute, minimizing idle time is essential; for instance, upgrading from a 100 Gbps link to a 3.2 Tbps connection reduces the time to transfer a petabyte of data from 22.2 hours to just 0.7 hours — a 97% reduction in AI compute idle time spent waiting for data. Our AI-native Cloud Interconnect is purpose-built for the high-bandwidth and low-latency needs of AI workloads, featuring an optimized data path with 400 Gbps links that scale in 3.2 Tbps increments to reach petabit-per-second capacity. It also offers traffic differentiation and flexible connection options, including direct fiber peering and colocation facilities. AI-native Cloud Interconnect supports petabit-scale data transfer with reliable, private connectivity necessary for your cross-cloud AI training and serving.

3. A resilient global network for the age of inference

Applications serving AI inference to a global user population or supporting an agentic enterprise are far more demanding than conventional web apps. The need for opportunistic use of expensive AI compute available at distant locations, distributed service dependencies, and the burstiness of the traffic demand high bandwidth network with a global footprint, as well as deep peering to SaaS providers, ISPs, and hyperscalers. To maintain responsiveness and "always-on" availability, applications need low latency and a highly resilient network.

With its connectivity, scale, and resilience, Google’s global network is well-equipped to handle the demands of the age of AI inference. Our network spans more than 10 million kilometers of terrestrial and subsea fiber, connects our 43 cloud regions, and features 200+ edge locations, providing the essential footprint for serving AI inference. Our Premium Tier network delivers the low latency and reliability needed for consistent, high-quality global user experience. By optimizing traffic entry and exit points, the network significantly boosts application performance, with resilience at the core of this "always-on" infrastructure.

Building the future, together

As a Google Cloud customer, these network innovations are built directly into your environment. Google’s network delivers the massive scale, capacity, reliability and performance essential for your AI workloads.

The AI era demands more than just raw compute; it necessitates a robust network fabric to scale. Our vertically integrated AI technology stack — from silicon to software ecosystems — is powered by the AI Hypercomputer to accelerate your transformation and make AI helpful for everyone. Whether through our megascale fabric, resilient global network for inference, or AI-native Cloud Interconnect, we ensure your AI journey is efficient and reliable. We look forward to building this future with you.