Data Analytics

Cool stuff Google Cloud customers built, April edition: BMW big on SLMs, MLB’s Scout Insights AI, personalized resort experiences

Wed, 15 Apr 2026 16:00:00 +0000

AI and cloud technology are reshaping every corner of every industry around the world. Without our customers, who are building the future on our platform, there would be no Google Cloud. In this regular round-up, we dive into some of the exciting projects redefining businesses, shaping industries, and creating new categories.

For our latest edition, we learn why BMW Group is experimenting with small language models (SLMs); catch AI-powered commentary from Major League Baseball; hit the slopes with Vail Resort’s AI concierge; build an intelligent grid with CTC Global; witness how ID.me created secure global scale; and see how Manhattan Associates supply chain tools now handle 1 billion daily API calls.

Be sure to check back next month to see how more industry leaders and exciting startups are putting Google Cloud technologies to use. And if you haven’t already, please peruse our list of 1,001 real-world gen AI use cases from our customers.

BMW tests the big potential of small models

Who: As one of the world’s leading providers of premium cars and motorcycles, BMW Group is always at the forefront of automotive technology. This ethos pushed the company to test what type of AI language models are ideally suited to driving situations, where access to cloud-based LLMs isn’t always possible.

What they did: BMW Group wanted to explore the potential of small language models (SLMs), which could run within the limited hardware on a vehicle. Finding the right trade-off between size and capability requires careful optimization, and the sheer volume of viable combinations renders manual searches for the optimal configuration an incredibly tedious, if not impossible, undertaking. To overcome this challenge, BMW and Google built automated, reproducible workflows through executable pipelines using Vertex AI.

Why it matters: The path from a general-purpose LLM to a specialized SLM isn’t straightforward. Every choice — from type of quantization to characteristics and contents of the fine-tuning domain-specific dataset — affects the quality and efficiency of the final model. This creates an exponential range of configurations, each with different trade-offs. It’s a great example of using AI to scale an optimization problem for other AI.

Learn from us: “With automated pipelines, we can rapidly adapt models to our domain and rigorously test and evaluate them against domain-specific benchmarks. This allows us to iterate and optimize models in hours rather than days.” – Dr. Céline Laurent-Winter, vice president, Connected Vehicle Platforms at BMW Group

MLB Scout Insights: AI-powered color commentary

Who: Major League Baseball is famous for its colorful announcers. Now, MLB is bringing more baseball color straight to your pocket, and Gemini is helping give it a voice.

What they did: Each season, millions of baseball fans use the MLB app and tap over to the Gameday feature for live, up-to-the-pitch action across more than a dozen games. Starting this season, the league launched MLB Scout Insights in Gameday, which uses Gemini models to quickly scan decades of game and player data, cross-references it with situational game scenarios, and then delivers game-relevant context during key matchups.

Why it matters: Given the sport’s storied history, 162-game regular season, and global reach, baseball fans are among the most sophisticated and passionate out there. To keep them engaged with Gameday and the MLB app, the league wanted to deliver insights that truly felt meaningful and interesting. Building the tool meant answering a rather squishy question: What makes an insight actually insightful, not just an accurate fact, and how can an AI learn that distinction? The answer came from some clever “surprisal” analysis.

Learn from us: "With Scout Insights, every fan can feel like the smartest person in the stands, at the water cooler, or on the couch. It’s about deepening connections to the game, and sharing that passion with others. That’s the magic of sports, and we’re making more of it possible with the magic of AI." – Josh Frost, senior vice president of product & Matt Graser, director of engineering, Major League Baseball

Vail Resorts makes personalized AI assistance easy

Who: Vail Resorts operates some of the most iconic and beloved mountain destinations in the world, including Whistler Blackcomb, Park City Mountain, Stowe, and Crested Butte.

What they did: Vail Resorts launched My Epic Assistant during the 2024-2025 snow season, and expanded it this year to add even more AI-powered chat features powered by Google’s powerful Gemini models. The result is an agentic guide to the slopes that can help skiers and snowboarders decide on the right season pass, share the latest snow report, check on lesson preparations, or suggest a good stop for cocoa.

Why it matters: Vail Resorts wanted more than a chatbot; they wanted a digital concierge that understands the nuance between a powder day at Whistler and a family trip to Beaver Creek. As the company implemented and refined personalization, improved search, summary capabilities, and conversational flow within My Epic Assistant, the app has delivered a 45% reduction in escalation to human agents since launch.

Learn from us: "Utilizing tooling from Google Cloud, we could lean into agentic design patterns that gave us a way to unlock natural, personalized conversations. These boosted customer satisfaction, while reducing the need for manual intent design. These tools also let us combine flexibility and control to enable the assistant to respond fluidly but always within the boundaries of our brand, policies, and product strategy.”— The Vail Resorts technical team

CTC Global turns the smart grid into an intelligent one

Who: CTC Global is a leading manufacturer of advanced transmission conductors and power lines. While many nodes in the grid contain IoT sensors, it recognized a literal gap in the transmission lines themselves.

What they did: CTC’s new GridVista platform threads fiber-optic cable into its high-strength carbon fiber composite core, and connects these to monitoring technology built with AI and monitoring technology from Google Cloud and Tapestry. With GridVista, CTC can turn every inch of transmission into a smart sensor.

Why it matters: GridVista gives CTC grid operators an accurate and reliable view of what’s happening across the entire line — based on actual, real-time data from the entire length of the conductor, not point estimates from a static model of line conditions or the occasional clamped-on sensor. This means they can significantly improve safety, manage costs, increase the line’s capacity to transmit power, and enhance reliability with more precise insights about events that might trigger an outage.

Learn from us: “This awareness allows for a grid that can truly sense its own health in real time and provide unprecedented awareness of conditions on the entire line. Whether that’s real time storm impacts, ice load, wind load, branches on the wire, or temperatures on or under the line. The GridVista system truly represents next generation capabilities. ” — J.D. Sitton, CEO, CTC Global

ID.me reduces risk while scaling past 160 million users

Who: ID.me is transforming digital identity security for the modern era, offering a single login that lets you easily prove you’re you across a wide range of platforms and wallets.

What they did: ID.me currently serves more than 160 million users, including as many as 40,000 at any time, so they can prove their identity online as easily as flashing their driver’s license in person. Over the last two years, ID.me migrated more than 50 terabytes of data across 15 database instances to Google Cloud with minimal downtime. They also introduced a two-tier architecture with Cloud SQL supporting its smaller and more standard services, while AlloyDB runs heavier workflows that form the backbone of the ID.me platform.

Why it matters: AlloyDB AI has allowed ID.me to scale its systems to handle 10X-20X of what was possible before — and at a lower price to boot. That responsiveness and reliability led the U.S. federal government to recognize ID.me for its role in preventing large-scale fraud within national systems.

Learn from us: "We’ve been able to scale both our infrastructure and trust. With a platform that’s faster, smarter, and built to handle portable identity at massive scale, we’re one step closer to our goal: a secure, digital way to prove who you are, wherever you need it, that works everywhere you need it." — Kevin Liu, Cloud Platform Architect, ID.me

Manhattan Associates powers more than a billion daily API calls

Who: Manhattan Associates is a global leader in supply chain and omnichannel commerce solutions, offering tools and platforms that reach more than 2 billion people across 20 billion consumer touchpoints.

What they did: Manhattan Associates modernized its Manhattan Active SaaS platform by migrating from legacy Oracle and DB2 systems to Google Cloud databases. Each capability of Manhattan Active now runs as an independent, containerized service orchestrated by Google Kubernetes Engine (GKE). Data flows through Pub/Sub into BigQuery for real-time analytics, while Cloud Logging and Cloud Monitoring deliver observability at scale.

Why it matters: With its new microservices-first design, Manhattan gained the agility to evolve faster and the confidence that mission-critical operations would remain resilient across regions. With Cloud SQL and BigQuery, the company now processes more than a billion daily API calls with average response times of less than 150 milliseconds. This evolution supports hundreds of thousands of monthly active users across tens of thousands of stores and distribution centers. The new platform also created the foundation for Manhattan’s Agentic AI suite, which includes prebuilt agents — like the Intelligent Store Manager and Labor Optimizer — that coordinate real-time decisions across store and distribution center operations.

Learn from us: "Operationally, the platform has become more elastic and efficient. The system automatically handles hundreds of thousands of scaling events per day, ensuring performance remains consistent during peak surges without expensive overprovisioning." — Narayana Reddy Kothapu, Senior Director, Manhattan Associates & Rajkumar Ramani, Technical Director, Manhattan Associates

Introducing BigQuery Graph: Unlock hidden relationships in your data

Tue, 14 Apr 2026 16:00:00 +0000

Today, we're thrilled to announce that BigQuery Graph is now available in preview. With BigQuery Graph, we’ve built an easy-to-use, highly scalable graph analytics solution for data engineers, data analysts, data scientists, and AI developers, empowering them to model, analyze and visualize massive-scale relationships in an entirely new way.

As data changes and grows, it’s important to understand how different entities such as people, places, and products relate to one another. After all, data is more meaningful when we know how entities are interconnected. With traditional SQL, to find a "friend of a friend of a friend" requires multiple nested JOIN operations, which are usually challenging to read and write, and the performance degrades exponentially at scale. Finding the "blast radius" of a supply chain disruption during a storm requires multi-hop traversals, a full-scale graph analysis. To better solve this challenge, data is often modeled as a graph representation of the physical world around us, which can be better at finding complex and hidden relationships than traditional relational data structures.

Graph challenges faced by enterprises across industries

Graph technology has been broadly used across industries for fraud detection, recommendation engines, supply chain management, knowledge graph applications, and many others. However, users face some major challenges in adopting graphs:

Data silos and maintenance overhead: Having to store and maintain graph data in a standalone graph database creates data silos, data inconsistency, additional cost — not to mention extra operational overhead.
Lack of graph expertise: Adopting graph technologies often requires learning a new language, paradigm, and potentially a new database. At the same time, organizations’ investment in SQL expertise are less relevant.
Performance and scalability concerns: Many standalone graph databases work well when traversing the graph from a handful of nodes, but struggle to scale to billions of entities as business demands grow.

BigQuery Graph addresses many of these challenges by supporting:

Built-in graph query experience: A more intuitive graph query language (GQL) allows you to find patterns and traverse relationships among disparate data sets, based on the newest ISO GQL standard.
Unified relational and graph data models: Tight integration between graph and relational data models allows you to choose the best tool to model the data on a single source of truth without data duplication or data movement. Full interoperability between graph queries and SQL allows you to continue to leverage existing SQL skills, while taking advantage of the expressiveness of graph queries.
Graph over structured and unstructured data: Rich AI functions, vector and full-text search capabilities are supported with BigQuery Graph, allowing you to use semantic meaning, keyword search on graphs, bridging the gaps of structured and unstructured data.
Graph visualization: You can easily explore, investigate, and explain how your data is connected in an intuitive graph format using BigQuery Studio notebook and Jupyter Notebook.
Industry-leading ease of use, performance and scalability: BigQuery Graph is built upon BigQuery's serverless, scalable, cost-effective and distributed analytics engine that can scale to billions of nodes and edges.
Integration with Spanner Graph: This provides a unified graph schema and graph query language that serve a full spectrum of real-time (Spanner Graph) and batch graph needs (BigQuery Graph). You can also build a virtual graph by combining the latest data from Spanner and historical data from BigQuery without data movement using federated queries.
Chat with your graph: Very soon, you will be able to chat directly with graphs with the Conversational Analytics Agent (stay tuned).

Common BigQuery Graph use cases

BigQuery Graph opens up a realm of possibilities across industries for building intelligent applications:

Financial fraud detection: Analyze complex relationships among users, accounts, and transactions to identify suspicious patterns and anomalies, such as money laundering and irregular connections between entities, which can be difficult to detect using relational databases.
Customer 360: Track customer relationships, preferences, and purchase histories. Gain a holistic understanding of each customer, enable personalized recommendations, targeted marketing campaigns, and improved customer service experiences.
Social networks: Capture user activities and interactions and use graph pattern matching for friend recommendations and content discovery.
Manufacturing and supply chain management: Use graph patterns for efficient stockout analysis, cost rollups, and compliance checks by modeling parts, suppliers, orders, availability, and defects in the graph.
Healthcare: Capture patient relationships, conditions, diagnosis, and treatments to facilitate patient similarity analysis and treatment planning.
Transportation optimization: Model places, connections, distances, and costs in the graph, and then use graph queries to find the optimal route.

BigQuery Graph in the real-world

Many customers across industries have leveraged BigQuery Graph to solve real-world business challenges. Here are a few examples of how they are putting these capabilities into practice:

BioCorteX: drug discovery

"Understanding disease isn't about collecting more data; it’s about understanding the relationships within that data. By leveraging pathway search in BigQuery Graph at a massive scale, reaching depths of 7+ hops, we are finally able to see more of the human metabolism map. This level of scale is what allows us to move beyond trial and error, identifying the precise biological levers that need to be pulled to cure complex diseases. We aren't just guessing anymore, we’re emulating life at the speed of compute." - Nik Sharma, CEO and Cofounder, BioCorteX

Curve: fraud detection

"By implementing BigQuery Graph, we have successfully moved away from the previous limited sql-based approach to a more scalable solution for fraud detection network analysis. This has allowed us to detect sophisticated fraud networks by uncovering hidden connections between seemingly unrelated accounts and transactions. This transition from traditional relational queries to graph-based analytics has showcased measurable business impact with ~£9.1M of savings. This shift has not only improved the precision of fraud detection but has also provided a scalable foundation for protecting the ecosystem without adding significant operational overhead." - Francis Darby, VP Data & ML, Curve

Virgin Media 02: fraud detection

"At Virgin Media O2, we are constantly evolving our defenses to stay ahead of increasingly sophisticated fraud networks. We’ve added a powerful new layer to our already robust fraud alerting system. Using BigQuery Graph, we can now execute complex 4-hop queries that map the hidden relationships between accounts, devices, and activities. This deeper visibility allowed us to identify networks of suspicious addresses. This doesn't just catch fraud; it acts as an early warning system, flagging new connections to known risk networks before they can cause damage." -- Jonathon Ford, Director Data Applications, Virgin Media O2

How to use BigQuery Graph

BigQuery Graph is more than just a new feature; it's a new way of thinking about data, empowering you to ask bigger questions, uncover deeper insights, and solve your most challenging problems.

You can get started in three simple steps:

Step 1: Create graph schemas on top of the relationship tables using DDL with a single copy of data.

Create a finance graph by mapping relational tables into “Account”, “Person”, "Loan" nodes and their relationships “Transfers”, “Owns”, "Repays" via edges.

code_block: <ListValue: [StructValue([('code', 'CREATE PROPERTY GRAPH graph_db.FinGraph\r\nNODE TABLES (\r\n graph_db.Account KEY(id),\r\n graph_db.Person KEY(id),\r\n graph_db.Loan KEY(id)\r\n) \r\nEDGE TABLES (\r\n graph_db.Transfers \r\n KEY (id, to_id, timestamp) \r\n SOURCE KEY (id) REFERENCES Account (id)\r\n DESTINATION KEY (to_id) REFERENCES Account (id), \r\n graph_db.Owns\r\n KEY (id, account_id, timestamp) \r\n SOURCE KEY (id) REFERENCES Person (id)\r\n DESTINATION KEY (account_id) REFERENCES Account(id),\r\n graph_db.Repays\r\n KEY (id, loan_id, timestamp) \r\n SOURCE KEY (id) REFERENCES Person (id)\r\n DESTINATION KEY (loan_id) REFERENCES Loan(id)'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b52f88820>)])]>

Step 2: Use intuitive SQL/GQL to traverse data relationships and find hidden connectivities.

Find the accounts owned by Jacob and the loans he repays from those accounts:

code_block: <ListValue: [StructValue([('code', 'GRAPH graph_db.FinGraph\r\nMATCH\r\n (person:Person {name: "Jacob"}) \r\n -[own:Owns]->(account:Account)\r\n -[repay:Repays]->(loan:Loan)\r\nRETURN\r\n account.id AS account_id,\r\n loan.id AS loan_id'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b52f888e0>)])]>

Combine vector search with graph traversals to find fraudster-like accounts and their transfer activities within 1-6 hops:

code_block: <ListValue: [StructValue([('code', "DECLARE similar_account_to_fraudster DEFAULT ((\r\n SELECT array_agg(base.id)\r\n FROM VECTOR_SEARCH(TABLE graph_db.Account, 'embedding',\r\n (SELECT * FROM graph_db.Account WHERE id=102), 'embedding', \r\n top_k => 6)\r\n));\r\nGRAPH graph_db.FinGraph\r\nMATCH\r\n (person:Person)-[own:Owns]->\r\n (account:Account)-[transfer:Transfers]->{1,6}\r\n (to_account:Account)\r\nWHERE to_account.id IN \r\n UNNEST(similar_account_to_fraudster)\r\nRETURN\r\n person.id AS person_id,\r\n account.id AS src_account,\r\n to_account.id AS to_account"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b52f884c0>)])]>

Step 3: Visualize graph results to detect connectivity of disparate data in a more intuitive way in BigQuery Studio notebook.

If you are looking for a specialized graph visualization tool, BigQuery Graph has integrated with industry leading partners including G.V(), Graphistry, Kineviz, Linkurious. They allow you to see a visualization of BigQuery Graph query results outside the Google Cloud console.

Ready to get started?

The future of data analysis is connected. With BigQuery Graph, you have the power to unlock that connectivity and transform your business into actionable insights grounded with your enterprise knowledge. Start exploring today and unleash the power of your data's interconnected relationships!

Visit the BigQuery documentation: find overview and quickstart guide.
Explore tutorials: get hands-on experience with BigQuery Graph through tutorials.
Share your feedback: join our community, and get your questions answered via bq-graph-preview-support@google.com.
Related blogs:

From operational to analytical: The unified Spanner Graph and BigQuery Graph solution

Tue, 14 Apr 2026 16:00:00 +0000

Understanding the relationships within your data is crucial for uncovering hidden insights and building intelligent applications. However, managing operational (OLTP) and analytical (OLAP) graph workloads usually means wrestling with disconnected databases, building brittle data pipelines, and managing complex integrations. This fragmentation creates data silos, increases operational overhead, and limits scalability.

Today, we're thrilled to introduce a unified graph database and analytics solution powered by Spanner Graph and BigQuery Graph. The solution consists of the two platforms, recommended blueprints for how to deploy them, and getting started guides for the most prominent use cases. In this blog, we review the solution’s components, provide an overview of the most common use cases, and hear from customers who have deployed the solution in the real world.

Spanner Graph for operational workloads

Spanner Graph reimagines graph data management, bringing together graph, relational, search, and generative AI capabilities into a single database. It is backed by Spanner’s signature unlimited scalability, high availability, and strong consistency.

With Spanner Graph, you get:

Integrated table-to-graph mapping: Define graphs directly over your existing Spanner relational tables, allowing you to view and query operational data as a graph without data duplication.
Interoperable graph and relational querying: Leverage an ISO-standard Graph Query Language (GQL) interface for intuitive pattern matching, and mix GQL with SQL in a single query to traverse both graph and tabular data together.
Advanced search and AI integration: Utilize built-in vector search, full-text search, and Vertex AI integration to retrieve data by semantic meaning and power intelligent applications directly within your database.

Customers are already using Spanner Graph to power high-throughput, low-latency applications - for identity resolution across millions of entities, identifying dependencies across vast complex environments, data lineage, customer 360 use-cases, and enhancing real-time fraud detection.

"Open Intelligence is our foundational intelligence layer that securely connects trillions of live data points from clients, partners and WPP in a privacy-first way and is now integrated and powers WPP’s agentic marketing platform, WPP Open. Enabled by Google Cloud's Spanner Graph, Open Intelligence is a significant advancement in AI-driven marketing and we are excited about extending the use case for analytical graph workloads on BigQuery Graph." - Rob Marshall, Head of Strategy, Data & Intelligence, WPP

BigQuery Graph for analytical workloads

While Spanner Graph handles your active operations, true large-scale analysis requires exploring relationships across billions of nodes and edges to identify patterns and query historical data. Just as SQL relies on distinct tools for databases and data warehouses, the graph landscape requires specialized tools for different workloads. That's why we built BigQuery Graph.

BigQuery Graph brings connected data analytics directly into your data warehouse. You can map existing BigQuery data to a graph schema and query it with SQL or GQL to uncover hidden relationships in massive datasets - without moving any data.

Key capabilities include:

Integrated table-to-graph mapping: Map your existing BigQuery tables to graphs instantly, uncovering hidden relationships in your data warehouse without building ETL pipelines or moving a single byte of data.
Interoperable graph and relational querying: Apply the same expressive pattern matching of GQL to massive historical datasets, and mix SQL with GQL in a single query to combine the familiarity of your data warehouse with powerful graph traversal.
Advanced search and AI integration: Leverage native integration with BigQuery AI for predictive analytics, alongside built-in vector search, full-text search, and geospatial functions to locate connected information across billions of records.

Spanner Graph and BigQuery Graph as a unified solution

While each platform is powerful on its own, their true value emerges when they are deployed together. By connecting your operational and analytical environments, you eliminate data silos and accelerate your time-to-insight without compromising database performance.

"Spanner Graph enables Yahoo to unify our data into a connected foundation at a global scale, powering real-time, intelligent decision-making across our agentic advertising platform. This enhances our AI-driven approaches that drive one of the largest digital advertising ecosystems, and we look forward to building on it with BigQuery Graph to unlock deeper analytics and predictive capabilities to power future innovation." - Gabriel DeWitt, Head of Consumer Monetization, Yahoo

Take financial fraud detection as an example: your application can use Spanner Graph to instantly identify a suspicious connection and block a transaction at checkout. Meanwhile, BigQuery Graph can analyze petabytes of historical transaction data to expose the complex, long-term fraud ring that initiated it.

Here is how these two engines integrate to create an end-to-end graph workflow:

1) A unified graph query and schema experience

A core advantage of this solution is the consistent schema and GQL shared across both platforms. This shared language reduces development time and minimizes the friction of context-switching.

For example, to find potential fraud rings originating from a specific account in real-time, you would use this Spanner Graph query:

code_block: <ListValue: [StructValue([('code', 'GRAPH FinGraph\r\nMATCH p=(:Account {id: @accountId})-[:Transfers]->{2,5}(:Account)\r\nRETURN PATH_LENGTH(p) AS path_length, TO_JSON(p) AS full_path;'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b6f32ff10>)])]>

To run that same analysis to find all accounts involved in historical fraud rings, the BigQuery Graph query is nearly identical:

code_block: <ListValue: [StructValue([('code', 'GRAPH bigquery.FinGraph\r\nMATCH p=(:Account)-[:Transfers]->{2,5}(:Account)\r\nRETURN PATH_LENGTH(p) AS path_length, TO_JSON(p) AS full_path;'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b6f32f1c0>)])]>

2) Query Spanner data in BigQuery Graph through Data Boost

With Data Boost, you can query Spanner Graph data directly from BigQuery without impacting performance of your transactional workloads. This allows you to build a "virtual graph" combining real-time operational data from Spanner with historical analytics in BigQuery - without moving any data.

For instance, you can combine real-time Account and User nodes from Spanner Graph with historical LogIn edges from BigQuery to identify suspicious login patterns across different devices.

To do this, you first connect BigQuery to Spanner using the CREATE EXTERNAL SCHEMA statement:

code_block: <ListValue: [StructValue([('code', "CREATE EXTERNAL SCHEMA spanner\r\nOPTIONS (\r\n external_source = 'google-cloudspanner:/projects/PROJECT_ID/instances/INSTANCE/databases/DATABASE',\r\n location = 'LOCATION'\r\n);"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b6f32fd60>)])]>

Next, define a BigQuery Graph, incorporating tables from both Spanner and BigQuery:

code_block: <ListValue: [StructValue([('code', 'CREATE OR REPLACE PROPERTY GRAPH bigquery.FinGraph\r\n NODE TABLES (\r\n -- Account and Person are stored in Spanner,\r\n -- made available in BigQuery through the `CREATE EXTERNAL SCHEMA` statement.\r\n spanner.Account KEY (account_id),\r\n spanner.Person KEY (person_id),\r\n -- Media is stored in BigQuery.\r\n bigquery.Media KEY (media_id)\r\n )\r\n EDGE TABLES (\r\n -- Transfers and Owns are stored in Spanner.\r\n spanner.Transfers AS Transfers\r\n KEY (transfer_id)\r\n SOURCE KEY(account_id) REFERENCES Account\r\n DESTINATION KEY(target_account_id) REFERENCES Account,\r\n spanner.Owns AS Owns\r\n KEY (person_id, account_id)\r\n SOURCE KEY(person_id) REFERENCES Person\r\n DESTINATION KEY(account_id) REFERENCES Account,\r\n -- LogIn is stored in BigQuery.\r\n bigquery.LogIn AS LogIn\r\n KEY (login_id)\r\n SOURCE KEY(media_id) REFERENCES Media\r\n DESTINATION KEY(account_id) REFERENCES Account,\r\n );'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b6f32f3a0>)])]>

Finally, execute a query on BigQuery Graph to access data across both Spanner (accounts, users, transfers, owns) and BigQuery (logins, devices), identifying potentially suspicious login patterns:

code_block: <ListValue: [StructValue([('code', 'GRAPH bigquery.FinGraph\r\nMATCH p=(owner:Person)-[:Owns]->\r\n (:Account)<-[login:LogIn]-\r\n (media:Media {blocked: true})\r\nRETURN TO_JSON(p) AS full_path\r\nORDER BY login.time\r\nLIMIT 20;'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b6f32fbb0>)])]>

3) Export BigQuery data into Spanner Graph through reverse ETL

When you need to bring analytical data back into Spanner for low-latency, real-time querying, you can use reverse ETL without additional pipelines. For example, you can import historical device data (IP addresses, device IDs) from BigQuery into Spanner Graph to enhance your real-time fraud detection operations:

code_block: <ListValue: [StructValue([('code', 'EXPORT DATA\r\n OPTIONS (\r\n uri = \'https://spanner.googleapis.com/projects/PROJECT_ID/instances/INSTANCE/databases/DATABASE\',\r\n format=\'CLOUD_SPANNER\',\r\n spanner_options="""{ "table": "Media" }"""\r\n ) AS \r\nSELECT * FROM Media;'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b6d18c370>)])]>

4) Visualize your graph data

Visualizing connected data is core to analysis, explorations and investigations. With Spanner Studio and BigQuery Studio (coming soon), you can instantly visualize your graph data without leaving your familiar environment or setting up external tools. For deeper programmatic exploration, you can also leverage Spanner Graph notebook and BigQuery notebooks to render query results directly within your existing data science workflows.

5) Graph visualization partner integrations

Spanner Graph and BigQuery Graph also integrate with leading graph visualization partners to provide a comprehensive suite of exploration tools:

Kineviz: Combines cutting-edge visualization with advanced analytics via GraphXR.
Graphistry: Extracts meaningful insights from large datasets using a GPU-accelerated visual graph intelligence platform.
G.V(): Offers a quick-to-install client for high-performance visualization and no-code data exploration.
Linkurious: Detects and analyzes threats in large volumes of connected data via the Linkurious Enterprise platform.

One unified solution for all your graph needs

Together, Spanner Graph and BigQuery Graph provide a unified solution for operational and analytical needs across various use cases:

Domains	Spanner Graph	BigQuery Graph
Financial Services	Instantly blocks anomalous, suspicious transactions.	Uncovers complex, long-term fraud rings.
Retail & E-commerce	Serves personalized product recommendations on the fly.	Analyzes vast purchasing histories to predict demand.
Cybersecurity	Isolates active threats and traces attack origins instantly.	Models historical vulnerabilities to strengthen defenses.
Healthcare	Powers clinical decision support systems at the point of care.	Analyzes population health trends and disease risk factors.
Supply Chain	Tracks goods globally and alerts teams to immediate disruptions.	Identifies systemic bottlenecks to optimize future routing.
Telecommunications	Creates a network digital twin for detecting anomalies, and root cause analysis in real-time.	Analyzes traffic patterns at scale to plan future infrastructure upgrades.

Get started with Spanner Graph and BigQuery Graph today

With Spanner Graph and BigQuery Graph, we’re excited to offer a unified graph data management experience across your operational and analytical needs.

Explore Spanner Graph's use cases and setup guide for your operational workloads, and the BigQuery Graph overview and creation guide for your analytical needs. To experience the full power of this combination, check out our unified solution guide and try the codelab.

Scaling unstructured enterprise knowledge with BigQuery Graph, and Kineviz GraphXR

Tue, 14 Apr 2026 16:00:00 +0000

Over 80% of enterprise data lives in unstructured form — PDFs, emails, reports, regulatory filings. Most of the time, such sources contain critical business information, yet they remain difficult to access and reason over at scale. Together, BigQuery Graph and Kineviz GraphXR give decision makers power over their unstructured data by creating a single, streamlined workflow that makes it much easier to uncover hidden business insights. BigQuery houses and builds the structures of the graph; Kineviz GraphXR lets analysts visually verify relationships, trace insights back to sources, and answer questions interactively.

Retrieval-augmented generation (RAG) and vector search have become the industry standard approach for working with unstructured data. When it comes to trend analysis, comparison across entities, multi-hop reasoning, and explainable decision support, graphs complement RAG by incorporating context and relationship mapping. Our "evidence-first" knowledge graph approach prioritizes preserving the nuance of the original evidence and maintaining the traceability of every single element in the graph, making the resulting analysis verifiable and trustworthy. In this post, we describe an example where BigQuery AI Functions, BigQuery Graph, and Kineviz GraphXR address business questions about Fortune 500 SEC filings without complex ETL pipeline, data duplication, or separate graph databases.

From fragmented to unified with BigQuery

Traditional unstructured analytics pipelines can be complex and sprawling. They typically involve multiple steps, including: object storage for raw files, a custom parsing service, a separate AI extraction layer, a standalone graph database, and finally, a BI tool for analysis. This complex setup can be difficult to maintain, involving data duplication, synchronization overhead, introducing multiple potential points of failure.

BigQuery streamlines this process. Raw documents are stored in Google Cloud Storage, and text extraction, Gemini-powered inference, and graph creation all run directly within the same platform. This removes the need for data movement between systems, complex service orchestration, or the accumulation of out-of-sync data copies.

With its tight integration, the pipeline is simple and maintainable, allowing full provenance without bespoke infrastructure.

BigQuery pipeline: From unstructured to structured

We used BigQuery pipeline to explore SEC 10-K filings of Fortune 500 companies from 2020 to 2024. Each filing includes around 100 pages of detailed, descriptive information.

We designed a schema such that each Company connects to Competitors (COMPETES_WITH), Risks (FACES_RISK), and Markets (ENTERING / EXITING / EXPANDING), and followed the following four-step process.

1. Ingest and parse. Retrieve 10-K filings from SEC EDGAR, transform Standard Generalized Markdown Language (SGML) to Markdown while preserving hierarchical structure, and load the raw text into BigQuery via Cloud Storage.

2. Focus on key signal sections. Instead of processing complete 100-page filings, we focus extraction only on sections related to market moves, risks, and competitors (specifically the Business, Risk Factors, and MD&A sections). Every row in BigQuery preserves essential metadata, including the year, company, CIK, section ID, and the direct URL to the original source filing.

3. Gemini for extraction. Utilizing AI.GENERATE_TEXT() with Gemini 3 Pro, each section is processed to return structured JSON. This output details competitors, risks, market actions, and opportunities, with every element grounded by evidence text from the initial filing. This process is completed entirely within BigQuery, with no external orchestration or data movement.

4. Declaring the graph. The structured JSON data is then broken down into distinct tables for nodes and edges. These tables are subsequently mapped into a fully traversable graph using a single Data Definition Language (DDL) statement, as shown below, enabling graph queries without the need for joins.

code_block: <ListValue: [StructValue([('code', 'CREATE PROPERTY GRAPH sec_filings.SecGraph\r\n NODE TABLES (\r\n nodes_company, nodes_competitor, nodes_risk, nodes_market, nodes_opportunity\r\n )\r\n EDGE TABLES (\r\n edges_competes SOURCE nodes_company DESTINATION nodes_competitor LABEL COMPETES_WITH,\r\n edges_faces_risk SOURCE nodes_company DESTINATION nodes_risk LABEL FACES_RISK,\r\n edges_entering SOURCE nodes_company DESTINATION nodes_market LABEL ENTERING\r\n -- plus EXITING, EXPANDING, PURSUING\r\n );'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b6d0aeca0>)])]>

The process extracted 87,000 entities and over 20,000 mentions of competitors. After resolution and normalization, these mentions were consolidated into roughly 8,100 distinct competitors, turning unstructured SEC filings into a knowledge graph for competitive landscape.

Unlocking hidden insights with Kineviz GraphXR

GraphXR, by Kineviz, connects directly to BigQuery Graph, providing the environment for analysts to explore and analyze the data interactively. Analysts can visually navigate relationships and drill into subgraphs through low-code workflows, without needing to write any queries. This means strategy, compliance, and research teams can work directly with the data and refine their analyses themselves.

GraphXR’s AI-assisted workflows allow users to define analytical tasks using natural language, such as "show me Apple’s competitive trajectory over time", generating dashboards linked to a live graph view. As the graph view changes, dashboard charts update dynamically. For example, structured data points extracted from SEC filings reveal that the number of companies that cited Apple as a competitor has remained relatively stable at around 14 over time, a pattern not apparent when examining individual filings.

Dashboard: Companies Citing Apple Over Time

The AI-powered Visual Analysis Agent enhances the accuracy and nuance of these assessments. For instance, after using GraphXR's "trace neighbor" function to identify companies that cite Google as a competitor, the Agent's analysis reveals complex cross-industry relationships. A key example is AES Corp., an energy utility, which appears in contexts indicating coopetition relationships, illustrating the broader market shift toward adopting cloud and AI infrastructure.

Competitive analysis with agent reasons over both graph structure and node properties

Our workflow places a strong emphasis on auditability. Every node in the graph is directly linked to its source within the original SEC filings. Analysts can trace insights back to their origin and validate findings in context. For example, in the image below, selecting a risk entity provides a URL link that takes the reader to the relevant location in the document where that specific risk was identified.

Risk analysis with a direct, clickable link to the precise location of the extracted information in the source document.

Why this matters

BigQuery Graph together with Kineviz GraphXR provide organizations with:

Simplicity: Fewer systems, fewer copies — the pipeline runs in a fully-managed, integrated platform where data stored in BigQuery gets explored and analysed in GraphXR without data movement or duplication.
Scalability: BigQuery handles millions of documents and billions of extracted facts without bespoke graph infrastructure.
Explainability: Every insight traces back to source evidence; validation is one click.
Flexibility: New questions or entity types don't force you to rebuild the extraction model — you can just extend the schema.

The majority of enterprise knowledge is locked away. Together, BigQuery AI Functions, BigQuery Graph, and Kineviz BI tools provide an end-to-end solution that turns graph-based reasoning, evidence-first analytics, and interactive exploration into a single, streamlined pipeline that unlocks the intelligence trapped within unstructured data.

Get started

Learn more about BigQuery Graph here and get started here. Kineviz GraphXR is available on the Google Cloud Marketplace. You can find the Fortune 500 tutorial in the GitHub notebook or watch the video here.

Related blogs:

Accelerating data curation with Google Data Cloud

Fri, 10 Apr 2026 17:00:00 +0000

In the enterprise landscape, data is often highly fragmented across multiple source systems. Data curation is the process of organizing, cleaning, and enriching raw data to transform it into high-quality, AI-ready data assets. The traditional process of merging and cleaning this data using ETL tools, manual SQL or Python to build dashboards is the primary bottleneck for AI and analytics.

Google Data Cloud provides several curation accelerators designed to reduce the time-to-insight and automate these workflows.

1. Cloud Storage auto-discovery for semi-structured data

The first step in modern curation is eliminating the manual effort of cataloging dark data in Cloud Storage.

Automatic data discovery: The automatic discovery feature in Dataplex Universal Catalog scans GCS buckets to automatically create external tables for structured data and catalog the metadata.
Ad-hoc analysis: This allows for immediate, Gemini-powered analysis via vibe querying to assess value and quality without having to load the data with a traditional ETL process.
Unified governance: This also lets you apply fine-grained access control and automated metadata generation directly on the raw storage layer, ensuring security and governance are baked in right from the start.

2. Metadata curation and augmentation

Curation acceleration relies on moving from columns and rows to a semantic understanding of the data.

Automated insights: Data insights automatically generates column descriptions, relationship graphs, along with suggested questions in natural language. This helps speed up metadata documentation and accelerate initial exploration and analysis when facing new or unfamiliar data.
Grounding Conversational Analytics: These insights later serve to ground conversational analytics in your data, giving agents the additional context to understand how assets relate to your business. This ensures more accurate responses when you chat with your data using natural language.

3. Integrated governance: Quality, profiling, and lineage

Trusted curation requires a robust metadata framework that tracks data health and movement.

Data profiling: Data profiling automatically identifies statistical characteristics (e.g., null counts, distribution) to catch anomalies early.
Quality Controls: Users can define and run data quality checks to ensure that data meets organization's quality standards. Auto data quality lets users automate scans, validate data against rules, and log alerts if the data doesn't meet quality requirements.
Lineage tracking: Table- and column-level lineage, allows engineers to trace how data moves through transformations. This transparency accelerates curation making it easier to debug pipeline errors.

4. Agentic workflows for pipeline development

Google Data Cloud introduces AI agents to handle the heavy lifting of code generation for ingestion and transformation.

Data Engineering Agent: This agent allows you to use Gemini in BigQuery to build and manage pipelines using natural language or by passing a technical design document.
Data Science Agent: Integrated into Colab Enterprise/BigQuery Notebooks, Data Science Agent automates exploratory data analysis (EDA) and generates Python/PySpark code for complex ML-ready pipelines.

5. Catalog-driven asset discovery and data products

To prevent redundant work in large organizations, curation must focus on reuse and internal marketplaces.

Discovery first: Before building new pipelines, teams use the Dataplex Data Catalog to discover existing assets.
Data products: Data is published as data products enriched with logical grouping of data assets, formally packaged to be discoverable, trusted, and accessible for solving specific business problems.
BigQuery sharing (formerly Analytics Hub): This enables in-place sharing, allowing internal and 3rd party teams to access curated data without moving or copying it, which maintains a single source of truth.

6. Built-in AI functions for multi-modal data curation

As enterprises generate increasing amounts of multi-modal data, curation now extends to unstructured formats like images, audio, and documents. The following capabilities address these evolving needs:

SQL reimagined with generative AI functions: By using standard SQL operators, data teams can classify and rank data by quality or criteria without specialized ML expertise. BigQuery AI functions allow users to perform sentiment analysis, summarization, and entity extraction directly within a SQL statement.
Embeddings generation: Curation pipelines can now generate vector embeddings to enable use cases like similarity searches, product recommendations, log analytics, entity resolution and deduplication and more across massive datasets.
Multimodal tables: Multimodal tables let you Integrate unstructured data into standard tables and work with multimodal data with SQL.

7. Real-time curation with continuous queries

For real-time curation, BigQuery provides simplified experience enabling no-code ingestion and SQL based transforms for constant data movement.

Pub/Sub to BigQuery: Direct subscriptions allow for no-code ingestion of streaming data into BigQuery tables.
Continuous queries: Continuous queries are SQL statements that run continuously, processing incoming data in real-time. Curated output can be immediately streamed to Pub/Sub, Bigtable, or Spanner to power downstream applications and real-time dashboards.

In summary, these curation accelerators remove the slow, manual work of cleaning and organizing data by automating the most time-consuming steps. Spend less time prepping and more time making decisions — explore these curation accelerators today to get started.

Data Studio returns as new home for Data Cloud assets

Fri, 10 Apr 2026 16:00:00 +0000

In today's data-rich environment, organizations possess vast amounts of information. Yet, bridging the gap between that data and the users who need to make daily, informed decisions remains a challenge. Users need a single place to curate and analyze their data from the many different sources that impact their business each day.

We are sharing the next step in our mission to solve this challenge and reintroducing a beloved and familiar name, Data Studio (formerly Looker Studio).

In addition to its powerful data visualization capabilities, Data Studio is playing a significant role in the AI era serving Google Data Cloud content. With Data Studio, you have a single place to browse and interact with a variety of Google data sources and assets — from Data Studio reports, to BigQuery conversational agents, to data apps built in Colab notebooks.

Data Studio: reports, data apps, and conversational agents in one place

Extending our vision for analytics in the AI era

Since bringing Data Studio to the Google Cloud family five years ago, customers have continued to innovate with Data Studio as a place to visualize and share their data assets. Meanwhile, as AI becomes a critical component of practically every business, we’ve heard from our customers that they need a single place to save, organize and browse their data assets.

As part of this reintroduction, with Looker as our enterprise business intelligence platform, we are evolving Data Studio to complement the Looker platform, independently. As we have redesigned Data Studio, Looker has also recently seen significant investments in its self-service and visualization offerings, including agentic capabilities for use cases that demand trusted, governed data powered by a central semantic model.

We believe the new Data Studio is the ideal choice for personal data exploration — a place to craft ad-hoc reports, and quickly visualize data across Google’s ecosystem, from BigQuery to Google Sheets and Ads. This strategic differentiation ensures customers have the right tool for the right job.

Two flavors: Data Studio and Data Studio Pro

The new Data Studio experience is available in two editions.

Data Studio continues to offer powerful, no-cost individual analysis and visualization, serving as the on-ramp for creating and sharing ad-hoc reports, transforming data to an interactive dashboard in minutes.
Data Studio Pro is designed for scaling teams and organizations that need more agility and control, including AI features and deep integration with Google Cloud for enterprise-grade security, management, and compliance capabilities. Pro licenses can be purchased directly from the Google Cloud console or the Google Workspace Admin Console.

Upgrading to the new Data Studio should be largely transparent for the many users who count on this product in their daily work. All existing reports, data sources, assets and users will be transitioned to the new experience with no action on your part. Learn more about what’s coming to Data Studio and our vision for Data Cloud and Analytics at Google Cloud Next ‘26 later this month.

Openness without compromises for your Apache Iceberg lakehouse

Wed, 08 Apr 2026 16:00:00 +0000

Today, at the Apache Iceberg Summit in San Francisco, we are announcing the preview of read and write interoperability between BigQuery and Iceberg-compatible engines, including Trino, Spark, and others in Apache Iceberg tables in Google-managed Iceberg REST Catalog. With this new capability, you get the benefits of enterprise-grade native storage for your lakehouse without sacrificing Iceberg's openness and flexibility.

Why it matters: If you're building a lakehouse today, you're probably using Apache Iceberg, which has gained massive popularity among data platform teams that need to support multiple compute engines (like Spark and BigQuery) that access the same data for different workloads. However, we consistently hear from customers that achieving openness often requires compromises.

Compared to using enterprise storage, there’s often price-performance overhead on using Iceberg, wiping out the cost benefits of a single-copy architecture. In order to make Iceberg work for all production use cases, data teams have to invest in custom infrastructure to handle real-time streaming, build complex pipelines to replicate operational data, and navigate fragmented governance across different compute engines. Ultimately, these limitations become bottlenecks to innovation.

Over the years, Google has purpose-built storage infrastructure to solve these exact challenges at scale, powered by highly scalable, real-time metadata, unified governance, and deep vertical integration across Cloud Storage, metadata, and various query engines. We are making this infrastructure available directly in Iceberg.

This enables access to BigQuery's advanced runtime, automatic table management, partitioning, multi-statement transactions, and change data replication for Google-managed Iceberg REST catalog tables. These features will be available in preview for Google-managed Iceberg REST catalog tables and will be generally available (GA) for BigQuery-managed Iceberg tables, coming next month.

Write and read interoperability across engines

Previously, customers building lakehouses chose between Iceberg tables in the Google-managed Iceberg REST catalog or tables managed by BigQuery based on their primary ETL engine. That means that customers relying on Apache Spark for ETL to Iceberg REST Catalog tables couldn’t write through BigQuery or use its storage management features.

With this preview, you can create, update, and query Iceberg tables in the Google serverless Iceberg REST catalog with BigQuery or other Iceberg-compatible engines such as Spark, Flink, Trino and others. This two-way read and write interoperability enables data teams to implement multi-engine use cases on a single table type in a fully open manner, using native Iceberg libraries.

Additionally, Iceberg REST Catalog offers table-level access controls using credential vending for uniform governance across BigQuery, Spark and other compute engines that query or modify your Iceberg tables.

Google Cloud also supports a robust ecosystem of partners integrated with the Iceberg REST Catalog across data platforms and engines, transformation and ingestion services, and governance platforms. We work closely with the Iceberg ecosystem to strengthen these partnerships with many more to come.

Improved price-performance with BigQuery and Spark

Automate table management

Achieving strong query performance on Apache Iceberg tables out of the box can be hard. You need to choose the optimal target file size (which tends to be different for different compute engines), data organization strategy (partitioning and sort-order choices have their tradeoffs), and take care of table management to avoid small files problems and metadata bloat.

Apache Iceberg lakehouse customers can now offload table maintenance — compaction and garbage collection — to Google Cloud BigLake, which optimizes performance for you. In addition to Iceberg tables in BigQuery, it will be available for Google-managed Iceberg REST catalog tables in preview, coming next month. You can opt-in to table management by setting a single property, and improve your BigQuery performance on the industry standard TPC-DS 10T benchmark by ~40%.

Improve BigQuery price-performance with advanced runtime

BigQuery advanced runtime offers a set of performance enhancements designed to automatically accelerate analytical workloads without requiring user action or code changes. In particular, it extends the vectorized query execution enhancements in BigQuery to open table formats. Advanced runtime will be available in preview for Google-managed Iceberg REST catalog tables and in GA for BigQuery-managed Iceberg tables, coming next month. According to an internal TPC-DS 10T benchmark, advanced runtime can help additionally accelerate BigQuery query performance on Iceberg tables, providing 2x faster performance vs. a self-managed approach based on internal benchmarking.

Chart based on benchmarks from internal data and testing.

Accelerate Spark performance with Lightning Engine

Apache Spark is a leading compute engine for Apache Iceberg lakehouses, for use cases ranging from ETL to feature engineering. However, achieving high performance and cost efficiency for Spark workloads at scale can be challenging. Lightning Engine accelerates Apache Spark query performance by over 4 times compared to open source Spark (based on a TPCH-like benchmark).

Optimize table layout with BigQuery partitioning and clustering

Many open-source libraries and engines rely on Iceberg table partitioning for effective data pruning. BigQuery time-based partitioning will be available in preview for Google-managed Iceberg REST catalog tables and will be generally available (GA) for BigQuery-managed Iceberg tables, coming next month. Additionally, when you are creating Iceberg tables in BigQuery, you can define clustering columns to organize data in Parquet files, helping to achieve optimal query performance and avoiding common issues with partitioning such as high-cardinality columns, small partition inefficiencies, and multiple filter columns. For example, one common pattern is to combine time-based table partitioning with clustering on other dimensions that are frequently used for query filtering, such as region, store, etc.

Advanced analytics with Apache Iceberg

Streaming with Apache Iceberg

To operationalize real-time analytics with Iceberg, you can leverage BigQuery’s Vortex streaming infrastructure for high-throughput ingestion with zero-read latency. This removes the need for bespoke infrastructure, addresses small file issues, and lets you query data immediately from the streaming buffer to achieve near-zero read latency. This feature is generally available for BigQuery-managed Iceberg tables and will be available in preview for Google-managed Iceberg REST catalog tables, coming next month.

Replicate data from operational databases to Iceberg tables with Datastream

You can now easily replicate data from a variety of operational datastores, including MySQL, Postgres, SQLserver, Oracle, Salesforce, and MongoDB , into managed Iceberg tables in BigQuery using Datastream integration (GA).

Illustration of Datastream creation to replicate MySQL data to managed Iceberg tables in BigQuery.

Incremental processing with change data capture ingestion to Iceberg tables

The BigQuery storage write API’s change data replication feature lets you stream insert, update, and delete changes from OLTP databases to Iceberg tables in real time, removing the need for complex MERGE-based ETL pipelines. This feature will be available in preview for Google-managed Iceberg REST catalog tables and generally available (GA) for BigQuery-managed Iceberg tables, coming next month.

Illustration of change data capture ingestion to a managed Iceberg table in BigQuery.

Multi-statement transactions

Many analytics workloads require transactions that span multiple tables to commit or roll back changes atomically. This provides consistency across logical groups of tables, synchronizes dimensions and fact tables, and simplifies multi-stage ETLs. You can now leverage BigQuery multi-statement transactions to radically simplify complex multi-table processing with Iceberg. This feature will be available in preview for Google-managed Iceberg REST catalog tables and generally available (GA) for BigQuery-managed Iceberg tables, coming next month.

Illustration of a multi-statement transaction in a managed Iceberg table in BigQuery.

Get started

With bidirectional interoperability across BigQuery and other Iceberg-compatible engines on Google-managed Iceberg REST catalog tables, you can modernize your lakehouse with Apache Iceberg without compromising on performance, governance, or advanced analytics.

Ready to start building today? Learn more about our lakehouse capabilities and explore our quickstart guides.

Under one roof: Rightmove reinvents property search with unified data

Tue, 07 Apr 2026 16:00:00 +0000

At Rightmove, we want to make home moving easier for everyone, from house hunters and homeowners to estate agents and brokers. Behind every search, listing, and connection on our platform lies a complex network of users, partners, and properties — and we’ve built our data and AI strategy to serve all three.

To deliver on this mission, we migrated from siloed, on-premises databases to Google Cloud. This move wasn’t just about technology. It was about unlocking smarter, faster, more personalized experiences for our users and partners, and helping them find the right match for each property more efficiently.

Our strategy is guided by four core data and AI value areas:

Delighting consumers with personalized search and discovery
Empowering partners, such as estate agents, with smarter tools and insights
Monetizing data through innovations such as property price prediction
Driving operational efficiency across our platform

Today, we're building this future with a unified analytics and AI stack — BigQuery, Vertex AI, and Looker — that we call “the data hive.” Already, around 300 team members (a third of our workforce) are tapping into its capabilities to turn data into action and insights into impact.

Making it easier to find a home with personalized, dynamic suggestions

When someone’s looking for a new home, they often have a wish list: a garden, a modern kitchen, maybe a home office. We’re using Vertex AI to make that search feel more intuitive and tailored than ever. By extracting metadata from property descriptions and images, we automatically create listing features and keywords, even ones that weren’t manually tagged before, to provide more accurate search results.

We’re also exploring ways to streamline communication. Recently released is an AI-powered feature that uses Gemini to help estate agents respond to inquiries faster. With context-aware, automatically generated replies, agents can keep conversations moving, and potential buyers and sellers can get answers faster, even during the busiest periods.

Helping partners work smarter with AI-powered recommendations

Our partners — which include estate agents,new home developers, mortgage lenders, and other industry professionals — rely on Rightmove to connect with the right audience at the right time. With Vertex AI and Gemini models working behind the scenes, we’re helping them do that more efficiently and effectively.

Take lead generation, for example. We’ve built a vendor scoring engine that analyzes user search patterns and on-site behavior to predict the likelihood that someone is a homeowner. This insight helps partners focus their time and marketing efforts on high-conversion leads, while offering more relevant products — such as mortgage options — to the right people at the right moment.

Next, we’re excited to use generative AI to build agentic, conversational user interfaces, enabling anyone across our network to interact with data or find insights using natural language. Whether it's a business user running a query, or a partner navigating market trends, we’re working toward a more natural, accessible way to engage with data.

Turning data into insight and insight into value

Another exciting way we’re unlocking the value of our data is through an Automated Valuation Model (AVM). This AI-powered tool predicts the sale and rental price of every property in the UK, every month, by analyzing a wide range of signals including market trends, supply and demand, and the condition of individual homes.

Traditionally, valuations were fairly static, based on fixed data points that didn’t reflect recent improvements or shifting market conditions. Vertex AI makes them dynamic. Whether it’s a newly renovated kitchen or a shift in local market conditions, we can factor in real-time changes to properties on our website, delivering more accurate, up-to-date valuations for both homeowners and estate agents.

These monthly valuations are invaluable to our partners. Mortgage lenders, and estate agents use them as trusted pricing guides to understand local markets and assess risk, especially when managing large property portfolios or backbooks.

Behind the scenes, the hive gives us access to both structured and unstructured data, including more than 25 years of property images that were previously siloed. Now stored securely in Cloud Storage, this rich visual data is fueling advanced use cases, including these enhanced valuation models and deeper market analysis.

Driving operational efficiency with a smarter, unified data platform

With our migration to the cloud, we’ve embraced a “hub and spoke” model to ensure both consistency and flexibility in how data and AI are used across the business. The “hubs” are our central teams — experts in BigQuery, Looker, and Vertex AI — who set best practices and help scale innovation. The “spokes” are our vertical business units, such as the New Homes department, that tap into the hive platform to run their own business intelligence and AI use cases, tailored to their specific needs.

By consolidating multiple legacy business intelligence tools into a single platform with Looker, we’ve simplified our tech stack and created operational gains. For example, the New Homes team has cut down meeting prep with developers from hours to minutes, thanks to easily accessible, self-serve Looker dashboards.

And we’re constantly discovering new ways to create value from our new platform. For example, as Google Cloud rolls out new features such as BigQuery’s Series FM function for low-code forecasting, our teams are quickly adopting them to move from descriptive to predictive analytics. Forecasting leads, time spent on site, or whatever KPI a business unit has, was previously unthinkable, with siloed data and manual processes for developing models and ingesting data somewhere else for analysis. In our new platform, we can quickly trial this kind of forecasting in a spoke using just 10 lines of code and our BigQuery-Looker integration. It only took weeks for many of our business units to start using Series FM for forecasting.

Helping users at every stage of homeownership with AI

As we look to the future, our AI strategy is expanding beyond helping people find a home. Rightmove is evolving into a smart, supportive companion for every stage of the home journey: find, afford, transact, move, and live.

That means using data and AI to continue helping users to find properties but also to make better-informed decisions throughout the entire lifecycle of homeownership after that. We’re already rolling out new capabilities, such as smarter mortgage in principle matching and insights into the total cost of ownership, including broadband, energy, and utility costs, so buyers can move with confidence.

One exciting step in this direction is using Vertex AI to power the models and data behind our Track a Property feature — a way for home-owners to regularly check the value of their home. This upgrade means the valuation models are built and trained faster, will improve their accuracy over the long term with added model engineering and tuning, as well as taking advantage of better cloud architecture to host them.

This is just the beginning. As Google AI continues to evolve, so does our platform, becoming a home and living assistant that supports not just the move, but the life that follows it.

How a leading consumer insight brand uses Dataproc to hyper-personalise faster

Mon, 06 Apr 2026 16:00:00 +0000

At RVU, we have a clear and vital mission: empower people, transform industries.

For our market-leading home management and switching brands — Confused.com, Uswitch, Tempcover, Money.co.uk, and Mojo Mortgages — transparency and accurate information are everything. Today’s consumer expects more than a simple comparison table; they want personalized recommendations tailored to their unique circumstances.

Delivering on that promise — building a true personalization engine that powers all our brands — requires a data foundation capable of processing massive, complex datasets for sophisticated ML models. Today, our platform powers hundreds of automated personalization campaigns, optimized with billions of data points from across all our brands. We tackled this challenge using the power of Google Cloud and its two solutions for Apache Spark: Dataproc and Google Cloud Serverless for Apache Spark. Together, we’re making our mission a reality.

The high-speed engine for feature engineering

Our relationship with Google Cloud isn’t new. In fact, we've used BigQuery as our unified data platform for over a decade. Coming from a performance marketing background, we’ve always dealt with a lot of data, but we recognized early on that we’re not a digital infrastructure company. Instead, our focus must always be on where the value is. Managed solutions like BigQuery that eliminate infrastructure and capacity headaches were a natural fit from the start.

The key challenge was stitching together a meaningful and coherent picture of customer behavior across our brands — turning countless fragmented interactions into something that genuinely reflects how a user behaves, clicks, and makes decisions. Instead of relying on isolated events and aggregate views, we’ve had to build a platform capable of connecting these signals into a narrative that works for our machine learning models.

Using Dataproc to support this was a gamechanger. The biggest impact has been its role as our core high-speed Spark processing engine, primarily for feature engineering for our ML model development. Feature engineering, which is the crucial process of shaping all that raw customer data for our data science models, is a real value-driver for us. It’s where we have a marked competitive edge.

The result has been a significant leap in our innovation velocity. With Serverless for Apache Spark, we now have the ability to shape our customer data for feature engineering in just a matter of days. Previously, this would have taken weeks. We’ve also dramatically reduced our time-to-market, which also used to take weeks. Now, a new contractor can join the team and deliver a model, including exploratory data analysis and all feature engineering, in only a week and a half. That’s incredibly fast.

Delivering personalized experiences

By improving our speed of innovation, we’re better positioned to deliver a personalized user experience to our customers and partners.

Our hyper-personalization journey accelerated once we moved to Spark. We can now run heavyweight data processing jobs that crunch vast amounts of behavioral and contextual data, allowing us to build models that generate genuinely meaningful predictions.

These models help us understand not just what to say to a customer, but when and how to say it — selecting the right moment and the right channel to deliver personalized insight that genuinely resonates.

Building a future vision

Google’s Data Cloud directly aligns with our culture of prioritizing value, and its impact on our business is profound. I call it the network effect, where everything seamlessly connects within the same ecosystem: Our data resides in BigQuery, our ability to validate, enrich, and transform that data is tied to Dataproc and Serverless for Apache Spark, and our capacity to deploy the ML models spans the network. It’s all co-located and integrated, powering the real-time accuracy of our consumer brands and giving us a competitive advantage.

For our engineers, the big win is the lack of infrastructure they have to deal with. They can press a button that processes all the data in 10 minutes, rather than having to set up a network of clusters and servers and make them talk to each other. It’s incredibly efficient and frees up time for more valuable work like building and iterating data products.

Dataproc has upped our speed, scale, and agility. It also gives us the tools to innovate with AI as we build the future of hyper-personalization. Today, we’re proud to say RVU’s cutting-edge tech and data are helping millions of UK consumers make smarter, more informed decisions, and truly transforming industries.

Inspired by RVU's success? Whether you need persistent clusters with Dataproc or the agility of Serverless Spark, Google Cloud has a managed solution to help you focus on value, not infrastructure. Discover the right Spark on Google Cloud for your use case.

Introducing Looker self-service Explores for faster ad-hoc analysis

Mon, 06 Apr 2026 16:00:00 +0000

By design, Looker is the enterprise semantic platform which ensures that every data set meets a high standard of accuracy by acting as a single source of truth and providing long-term consistency of your metrics. Today, we are introducing a complement to this governed framework: self-service Explores, to accelerate high-velocity, ad-hoc analysis. Self-service Explores allows you to bring your own data directly into the Looker semantic layer, providing instant access to insights while maintaining the integrity of your existing governed data ecosystem.

Data teams often find themselves caught between two worlds. On one side, there’s the trusted, governed world of modern BI, where every metric is defined and every row is verified. Then there’s the agile, anything goes nature of spreadsheets and CSV files where you can get answers in seconds but run the risk of ending up in a siloed data vacuum.

Self-service Explores bring the value of modern, governed BI to the experimentation self-starting capability of spreadsheets, allowing anyone with the right permissions to turn a flat file into a fully functional Looker Explore in seconds. You can also import from Cloud from Google Drive and quickly transform Google Sheets data into conversational analytics. No code, no waiting — just insights.

You can drag and drop a comma separated or spreadsheet file (.csv, .xls, or .xlsx) or pull directly from Google Sheets, and Looker automatically creates an Explore. Behind the scenes, these files are securely stored in your own BigQuery instance, ensuring your data remains within your controlled infrastructure.

From importing a CSV to a self-service Explore in seconds

Key capabilities to power your analysis

With Looker self-service Explores, you gain:

Instant file uploads: Use a simple drag-and-drop interface to ingest local files for one-off analyses or testing a hypothesis before committing it to a permanent model.
Connect directly to Google Sheets: Easily import data via Google Sheets either via Oauth or by specifying the Google Sheets URL and sharing the document.
Merge queries (BigQuery): You can now combine your uploaded local files with standard, modeled Looker data, allowing you to enrich official company metrics with external data points to find new correlations. We’ve also added enhanced merge queries capabilities so you can perform merges on unlimited data as long as the data is on the same BigQuery connection.
Re-upload and refresh: You can easily re-import or update files within existing self-service Explores to keep your ad-hoc dashboards current.

Exploration through conversation, with red-tape free governance

Business intelligence in the agentic era has created new opportunities for business and data leaders throughout your organization to engage with their information faster and more intuitively, without significant technical demand. Self-service Explores bring support for conversational analytics, enabling you to ask questions in natural language about your uploaded data and get immediate answers, outlined visually, with grounding in Looker’s semantic layer. Users can chat with their data to drill down deeper, gaining more insights with follow-up questions and refinement.

Additionally, self-service Explores are built with admin controls at the forefront. Admins have full visibility and monitoring capabilities, with clear distinctions between ad-hoc data and modeled data. This gives users the freedom to explore, while maintaining the integrity of your core business logic. Self-service Explores give you the agility of a spreadsheet with the scale and security of BigQuery. It’s about spending less time waiting for a data model and more time actually using your data. Get started today.

Conversational Analytics now available for Looker Embedded environments

Mon, 06 Apr 2026 16:00:00 +0000

Looker Embedded analytics are at the heart of many next-generation data products, enabling monetization with live metrics and customizable user experiences. In the AI era, users expect apps to be highly interactive and conversational, and for data to be contextual, accessible and intuitive. Today, we are delivering conversational analytics in Looker Embedded environments with the Conversational Analytics API, now generally available, extending the natural language experience users have grown to expect from Looker to more surfaces, limited only by customers’ imagination.

Leading the shift to composable and agentic BI

Organizations are building integrated data experiences that meet users where they work. This is happening alongside the rise of agentic IDEs, where developers use AI agents to plan, code, and execute complex engineering tasks. Leveraging Looker’s embedded architecture within these environments incurs many benefits:

Differentiated agent experiences: Build high demand, unique conversational experiences that set your products apart.
Accelerated data monetization and developer extensibility: Turn raw data into sophisticated embedded products, transforming data assets into high-margin revenue streams.
AI-readiness in the IDE: Build with agentic IDEs with confidence, using Looker’s semantic layer to reduce hallucinations and maximize speed-to-market.

Conversational Analytics generally available for embedded users

Building on Looker’s existing embedded framework and governed semantic layer, you can now integrate Gemini-powered natural language querying and AI recommendations via a low-code iframe implementation or our extensible SDKs. This makes it easier to ship production-ready, conversational AI within any application.

With native support for querying multiple Looker Explores, a built-in code interpreter, and customizable theming, you can now provide a private-labeled, high-reasoning agent experience alongside your core features, bridging the gap between complex data models and intuitive user discovery.

Embed Conversational Analytics in Looker anywhere in your application using an iframe

Conversational Analytics now available from the Looker API

With our Conversational Analytics APIs, you can now build conversational experiences backed by Looker Explores directly within your customer-facing applications. By leveraging the Looker API, you can create multi-turn conversational workflows that offer AI-powered recommendations, while also verifying and explaining the underlying SQL query. The Looker API turns to the Looker SDK for user authentication and management, making it easy to integrate agentic conversations anywhere in your app.

Building a custom UI with Conversational Analytics in Looker API endpoints

Data exploration is a conversation away

The shift from static and inflexible dashboards to AI-driven exploration is transforming how businesses deliver value to their customers, empowering them to do more. Grounding these new conversational capabilities in Looker’s semantic layer means the insight accuracy you’ve always relied on from Looker is now available in third party applications, so you can know the insights are verifiable.

Whether you use the embedded iframe option or the Looker SDK, you can now build data experiences that don't just show information, but engage users in a dialogue. To start building conversational experiences in your own products with Looker Embedded, check out the documentation and best practices guide.

How Honeylove boosts product quality and service efficiency with BigQuery

Thu, 02 Apr 2026 16:00:00 +0000

Building the perfect bra takes thousands of data points.

That’s why Honeylove isn’t just another intimates brand. We’re a technology company that happens to make exceptional bras, tops, shapewear, and bodysuits. Technology shapes everything we do, from how we iterate garments based on customer feedback to how we optimize sizing across those thousands of data points.

When Honeylove was born in 2018, though, our data wasn’t consolidated. We were looking at analytics in Shopify, checking email campaign performance in one platform, and reviewing ad metrics in another. We weren’t connecting the dots as effectively as we could have.

Then we fell in love with BigQuery. In this post, we’ll cover how Honeylove uses BigQuery and Gemini to unify our data, automate key business insights, and leverage AI to boost product quality and service efficiency — as well as how other organizations looking to make the most of their data can follow our approach intimately.

Transforming insights with BigQuery and Gemini

The first step was getting all our data in one place. BigQuery gave us exactly what we needed: a performant, economical, unified data platform that integrates seamlessly with the tools our team already uses within the Google ecosystem, such as Google Ads and Google Sheets . This helped eliminate manual data silos and enabled us to quickly adopt AI and ML capabilities across the business.

The real transformation came when we started leveraging BigQuery ML functions for contribution analysis. We built models to analyze the key drivers behind some of our most critical metrics: conversion rate, customer satisfaction scores, website performance, and return rates.

What’s really powerful for us is that we can feed these contribution analysis results directly into Gemini to produce accessible reports and summaries. Before implementing this approach, 10 to 15 people would spend an hour before key meetings manually reviewing dashboards, trying to drill into the data and find meaningful insights. We’ve saved hundreds of hours per year just by automating this process with Gemini.

But the impact of BigQuery and Gemini goes beyond time savings. These tools help us find patterns and insights we would’ve missed entirely. Even if you have the best marketing analysts looking over dashboards, they just wouldn’t be able to slice it in the same way these reports allow us to do.

We’ve also been able to transform forecasting inventory and demand planning, another area where manual processes previously dominated. By deploying and training BigQuery ML’s ARIMA univariate forecasting models, we’ve used high-accuracy SKU-level demand forecasts that automatically adjust for seasonality and recent changes.

These automated forecasts consistently come within 5% of what we calculate manually — a huge improvement over third-party vendors that were sometimes off by 20% to 30%. Having that additional checkpoint gives us more confidence when making critical inventory decisions.

Unlocking value and creative with multimodal embeddings

Customer service tickets can be a treasure trove of valuable feedback and information for ecommerce brands. But only if you can extract insights from them, and with Google Cloud, we can.

We leverage Gemini embedding models and BigQuery vector search to transform the unstructured text of our tickets into actionable data. We generate vector embeddings for tickets already in our data warehouse using simple SQL commands, and then use those vectors for semantic searching through retrieval-augmented generation (RAG).

This allows us to ask precise, natural-language questions, such as “What do customers love about our bras?” or “What changes would you like to see to our bodysuits?” In response, Gemini instantly identifies similar use cases, enabling us to move beyond keyword matching and quickly find the root causes of any issues, which are often nuanced. This proactively guides product improvements and enhances service efficiency. We’re saving about 30 seconds per ticket, which might not sound dramatic until you multiply it across thousands of interactions.

We’re also experimenting with multimodal embeddings for video asset search across our ad and influencer content library. It’s been fun to test queries like “find me videos with dogs” or “find me a video with a red dress” and watch it actually work. The next step is to use those embeddings to compare new creative assets with existing ones and predict performance based on our historical data.

Growth creative has traditionally been driven by gut feelings rather than numerical analysis, but we hope to change that by using our huge library of existing ad creative to inform what we test and create in the future.

Building for the future with Google Cloud

Today, Google Cloud and BigQuery are a central pillar of our company. They allow us to spend less time on manual tasks and more time on high-value work that solves real-world problems, making us very efficient as a small team.

Working with the Google Cloud team is invaluable. They’ve been a true partner, and they continue to support our roadmap. We’re leaning further into BigQuery ML functionality, moving more of our data science work into automated, always-available models rather than offline analyses.

We’re also developing internal knowledge bots using the Vertex AI RAG Engine, connected directly to our internal documents hosted on Google Drive, to provide instant answers to internal policy and process questions. Additionally, we’re experimenting with Conversational Analytics API to provide a “BI in a box” experience so our teams can ask plain-text questions and get metrics and charts without needing an analyst.

As a technology-first company, this transformation continues to have a profound impact on what we do at Honeylove. It accelerated innovation in product quality, improved operational efficiency, and ensured that our customers receive a more intelligent and consistent service experience.

What’s new with Google Data Cloud

Thu, 02 Apr 2026 16:00:00 +0000

March 23 - March 27

We showed you how you can scale your reads with Cloud SQL autoscaling read pools. This feature allows you to provision multiple read replicas that are accessible via a single read endpoint and to dynamically adjust your read capability based on real-time application needs.
Our customers are leveraging the full power of Conversational Analytics and Looker to drive major business and technical breakthroughs in the AI era. Companies like Telenor, Pet Circle, Fluent Commerce, Lighthouse Intelligence, Wego, and ROLLER are turning data into insights and actions, grounded by Looker’s semantic layer.

March 16 - March 20

We introduced an enhanced Gemini assistant in BigQuery Studio, transforming the agent from a code assistant into a fully context-aware analytics partner.

February 23 - February 27

We introduced managed and remote MCP support for Google Cloud databases, including AlloyDB, Spanner, Cloud SQL, Bigtable and Firestore, to power the next generation of agents. This announcement extends the ability for AI models to plan, build, and solve complex problems, connecting to the database tools our customers leverage daily as the backbone of their work environment.
We outlined how you can build a conversational agent in BigQuery using the Conversational Analytics API to help you build context-aware agents that can understand natural language, query your BigQuery data, and deliver answers in text, tables, and visual charts.

February 16 - February 20

Our customers are leveraging the full power of Looker to drive major business and technical breakthroughs. Companies like Arrive, Audika, Carousell, Framebridge, GumGum, Intel, Overdose Digital, Ocean Network Express, Subskribe and Promevo are leveraging Looker’s newest AI-driven capabilities, including Conversational Analytics, to transform data to insights and actions, and empower their entire organization with a single source of truth, powered by Looker’s semantic layer.

February 2 - February 6

Join us on March 4 for our webinar, Win Your AI Strategy with Cloud SQL Enterprise Plus, to learn how to power your generative AI workloads with 3x higher performance and 99.99% availability. Register today to discover how to build a scalable, enterprise-grade foundation for your most demanding AI applications.

January 26 - January 30

We introduced Conversational Analytics in BigQuery, which allows users to analyze data using natural language. Conversational Analytics in BigQuery is an intelligent agent that generates, executes and visualizes answers grounded in your business context directly in BigQuery Studio, making data insights for data professionals more conversational.
We outlined how data products have become the foundation for AI agents, providing the context needed to make autonomous agents reliable and trusted for real business use, backed by organized business logic and semantic understanding.
We highlighted how you can supercharge data analytics workflows, and outlined Google Cloud’s AI agent offerings for data engineering, data science, and development tools, so you can integrate agentic workflows in your applications, empower your teams and speed discovery.

January 19 - January 23

We have fundamentally reimagined Firestore with pipeline operations for Enterprise edition. Experience a powerful new engine featuring over a hundred new query features, index-less queries, new index types, and observability tooling to improve query performance. Seamlessly migrate using built-in tools and leverage Firestore’s existing differentiated serverless foundation, virtually unlimited scale, and industry-leading SLA. Join a community of 600K developers to craft expressive applications that maximize the benefits of rich queryability, real-time listen queries, robust offline caching, and cutting-edge AI-assistive coding integrations.
Introducing Google Cloud SQL on MSSQLTips: We are highlighting a new technical guide published on MSSQLTips titled "Introducing Google Cloud SQL." This article serves as an essential resource for SQL Server administrators and developers exploring Google Cloud's fully managed database service. It provides a detailed overview of Cloud SQL capabilities, including high availability, security integration, and the seamless transition of on-premises SQL Server workloads to the cloud, making it an ideal resource for those planning their migration strategy.
We are excited to announce the Public Preview of Microsoft Entra ID (formerly Azure Active Directory) integration with Cloud SQL for SQL Server. Designed to tackle the challenge of identity sprawl in multi-cloud environments, this integration allows organizations to govern database access using their existing Microsoft identity infrastructure. Key benefits include centralized identity management, enhanced security features like Multi-Factor Authentication (MFA), and simplified user administration through direct group mapping. This feature is available for SQL Server 2022 and supports both public and private IP configurations.

January 12 - January 16

Google-built JDBC Driver for BigQuery is now available in Preview
We are excited to announce the launch of the new, Google-built JDBC driver for BigQuery. This new open-source driver provides a direct, high-performance connection for Java applications to BigQuery and is developed entirely in-house by Google. Download a new driver and connect your Java application to BigQuery.
Troubleshoot Airflow tasks instantly with Gemini Cloud Assist investigations: Cloud Composer just got smarter. We are excited to announce that Gemini Cloud Assist investigations are now available directly within Cloud Composer 3. Instead of manually sifting through raw logs, you can now simply click "Investigate" on a failed Airflow task. Gemini analyzes logs and task metadata to identify failure patterns—such as resource exhaustion or timeouts—and provides actionable recommendations driven by Gemini Cloud Assist to resolve the issue. This integration shifts the debugging experience from manual toil to automated root cause analysis, significantly reducing the time required to restore your pipelines. Learn more about AI-assisted troubleshooting.

How FM Logistic tackled the traveling salesman problem at warehouse scale with AlphaEvolve

Thu, 26 Mar 2026 16:00:00 +0000

The traveling salesman problem asks a deceptively simple question: What's the shortest route that visits every point exactly once? It's one of the hardest problems in computer science, and mathematicians have been working on it for nearly a century. It's also what FM Logistic's warehouse operators face every day in Poland.

The facility spans eight football fields. It holds over 17,700 picking locations. And across every shift, up to several dozen operators on ride-on electric trucks crisscross the floor collecting cartons, each one navigating dozens of storage locations per tour. Every unnecessary step adds up: in time, in wear on the fleet, and in delayed fulfillment.

FM Logistic, a global logistics provider operating in over 14 countries, had already optimized their routing once. Their existing model used a fast, cost-prioritized allocation logic built for real-time responsiveness. It worked well, but it made decisions step by step, which limited how well it could coordinate routes across the full warehouse. With dozens of operators working the same floor across shifts, even a small routing improvement would compound quickly.

So they turned to AlphaEvolve on Google Cloud.

Teaching an AI to write better algorithms

AlphaEvolve is an evolutionary coding agent that generates and refines algorithms autonomously using Gemini models. Rather than calculating a schedule from fixed rules, it works as a coding partner: writing new code, scoring it, and iterating until it finds a better solution than the one it started with.

The team didn't start from scratch. They gave AlphaEvolve a "seed" program: their existing algorithm, which made routing decisions one step at a time based on what looked best in the moment. This gave the agent a working baseline that already solved the problem, just not optimally. From there, AlphaEvolve used Gemini to generate variations of this code, introducing mutations and new logic to see if it could beat the human-designed original.

Measuring what good looks like

For AlphaEvolve to improve, it needs a way to measure how well each algorithm performs. FM Logistic designed a custom evaluation function using a representative dataset of 60 tours (over one hour of workforce data), letting the agent test thousands of generated algorithms against real-world conditions.

The evaluation scored every new piece of code on a primary goal: minimize the average travel distance per pick, while avoiding operational failures. The team built in specific penalties to steer the model away from unworkable solutions — things like exceeding forklift capacity, missing pending orders, assigning the same box twice, violating FIFO priority for older orders, or exceeding the computation time required for real-time operations.

The results

The new routing logic delivered immediate, measurable gains over the previous best baseline:

10.4% improvement in routing efficiency over the previous best solution.
15,000+ fewer kilometers of warehouse travel per year at full operational scale.

That efficiency gives FM Logistic room to handle larger order volumes with the same team and equipment, without adding headcount or expanding their fleet.

"Through our partnership with Google Cloud and the implementation of AlphaEvolve and Gemini, we further optimized our routing approach for fast-moving operations," said Rodolphe Bey, Group CIO at FM Logistic. "The 10.4% improvement was achieved on top of an already highly optimized baseline, where further gains are typically hard to come by. This translates directly to faster fulfillment, improved working conditions for our teams, and reduced wear on our fleet."

What the winning algorithm actually does

By running a series of experiments, each generating hundreds of candidate programs, AlphaEvolve developed a new algorithm that outperformed the previous best human-engineered one. The result is a set of clear, human-readable rules that warehouse teams can review and adjust as needs change.

The three core improvements:

Density-based starting points (Anchor selection): The previous system chose a starting mission based on the single location where the most missions overlapped. The new algorithm looks more broadly, identifying clusters of items that are close together and using those dense areas as "starting anchors" for building routes. Every tour begins with a highly efficient core.
Two-step filtering with distance simulation: To maintain real-time speed, the algorithm uses a two-stage process. First, a quick filter eliminates orders that do not fit the route's logic. Second, a precise distance simulation runs only on the best remaining candidates to find the most efficient path, without slowing down warehouse operations.
Flexible route building: If the algorithm can’t fill a truck efficiently around a specific starting point, it doesn’t force a bad route. It returns those orders to the main pool so they can be picked up by a better-fitting route later, improving efficiency across the entire warehouse.

What’s next

The Poland pilot (which is now running in production) demonstrated what evolutionary AI can do for complex routing at warehouse scale. FM Logistic is now exploring extensions — applying the algorithm to other high-volume e-commerce facilities, researching how AlphaEvolve might help optimize road transport for less-than-truckload shipments, and investigating AI-driven product placement inside warehouses to further cut travel distances.

^{This project was a collaboration between the FM logistic team including: Mateusz Klimowicz, Jarosław Urbański, Florent Martin and Alberto Brogio and the AI for Science team at Google Cloud including (but not limited to): Kartik Sanu, Laurynas Tamulevičius, Nicolas Stroppa, Chris Page, Gary Ng, John Semerdjian, Skandar Hannachi, Vishal Agarwal and Anant Nawalgaria as well asMariusz Czopiński from the Google account team as well and partners at Google DeepMind}

BigQuery Studio is more useful than ever, with enhanced Gemini assistant

Mon, 16 Mar 2026 16:00:00 +0000

Modern data teams dedicate a huge portion of their time to managing analytics overhead rather than just analyzing data. This includes tasks such as identifying necessary data, configuring schedules, or investigating the reasons behind a stalled job. Beyond these operational challenges, they also need an assistant that is versed in their data and has the context of their current work.

The latest Gemini-powered assistant in BigQuery Studio, available today, has new capabilities that allow you to interact with your data environment differently, transforming the agent from a code assistant into a fully context-aware analytics partner.

Here is a deep dive into the major improvements you can use right now.

1. Context-aware interoperability

The query editor tab and chat interface are now highly interoperable. The assistant is now aware of your active and open query tabs.

This means you no longer have to copy-paste code snippets or explain your context from scratch. Simply ask questions or request optimizations based on the active query tab, and the assistant intelligently understands exactly what code and resources you are referring to.

Advanced SQL generation: Beyond standard queries, the assistant can now generate advanced SQL that utilizes AI operators and federated queries, helping you unlock more complex analytical use cases with simple natural language prompts.

Fig 1.1 - Assistant is context-aware of the active tab and what “query” is being referred to

2. Intelligent resource discovery

As organizations grow, data gets scattered across different projects, datasets, and tables. Finding the specific resource you need can feel like finding a needle in a haystack.

The assistant in BigQuery Studio now features resource discovery, utilizing Dataplex Universal Catalog search to find resources across single or multiple projects. You can now search for a wide range of BigQuery resources, including datasets, tables, models, saved queries, and even scheduled queries. Now, you can:

Ask questions in plain English: You no longer need to remember exact table IDs. You can search using intent-based prompts like "Where can I find demographics such as age and location for new users?" or "Do I have any dataset named ecommerce?"
Deep dive into metadata: Once the assistant finds the right dataset, the conversation doesn't stop. Ask follow-up questions to understand the structure of the data before you even write a line of code, with.
- Visual schemas: The assistant displays table schemas and dataset details in a user-friendly UI directly within the chat window.
- Optimized queries: Ask "Is this table partitioned?" or "What’s the clustering on this table?" so that you write efficient queries from the start.
- Owner identification: Ask "Who owns this dataset?" if you need to request access.

Fig 1.2 -Assistant is able to search across projects to list datasets relevant to user prompt

Further, this feature respects your organization’s security policies: it only retrieves metadata for resources you actually have permission to view.

3. Instant job analysis and troubleshooting

We’ve all been there: a query that usually takes a few seconds is hanging. Or perhaps you received a bill that was higher than expected. Traditionally, this meant digging into information schemas or logs.

With the new job analysis capability, the assistant can now search both personal and project job history to provide insights.

Debug long-running queries: Instead of guessing why a job is stalling, simply copy the Job ID and ask: "Why is this job [Job ID] taking so long?" The agent analyzes the job's status and returns key statistics explaining the delay, such as slot contention, large row scans, or high data volume.
Root cause analysis: When a scheduled job fails, perform root cause analysis by asking, "Why did this scheduled job [Job ID] fail?" The assistant also provides recommendations on how to fix the problem.
Cost control: Audit your resource consumption by asking, "What are the 3 most expensive queries in the last 2 days?" The agent returns the right SQL needed to query the Information schema to get this information.

Fig 1.3 -Assistant can analyze jobs and provide optimization

With these advanced features within the Gemini-powered chat, the BigQuery Studio assistant is evolving into a context-aware, agentic partner that supports your entire data lifecycle. By simplifying resource discovery, automating SQL workflows, and streamlining troubleshooting, these enhancements allow you to focus on high-value insights instead of operational management.

To explore the full range of what the assistant can do and how to get started, visit our documentation page.

PayPal's historically large data migration is the foundation for its gen AI innovation

Thu, 26 Feb 2026 17:00:00 +0000

With the dawn of the gen AI era, businesses are facing unprecedented opportunities for transformative products, demanding a strategic shift in their technology infrastructure. A few years ago, PayPal, a digital-native company serving hundreds of millions of customers, faced a significant challenge. After 25 years of success in expanding services and capabilities, we’d created complexity in our data analytics infrastructure. Some 400 petabytes of data was spread across a dozen siloed systems due to limitations of scale and acquisitions of companies like Venmo, Braintree, and others.

Our very success in growth and innovation had created complexity that threatened our next evolution.

To continue leading the next wave of innovation in financial services, we knew we had to modernize our data foundation. Today, we’re proud to share how PayPal successfully completed what’s arguably one of the largest data migrations in history, culminating with the move of our analytics to BigQuery, Google Cloud’s enterprise data warehouse. This effort marks a significant leap in creating the robust data framework we’ll need to expand and advance our business priorities and meet the ever-evolving financial needs of our customers.

This migration was essential, but the scale was daunting. In fact, by some measures, such as our now sunset Teradata system, we believe this was one of the biggest data migrations in history. Befitting of such history, we wanted to offer some insights into how we tackled this migration and what others might consider when undertaking a significant migration of their own.

Untapped potential of data

As one of the original digital payment pioneers, PayPal processes billions of transactions, and houses decades of valuable customer insights. We have a mountain of data — really a mountain range — that had developed over decades without being fully leveraged in the service of our customers and merchants.

Each acquisition and new service added valuable capabilities but also introduced new data challenges. For example, a small business owner might use PayPal for online sales and Venmo for local transactions. However, providing a unified view of their business required complex processes that were costly and slow.

The fragmentation of data limited our ability to offer personalized experiences to consumers, thereby reducing the potential to maximize the value of their money and hindering our ability to gain deeper insights from the data.

As the gen AI era dawned, our digital fragmentation was becoming more than just a technical inconvenience. With AI becoming a transformative force in financial services with huge potential ROI, we knew fragmented data would severely limit our ability to create the intelligent experiences customers have come to expect. These could run from further strengthening our industry-leading fraud detection models to providing a best-in-class commerce platform for merchants to help them succeed in the competitive global economy.

To get there, we had to get our disparate data platforms in order, first.

Legacy systems, modern ambitions

The scope was massive. We needed to consolidate multiple data platforms, including what’s believed to be the world’s largest Teradata deployment, along with Hadoop clusters, Redshift, Snowflake, and various other systems processing petabytes of transaction data. This migration also had to be executed while maintaining the uninterrupted security and reliability our customers depend on.

As a technology company, PayPal has considerable internal resources, so we first had to decide whether to tackle this challenge ourselves. We weighed the costs and benefits and decided that if we were to unify and scale our on-premise infrastructure to meet our future needs, the cost and time-to-complete would have been prohibitive. Plus, the innovations in AI were happening at a rapid pace in the cloud. To truly leverage the power of our data, we needed to be where that innovation is happening.

We assessed various data warehousing solutions and chose BigQuery due to its numerous advantages. It is a fully managed, cloud native platform with disaggregated compute and storage that can scale independently. It has powerful capabilities at the scale and performance we needed, and a familiar SQL interface meant a gentler learning curve for our developer community.

Most importantly, BigQuery’s native integrations with AI enable seamless and efficient data analytics.

The journey to unified data

After choosing Google Cloud as our data partner, we embarked on our historic data migration. This may sound hyperbolic, but when you consider the scale of PayPal’s business, the geographies across which we operate, the regulations within each, the sensitive and quite literally valuable nature of this data, the scope of the challenge starts to be clear.

With the help of partners and experts from Google Cloud Consulting, we migrated more than 300 petabytes of data and streamlined operations, decommissioning around 25% of workloads. And we managed this all while maintaining zero downtime of our business operations and with no impact to customers. Here are some key factors that contributed to our success.

Alignment: The first hurdle in achieving transformations at scale is aligning stakeholders on a shared goal. So, we made it an enterprise-wide priority.

Discovery and analysis: Detailed inventories of data, workloads and inbound/outbound data streams is crucial for defining scope, effort and forecasting budget. Establishing lineage allowed us to trace the origins and relationships of various components, thereby providing a clear and comprehensive view of the dependency graphs.

Strategy: It is crucial to establish fundamental principles for the migration process, such as deciding between lift-and-shift versus modernization, defining security principles, setting governance guardrails, and determining how consumption will be tracked.

Execution: We automated every possible task and developed live dashboards to continuously monitor the progress of migrations. FinOps was integrated through the migration process with clear visibility of consumption and performance.

Benefits from BigQuery and beyond

We’ve achieve faster insights. Queries are 2.5x to 10x faster, including complex queries used by data scientists. This unlocks real-time insights, enabling PayPal to personalize product recommendations, offers, and customer support.

We’ve built new AI foundations. Data accessible for model training is 16x fresher. Feature engineering, a crucial step in AI development, is improved by instant access to clean, governed data. This accelerates the development personalized financial guidance, and predictive analytics for both consumers and businesses.

We’ve optimized operations. By migrating to BigQuery Data infrastructure vendors were reduced from four to one, streamlining operations and reducing complexity. Data duplication between platforms was entirely eliminated.

Our new unified data platform in BigQuery has become the source for PayPal's next wave of innovation, enabling us to create more intuitive, personalized experiences across our entire ecosystem and to leverage the power of gen AI.

AI-powered innovation unleashed

Looking ahead, we're exploring how this unified data platform will enable us to deliver AI-powered experiences that weren't possible before, including:

Predictive fraud prevention that spots potential issues before they affect our customers.
Personalized financial insights that help merchants optimize their businesses.
Seamless payment experiences that adapt to each customer's preferences and patterns.
More intelligent risk assessment that could help expand financial access to underserved communities.

Agentic commerce and future possibilities we are now able to imagine.

Lessons for the AI era

While our migration may be extraordinary in its scale, we are not alone in our needs or ambitions. There are ample considerations for companies within and well beyond financial services who may be pondering their own data foundations at this time.

First off, do not underestimate how under-utilized your data may be, and how unorganized.

Making sure your data is centralized, accurate, and consistent paves the way for AI experimentation and deployment. Organizations that spend time cleaning up their data fabric will be able to bring machine learning and generative AI applications to market more quickly, and do so at scale.

Second, ensuring data is accessible to everyone within your organization, with the proper controls, unlocks so much potential. Data orchestration and enterprise search, coupled with generative AI, has the potential to break down longstanding organizational silos and speed up decision-making across your organization. It’s one of the most promising applications of AI.

The financial world will continue to evolve, driven by new technologies and changing customer expectations. PayPal’s data transformation shows how even established companies can reinvent themselves to stay ahead of this change — provided they're willing to tackle the fundamental challenges that stand in their way.

In doing so, we've not only preserved our position as a digital payments pioneer but set ourselves up to continue leading the next wave of innovation in digital commerce.

Cool stuff Google Cloud customers built, Feb. edition: Telco data reinvention; Golden State’s “G.O.A.T.T.”; John Lewis explores DORA

Thu, 26 Feb 2026 17:00:00 +0000

AI and cloud technology are reshaping every corner of every industry around the world. Without our customers, there would be no Google Cloud, as they are the ones building the future on our platform. In this regular round-up, we dive into some of the exciting projects redefining businesses, shaping industries, and creating new categories.

For our latest edition, we explore a new data approach for Vodafone and Fastweb; evaluating John Lewis Partnership’s developer platforms; the Golden State Warrior’s AI playbook; healthy, stable networks at Hackensack Meridian Health; and Ab Initio brings better context to data for AI.

Be sure to check back next year to see how more industry leaders and exciting startups are putting Google Cloud technologies to use. And if you haven’t already, please peruse our list of 1,001 real-world gen AI use cases from our customers.

Fastweb + Vodafone reimagined data workflows

Who: Following the acquisition of Vodafone Italy by Swisscom in 2025, these leading European telecom providers wanted to rethink how they serve customers and deliver timely, personalized experiences across mobile, broadband, and digital channels.

What they did: Both companies had already begun modernizing customer data workflows with BigQuery, but combining ecosystems exposed certain limits of the existing setup. In order to give every channel real-time access to accurate customer data, they implemented Spanner as a service and governance layer, delivering low-latency reads, horizontal scalability, high availability, and a fully managed environment with zero ops overhead. The team is also using Gemini to generate clear documentation directly from the code, which saves hours of manual work.

Why it matters: Using Spanner Graph allowed the organization to map lineage in a way that reflects how its platform actually works: which tables drive specific jobs, how transformations cascade, and where dependencies sit. Call centers now see more complete, up-to-date customer information, digital channels can rely on consistent data without custom integrations, and partners can access what they need with low latency through Apigee.

Learn from us: “Rebuilding our Customer 360 platform with Google Cloud services has already changed how Fastweb + Vodafone works. Workflow monitoring is simpler, pipelines are leaner, and real-time serving is now the norm. ” – Vincenzo Forciniti, IT AI Adoption & Platform Engineering Lead, Fastweb + Vodafone

John Lewis measures the value of its developer platform

Who: The John Lewis Partnership is a major UK retailer operating John Lewis department stores and Waitrose supermarkets. To power their digital transformation, they built the John Lewis Digital Platform (JLDP) to support dozens of product teams building high-quality software for johnlewis.com.

What they did: Moving beyond simple usage metrics, John Lewis developed a sophisticated, multi-stage approach to measuring the real value of their platform. They transitioned from initial speed-based metrics (like "Onboarding Lead Time") to a comprehensive model using DORA metrics and subjective engineer feedback via the DX platform. This included a custom "Technical Health" feature that uses small, automated jobs to monitor more than 35 health measures — such as Kubernetes best practices, security, and operational readiness — providing teams with real-time "traffic light" indicators of their service health.

Why it matters: By focusing on value rather than just activity, John Lewis ensured the platform was actually reducing friction for developers rather than just being a mandatory tool. Their automated Technical Health checks allow product teams to manage technical debt and security vulnerabilities proactively. This approach has decoupled centralized operations teams from individual services, leading to faster incident resolution (MTTR), fewer outages, and significant cost savings.

Learn from us: "Measurement is a journey, not a destination. Start by measuring something meaningful to your stakeholders, but be prepared to adapt as your platform evolves. The things that mattered when you were proving out the platform's viability are unlikely to be what are important several years later when your features are mature." – Alex Moss, Principal Platform Engineer, John Lewis Partnership

Hackensack Meridian Health de-risks network migration using VPC Flow Logs

Who: Hackensack Meridian Health is a leading not-for-profit healthcare organization and the largest hospital system in New Jersey. System reliability is a cornerstone value for HMH as they manage a vast network of hospitals, urgent care centers, and physician practices.

What they did: Preparing for a large-scale migration to a new Google Cloud network design, Hackensack Meridian Health used VPC Flow Logs and Flow Analyzer to eliminate the "black box" of hybrid traffic. By enabling logs on their Cloud Interconnect VLAN attachments, they captured granular telemetry — including source/destination IPs, ports, and protocols. They then exported this data to create a visual "who-is-talking-to-what" map. This allowed them to identify critical traffic patterns between on-premises data centers and specific Google Cloud regions, VPCs, and applications.

Why it matters: In a healthcare environment, even minor network disruptions can have major consequences. By mapping traffic proactively, Hudson Meridian Health pinpointed exactly which moments in the cutover carried the highest risk. This preparation allowed them to detect a migration issue in just three minutes and resolve it within five — a process that previously could have taken hours. Beyond migration, this level of visibility enables the organization to better manage capacity planning, cost attribution, and security compliance across their hybrid infrastructure.

Learn from us: "Getting a clear picture of our interconnect traffic always felt like a black box. Enabling VPC Flow Logs and feeding it into Flow Analyzer finally gave us the map we needed. Identifying those critical traffic flows before we changed any routes was key to de-risking the entire migration." — Randall Brokaw, Cloud Engineering Manager, Hackensack Meridian Health

The Golden State Warriors’ AI-powered back office

Who: The Golden State Warriors are one of the NBA’s most successful modern franchises. Behind their on-court wins are a specialized operations team who run what might be called organization’s "G.O.A.T.T." (Greatest of All-Time Technologies), a data and AI platform that helps drive game-time insights, trading decisions, and fan experience enhancements.

What they did: The Warriors transitioned from a "gut-feeling" culture to an "analytics-first" strategy by building an internal "digital brain" on Google Cloud. Using BigQuery and Gemini, the team now automates complex workflows that previously took hours, such as generating pre-game scouting reports. They use machine learning to run thousands of trade simulations that prioritize "team fit" over raw individual stats and employ computer vision to track the "shot quality" of every attempt in the NBA. On the business side, they built a content recommendation engine using the Discovery API to deliver personalized digital experiences to their global fan base.

Why it matters: This AI-driven approach narrows the decision tree for leadership, allowing them to focus human expertise on the most viable options. By automating the “science” of data processing, coaches and scouts have more time for the "art" of face-to-face training, planning, and player development. This integration has not only influenced on-court strategy — like the three-point revolution — but has also improved business efficiency, with employees now proactively bringing AI-driven ideas to the IT team rather than waiting for top-down mandates.

Learn from us: "You can never reach a point where either humans or machines are making all the decisions. The sweet spot is finding that middle ground where intuition and data converge on the same conclusion. Data helps us narrow our decision tree before we even start evaluating specific options." — Nick Manning, Senior Director of Consumer Products & Emerging Technology, Golden State Warriors

Ab Initio unlocks enterprise data for the agentic AI era

Who: Ab Initio is an enterprise software company specializing in high-volume data integration and governance. Their platform is trusted by large-scale organizations to manage complex data lifecycles across hybrid and multi-cloud environments.

What they did: To solve the challenge of grounding AI agents in accurate data, Ab Initio partnered with Google Cloud to integrate its data fabric with BigQuery, Dataplex Universal Catalog, and Gemini. They launched a suite of more than 500 metadata and data connectors that bridge the gap between legacy systems (like mainframes, COBOL, and SAS) and modern cloud environments. This integration provides field-level, end-to-end lineage, allowing Gemini to access well-documented, "AI-ready" data regardless of where it resides.

Why it matters: AI agents are only as effective as the data they can access. By using Ab Initio as a "neutral hub," enterprises can federate data from on-premises and multi-cloud sources into a single unified layer without moving the data itself. This provides the rich semantic context and lineage needed for Gemini to perform grounded, explainable reasoning. For businesses, this means faster transition from experimental AI to production-ready agentic workflows that are auditable, compliant, and capable of making complex, automated decisions.

Learn from us: "Agentic AI requires trusted, AI-ready data and metadata. Understanding the origin, quality, and meaning of information matters as much as the data itself. Gemini serves as a key component of the agentic layer, using this context to make decisions that are explainable and auditable." — Scott Studer, Head of Development, Ab Initio & Chai Pydimukkala, Data Governance, Sharing & Integration Product Lead, Google Cloud

A developer's guide to production-ready AI agents

Wed, 25 Feb 2026 17:00:00 +0000

Something has shifted in the developer community over the past year. AI agents have moved from "interesting research concept" to "thing my team is actually building." The prototypes are working. The demos are impressive. And now comes the harder question: How do we ship this?

That question turns out to be a multi-part one. Agents don't behave like traditional software. They reason, act, and adapt, which means they need different approaches to testing, memory, orchestration, and security. The patterns that served us well for deterministic code don't fully translate.

To help developers work through these challenges, we've published a collection of guides covering the full agent lifecycle. These resources first appeared during Kaggle’s 5 days of AI Agents Intensive, and they’ve proven so popular and useful, we wanted to make sure a wider audience had access, as well.

These guides offer practical frameworks and code samples you can adapt to your own projects. Below, we'll walk through the key concepts — from agent architecture to production deployment — so you can decide where to dig deeper.

What is an agent?

At its core, an agent is an autonomous entity that reasons, takes action, and improves over time. The agent's brain is a large language model — a cognitive engine that understands tasks, generates responses, and makes decisions based on context. Unlike a static tool, an agent adapts as it works. It follows a recursive loop: Think, then Act, then Observe. Each cycle moves the agent forward, refining its approach as it goes.

Surrounding this core is the orchestration layer — the nervous system that manages communication and data flow. Think of it as a conductor coordinating specialized tools and external services. These include short-term memory (Session State) for immediate recall, long-term memory (Memory Service) for retaining past interactions, information retrieval (RAG), and modules for executing actions in the outside world (Tool Use). A security framework ensures the agent operates safely and within its intended boundaries. The goal of this architecture is to create an intelligent, helpful, and trustworthy assistant.

For a deeper exploration of these foundational concepts, see the full Introduction to Agents guide.

Tools and interoperability

For agents to be truly useful, they need to interact with tools, data sources, and other agents. Two emerging protocols offer standardized approaches to these connections.

Anthropic's Model Context Protocol (MCP) gives agents a standardized way to connect with external data sources and stateless tools. Instead of building custom integrations for every service, developers can use MCP's standardized interface to simplify development and improve interoperability.

Google's Agent2Agent Protocol (A2A) takes this further by enabling agents to communicate directly with each other, regardless of their underlying frameworks. Agents using A2A can discover each other's capabilities, negotiate how they'll interact, and collaborate on tasks through a secure and structured exchange of messages.

Together, these protocols create the foundation for agents that work within a broader ecosystem — connecting to tools, data, and each other. The Tools and Interoperability with MCP guide explains both protocols in detail with implementation examples.

Context engineering

If the LLM is the agent's brain, context engineering is the practice of feeding it the right information at the right time. This includes prompt design, retrieval mechanisms, tool selection, and conversation history — everything that shapes how the agent understands and responds to each request.

Context engineering transforms a generic model into a personalized assistant. It determines which memories to retrieve, which tools to offer, and how to frame each interaction. Effective context engineering creates agents that feel coherent and helpful across sessions. Without it, agents forget, repeat themselves, or miss the point entirely.

The Context Engineering guide covers context engineering frameworks and practical techniques for implementation.

Testing and evaluation

Autonomous agents require new approaches to quality assurance. When an agent makes its own decisions, success depends on sound judgment throughout the process, not just correct outputs.

Agent evaluation focuses on trajectories — the full sequence of decisions and actions an agent takes to reach a result, not just the final answer. Two agents might arrive at the same conclusion through very different paths, and understanding those paths matters. Good evaluation examines tool selection, reasoning quality, error recovery, and whether the agent asked clarifying questions when it should have.

A practical evaluation approach includes unit tests for individual components, trajectory analysis for multi-step decision sequences, and staged rollouts from sandbox to canary to production. Each stage validates different aspects of agent behavior before you expose it to more users.

For detailed evaluation frameworks and testing methodologies, see the Agent Quality guide.

Deploying agents to production

Moving from prototype to production requires infrastructure designed for agent-specific needs. Traditional deployment patterns need adaptation for systems that maintain state, use tools dynamically, and operate autonomously.

Production agents need session management to maintain context across interactions, persistent memory systems for long-term recall, tool integration with appropriate authentication and permissions, and real-time logging to trace agent decisions and actions.

Most teams deploy in stages: sandbox for internal testing, canary for limited real-world exposure, and production for full rollout. Each stage validates performance and catches issues before you expand access.

The Prototype to Production guide provides architectural guidance and code samples for building production-ready agent infrastructure.

Where to start

Your starting point depends on where you are in the journey. The Introduction to Agents guides covers foundational concepts, while Tools and Interoperability with MCP and Context Engineering address the practical challenges of building. When you're ready to validate and ship, Agent Quality and Prototype to Production will get you there.

The agents space is moving fast, but you don't have to figure it out alone. Pick the resource that matches your current challenge and start building.

Simplify your AI workflow with autonomous embedding generation in BigQuery

Thu, 19 Feb 2026 17:00:00 +0000

In the world of generative AI and Retrieval-Augmented Generation (RAG), embeddings are the "secret sauce" that allow machines and AI agents to understand the semantic meaning of data. As BigQuery extends its autonomous data-to-AI platform, embeddings unblock valuable multimodal use cases. However, for many data engineers, managing embeddings is a headache. Traditionally, users have to set up embedding generation pipelines themselves to propagate source content updates, embedding generation, and storage.

To help BigQuery users with their AI workloads, we’re introducing autonomous embedding generation. This feature allows BigQuery to automatically maintain an embedding column on a table based on a source column. No more manual pipelines, no more synchronization issues, just easy, AI-ready data.

Managing embeddings, the old way

Before autonomous generation, the process of updating your vector search database usually looked like this:

Detect new rows in your source table.
Generate embeddings via functions like AI.EMBED.
Handle rate limits and retries.
Update the destination table with the new vectors.
Monitor the progress of your embedding generations.

If your data changes frequently, keeping these vectors in sync can be a full-time job for the user/administrator. With this as the backdrop, we set out to enhance BigQuery with the following capabilities.

1. Help the user directly work with their data

We want to simplify the search experience for the user, so that they can do simple things like AI.SEARCH(TABLE mydataset.products, 'product_description', "A really fun toy"), without having to interact or understand the embeddings.

2. Automatic synchronization

BigQuery should manage embedding generation on behalf of the user and keep generated embeddings in sync with the source data.

3. Tight integration with vector indexes

BigQuery’s VECTOR_SEARCH has many users, and we want to ensure that the managed embedding was integrated into it.

The solution: autonomously generated embedding columns

We solved this by treating embeddings as a managed part of your table. Using a familiar SQL syntax, you can now define an autonomous embedding column that BigQuery manages for you.

code_block: <ListValue: [StructValue([('code', "CREATE TABLE mydataset.products (\r\n name STRING,\r\n description STRING,\r\n description_embedding STRUCT<result ARRAY<FLOAT64>, status STRING>\r\n GENERATED ALWAYS AS (AI.EMBED(\r\n description,\r\n connection_id => 'us.test_connection',\r\n endpoint => 'text-embedding-005'\r\n ))\r\n STORED OPTIONS( asynchronous = TRUE )\r\n);"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b681c2940>)])]>

For more information, please refer to the guide.

Integration with vector index and vector search

In addition, BigQuery’s vector index and vector search are also integrated with the generated embedding column. You can directly create a vector index associated with the source data column and query your data without managing embeddings manually. BigQuery automatically applies the base table's model to generate compatible embeddings for your query.

Introducing AI.SEARCH

We also launched a new function, AI.SEARCH, to provide a simplified signature for you to get started with the data-centric search experience. AI.SEARCH automatically uses the embedding model associated with the generated embedding column from the base table, so you don’t need to interact with the embedding configuration when using AI.SEARCH or VECTOR_SEARCH.

code_block: <ListValue: [StructValue([('code', 'SELECT base.name, base.description, distance\r\nFROM AI.SEARCH(TABLE mydataset.products, \'description\', "A really fun toy");'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b681c2220>)])]>

Simple management

Autonomous embedding generation is in preview, ready for you to use as part of your data analytics pipelines today. We’ve also invested in a few features to help make the process simpler to manage end to end:

Embedding status metadata: You can track the progress of embedding generation by querying the percentage of non-null embeddings in your table:

code_block: <ListValue: [StructValue([('code', "SELECT\r\n COUNTIF(description_embedding IS NOT NULL\r\n AND description_embedding.status = '') * 100.0 / COUNT(*) AS percent\r\nFROM mydataset.products;"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b681c21c0>)])]>

While you can initiate the creation of the vector index at any time, generating an index model will only happen at a scale when performance will benefit.

Native access to Vertex AI models: By ensuring your BigQuery connection has the Vertex AI User role, embedding generation can securely "talk" to a remote state of the art Vertex AI embedding models on your behalf.
Native error monitoring: If any step in the embedding generation pipeline fails, , you can view the status of recent background jobs via INFORMATION_SCHEMA jobs view (example), Here you can find detailed error info to help you resolve the issue.

What’s next

Autonomous embedding generation represents a shift toward AI-native multimodal data foundation that’s built for processing and activation of all data types. By automating and coupling embedding generation within the data platform, we’re helping developers spend less time on plumbing and more time on building intelligent applications. And we’re not done yet, and are hard at work building:

Simpler connection creation via Data Definition Library (DDL) and Data Control Language (DCL)
The ability to add a generated embedding column to existing tables via ALTER TABLE ADD COLUMN DDL
API and UI support for managing generated embedding columns
Direct support for multimodal data using ObjectRef

Ready to try it? Check out the official BigQuery documentation to set up your first managed embedding table today.

Building a conversational agent in BigQuery using the Conversational Analytics API

Thu, 19 Feb 2026 17:00:00 +0000

Bringing data into BigQuery centralizes your information, but the real challenge is making that data accessible. Often, technical barriers separate the people with questions — from execs to analysts — from the answers they need.

With the Conversational Analytics API, powered by Gemini, you no longer need intricate systems to get insights. The API is engineered to help you build context-aware agents that can understand natural language, query your BigQuery data, and deliver answers in text, tables, and visual charts.

Now, you can build any solution that can interface with the API. For example, you can integrate it with the Agent Development Kit (ADK) to build a multi-agent systems, or to implement these data strategies:

Self-service triage for operations: Give teams like Support and Sales an agent that answers data questions instantly. Instead of filing a ticket to ask, "Why did signups drop last week?", they get the answer immediately.
Differentiate your SaaS product: Differentiate your platform by embedding a powerful chat interface directly into your platform. Let your customers query and visualize their own usage data using plain English.
Dynamic reporting: Move beyond static PDFs. Automate the core reporting function and enable stakeholders to ask nuanced, follow-up questions for deeper investigation, effectively replacing report versions with real-time conversation.

In this post, we’ll share ways to build a conversational agent in BigQuery using the Conversational Analytics API.

Step One: Configure and create the agent

The deployment of a Data Analytics Agent involves configuring its access, context, and environment before making the final creation call.

In our included example, the Python SDK is used, but the Conversational Analytics API supports many other languages, depending on your preference and environment.

Initialize the client and define BigQuery sources

Begin by instantiating the necessary client (DataAgentServiceClient) to interact with the API. This client is used in conjunction with explicit BigQueryTableReference objects, which authorize the agent's access to specific tables (defined by project_id, dataset_id, and table_id). These individual references are then aggregated into a DatasourceReferences object under the bq field.

code_block: <ListValue: [StructValue([('code', 'from google.cloud import geminidataanalytics\r\n\r\n# Set project-specific variables (client, location, project IDs)\r\ndata_agent_client = geminidataanalytics.DataAgentServiceClient()\r\nlocation = "global"\r\nbilling_project = "your-gcp-project-id"\r\ndata_agent_id = "google_trends_analytics_agent"\r\n\r\n# Define the BigQuery table sources\r\nbq_top = geminidataanalytics.BigQueryTableReference(\r\n project_id="bigquery-public-data", dataset_id="google_trends", table_id="top_terms"\r\n)\r\nbq_rising = geminidataanalytics.BigQueryTableReference(\r\n project_id="bigquery-public-data", dataset_id="google_trends", table_id="top_rising_terms"\r\n)\r\ndatasource_references = geminidataanalytics.DatasourceReferences(\r\n bq=geminidataanalytics.BigQueryTableReferences(table_references=[bq_top, bq_rising]))'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b681abc40>)])]>

Set the agent context

Construct the context object by bundling the system_instruction (defining the agent's behavior/role) and the datasource_references (defining its permitted data access). This complete Context is then nested within the DataAnalyticsAgent structure of the final DataAgent object.

While you can provide a string based system instruction, we recommend that you use the more robust context object to provide instruction to the agent. The object can still be provided with additional system instructions to help provide supplemental guidance.

code_block: <ListValue: [StructValue([('code', '# Set the context using our system_instruction string\r\npublished_context = geminidataanalytics.Context(\r\n system_instruction=system_instruction,\r\n datasource_references=datasource_references\r\n example_queries=example_queries\r\n)\r\n\r\ndata_agent = geminidataanalytics.DataAgent(\r\n data_analytics_agent=geminidataanalytics.DataAnalyticsAgent(\r\n published_context=published_context\r\n ),\r\n)'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b6f299cd0>)])]>

Create the agent

Call data_agent_client.create_data_agent. This request includes the parent resource path (projects/{billing_project}/locations/{location}), the unique data_agent_id, and the fully configured data_agent object to complete the deployment.

code_block: <ListValue: [StructValue([('code', '# Create the agent\r\ndata_agent_client.create_data_agent(request=geminidataanalytics.CreateDataAgentRequest(\r\n parent=f"projects/{billing_project}/locations/{location}",\r\n data_agent_id=data_agent_id,\r\n data_agent=data_agent,\r\n))'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b6f299fd0>)])]>

Your agent now exists and is defined by that published_context.

Step two: Creating a conversation (stateful vs. stateless)

The Conversational Analytics API can handle conversations in two ways:

Stateless: You send a question and the agent's context. You must manage the conversation history in your own application and send it with every new request.
Stateful: You create a "conversation" on the server. The API manages the history for you. This is what allows users to ask follow-up questions.

We'll configure a stateful conversation. We create a conversation object associated with our new agent.

code_block: <ListValue: [StructValue([('code', 'def setup_conversation(conversation_id: str):\r\n data_chat_client = geminidataanalytics.DataChatServiceClient()\r\n conversation = geminidataanalytics.Conversation(\r\n agents=[data_chat_client.data_agent_path(\r\n billing_project, location, data_agent_id)],\r\n )\r\n request = geminidataanalytics.CreateConversationRequest(\r\n parent=f"projects/{billing_project}/locations/{location}",\r\n conversation_id=conversation_id,\r\n conversation=conversation,\r\n )\r\n try:\r\n # Check if it already exists\r\n data_chat_client.get_conversation(name=data_chat_client.conversation_path(\r\n billing_project, location, conversation_id))\r\n except Exception:\r\n response = data_chat_client.create_conversation(request=request)\r\n print("Conversation created successfully.")\r\n\r\nconversation_id = "my_first_conversation"\r\nsetup_conversation(conversation_id=conversation_id)'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b6f299e80>)])]>

Step three: Create a streaming chat loop

To allow for interactive analysis, we implement a function, stream_chat_response, to manage the conversation flow. The Data Analytics Agent API is designed to return a response as a stream, which is crucial for delivering updates on the agent’s progress in real-time.

A typical response stream can include distinct components, such as:

Schema: Confirmation of table resolution.
Data (query): The generated SQL query (excellent for debugging and transparency).
Data (result): The resulting data structure (e.g., a Pandas-like DataFrame).
Chart: A Vega-Lite JSON specification for data visualization.
Text: The final, synthesized natural language summary.

Defining the function

The function is defined to accept the user's question. Inside, we initialize the DataChatServiceClient and define a simple flag (chart_generated_flag) to track if a chart needs to be rendered after the stream completes. The user's question is wrapped in a Message object, which is required for the API request.

code_block: <ListValue: [StructValue([('code', "def stream_chat_response(question: str):\r\n data_chat_client = geminidataanalytics.DataChatServiceClient()\r\n chart_generated_flag = [False] # Flag to help with visualization\r\n \r\n # Format the user's question into an API-ready Message object\r\n messages = [\r\n geminidataanalytics.Message(\r\n user_message=geminidataanalytics.UserMessage(text=question)\r\n )\r\n ]"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b6f299c70>)])]>

Processing the stream

The ConversationReference is essential as it ties the current request to the stateful conversation and links it back to the specific data_agent we created earlier. Once the request object is fully assembled with the parent path, messages, and reference, we call data_chat_client.chat.

We then iterate over the returned stream. A utility function, show_message, is used here to parse and appropriately format the different response types (Text, Chart, Data) for the user. Finally, if the chart_generated_flag was set during the stream, a post-processing utility (preview_in_browser) handles the rendering of the visualization.

code_block: <ListValue: [StructValue([('code', '# Reference the stateful conversation and the created Data Agent\r\n conversation_reference = geminidataanalytics.ConversationReference(\r\n conversation=data_chat_client.conversation_path(\r\n billing_project, location, conversation_id\r\n ),\r\n data_agent_context=geminidataanalytics.DataAgentContext(\r\n data_agent=data_chat_client.data_agent_path(\r\n billing_project, location, data_agent_id\r\n ),\r\n ),\r\n )\r\n \r\n # Prepare the chat request\r\n request = geminidataanalytics.ChatRequest(\r\n parent=f"projects/{billing_project}/locations/{location}",\r\n messages=messages,\r\n conversation_reference=conversation_reference,\r\n )\r\n \r\n # Process the streaming response\r\n stream = data_chat_client.chat(request=request)\r\n for response in stream:\r\n # \'show_message\' is a utility function that formats\r\n # and prints the different response types (text, data, chart)\r\n show_message(response, chart_generated_flag)\r\n\r\n # If a chart was generated, \'preview_in_browser\'\r\n # is a utility to save and serve it as HTML\r\n if chart_generated_flag[0]:\r\n preview_in_browser()'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b6f299f70>)])]>

Step four: Talk to the agent

Asking questions

Now for the payoff. We can use our stream_chat_response function to have a conversation.

Checking the context

Let's start by seeing if the agent understands its own context.

Python

code_block: <ListValue: [StructValue([('code', 'question = "Hey what data do you have access to?"\r\nstream_chat_response(question=question)'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b6f299af0>)])]>

The agent will respond with a summary of the top_terms and top_rising_terms tables, using the descriptions we provided in the system_instruction.

Natural language to SQL to Chart

Now for a complex query. Notice we ask for a chart in plain English.

Python

code_block: <ListValue: [StructValue([('code', 'question = "What are the top 20 most popular search terms last week in NYC based on rank? Display each term and score as a column chart"\r\nstream_chat_response(question=question)'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b6f299160>)])]>

The agent will stream its process:

It will show the SQL query it generated to hit the top_terms table, filtering by dma_name = 'New York NY' and the most recent week.
It will print the resulting data as a table.
It will generate a Vega chart specification.
The preview_in_browser utility will serve this as an index.html file, showing a column chart.

The stateful follow-up

This is where the stateful conversation (Step 2) pays off.

Python

code_block: <ListValue: [StructValue([('code', 'question = "What was the percent gain in growth for these search terms from the week before?"\r\nstream_chat_response(question=question)'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f5b6f299eb0>)])]>

The agent remembers "these search terms" refers to the results from Question 2. It will generate a new query, this time INNER JOIN-ing the top_terms and top_rising_terms tables (as guided by our join_instructions) to find the percent_gain for that same list of terms.

Step five: Managing the agent

For a more in depth lifecycle management of the agent and messages, visit the Conversational Analytics API documentation page for the many various API requests you can make (HTTP / Python). You will find information on how to manage agents, how to invite new users to collaborate via the SetIAM, GetIAM APIs, and more.

Pro tip: Bridge the gap between data and people

By providing clear system instructions and schema descriptions, you can build an agent that is more than just conversational, as it becomes a domain expert. This interactive approach moves beyond static dashboards to provide truly accessible data analysis.

Data Analytics

Cool stuff Google Cloud customers built, April edition: BMW big on SLMs, MLB’s Scout Insights AI, personalized resort experiences

BMW tests the big potential of small models

MLB Scout Insights: AI-powered color commentary

Vail Resorts makes personalized AI assistance easy

CTC Global turns the smart grid into an intelligent one

ID.me reduces risk while scaling past 160 million users

Manhattan Associates powers more than a billion daily API calls

Introducing BigQuery Graph: Unlock hidden relationships in your data

Graph challenges faced by enterprises across industries

Common BigQuery Graph use cases

BigQuery Graph in the real-world

How to use BigQuery Graph

Ready to get started?

From operational to analytical: The unified Spanner Graph and BigQuery Graph solution

Spanner Graph for operational workloads

BigQuery Graph for analytical workloads

Spanner Graph and BigQuery Graph as a unified solution

One unified solution for all your graph needs

Get started with Spanner Graph and BigQuery Graph today

Scaling unstructured enterprise knowledge with BigQuery Graph, and Kineviz GraphXR

From fragmented to unified with BigQuery

BigQuery pipeline: From unstructured to structured

Unlocking hidden insights with Kineviz GraphXR

Why this matters

Get started

Accelerating data curation with Google Data Cloud

1. Cloud Storage auto-discovery for semi-structured data

2. Metadata curation and augmentation

3. Integrated governance: Quality, profiling, and lineage

4. Agentic workflows for pipeline development

5. Catalog-driven asset discovery and data products

6. Built-in AI functions for multi-modal data curation

7. Real-time curation with continuous queries

Data Studio returns as new home for Data Cloud assets

Extending our vision for analytics in the AI era

Two flavors: Data Studio and Data Studio Pro

Openness without compromises for your Apache Iceberg lakehouse

Write and read interoperability across engines

Improved price-performance with BigQuery and Spark

Advanced analytics with Apache Iceberg

Get started

Under one roof: Rightmove reinvents property search with unified data

Making it easier to find a home with personalized, dynamic suggestions

Helping partners work smarter with AI-powered recommendations

Turning data into insight and insight into value

Driving operational efficiency with a smarter, unified data platform

Helping users at every stage of homeownership with AI

How a leading consumer insight brand uses Dataproc to hyper-personalise faster

The high-speed engine for feature engineering

Delivering personalized experiences

Building a future vision

Introducing Looker self-service Explores for faster ad-hoc analysis

Key capabilities to power your analysis

Exploration through conversation, with red-tape free governance

Conversational Analytics now available for Looker Embedded environments

Leading the shift to composable and agentic BI

Conversational Analytics generally available for embedded users

Conversational Analytics now available from the Looker API

Data exploration is a conversation away

How Honeylove boosts product quality and service efficiency with BigQuery

Transforming insights with BigQuery and Gemini

Unlocking value and creative with multimodal embeddings

Building for the future with Google Cloud

What’s new with Google Data Cloud

March 23 - March 27

March 16 - March 20

February 23 - February 27

February 16 - February 20

February 2 - February 6

January 26 - January 30

January 19 - January 23

January 12 - January 16

What’s new with Google Data Cloud - 2025

How FM Logistic tackled the traveling salesman problem at warehouse scale with AlphaEvolve

Teaching an AI to write better algorithms

Measuring what good looks like

The results

What the winning algorithm actually does

What’s next