Retail

Easy as a green run: How Vail Resorts built an AI assistant to automate personalized recommendations

Fri, 27 Mar 2026 16:00:00 +0000

For skiers and snowboarders, every moment on the mountain is about maximizing the fun — chasing fresh lines, perfecting a new trick, or exploring new terrain. Whether they're exploring a familiar favorite or visiting a new mountain for the first time, riders want the right info at their fingertips so they can move with confidence, find hidden gems, and feel like they belong from the moment they arrive.

It’s a big part of the reason Vail Resorts launched My Epic Assistant during the 2024-2025 snow season. Vail operates some of the most iconic and beloved mountain destinations in the world (Whistler Blackcomb, Park City Mountain, Stowe, and Crested Butte, to name a few). The company wanted every guest to feel fully supported by their new app — able to get quick, helpful answers and discover everything their mountain has to offer, all in a way that keeps them immersed in the experience.

Just think, would you rather be looking for an info booth, or getting help from the app while riding the lift to your next run?

My Epic Assistant is an AI-powered assistant that takes the expertise of Vail Resort’s IT, hospitality, and operational teams and feeds it into Google’s powerful Gemini models. The result is an agentic guide to the slopes that can help decide on the right season pass, share the latest snow report, check on lesson preparations, or suggest a good stop for cocoa. Vail Resorts wanted more than a chatbot; they wanted a digital concierge that understands the nuance of a powder day at Whistler versus a family trip to Beaver Creek.

To take these offerings to the next level, the teams at Vail Resorts and Google Cloud specialist 66degrees updated My Epic Assistant with new features for the 2025-2026 season, including season pass recommendations and improved personalization. This feature now intelligently guides guests to the best pass for them — understanding guest questions and handling complex requests.

In this post, we’ll guide you through how we navigated the moguls of orchestrating a multi-agent system that can effectively answer the complex and nuanced questions from discerning customers looking to get help fast so they can get back to enjoying the mountain. It’s not just savvy guests who can benefit from AI-powered tools like these, either.

It’s not just savvy guests who can benefit from AI-powered tools like these, either.

In this blog post, we’ll detail how we built the My Epic Assistant, orchestrating a multi-agent system that can expertly slalom around the unique challenges of this rugged corner of the hospitality industry. It’s an approach that organizations in other customer-centric sectors can follow to help build agentic system for their own unique situations.

A new season, and better results

As we’ve implemented and refined personalization, improved search, summary capabilities, and conversational flow within My Epic Assistant, the app has achieved a 45% reduction in escalation to human agents since its initial launch.

By automating routine logistics and expanding personalization, Vail Resorts' Guest Experience Technology team has been able to ensure that technology never replaces the personal touch, it only enhances it. The overall experience for guests seeking support is now faster, more reliable, and available in more ways and times.

According to the team, "Utilizing tooling from Google Cloud, we could lean into agentic design patterns that gave us a way to unlock natural, personalized conversations. These boosted customer satisfaction, while reducing the need for manual intent design. These tools also let us combine flexibility and control to enable the assistant to respond fluidly but always within the boundaries of our brand, policies, and product strategy.”

The steps to building My Epic Assistant

First, My Epic Assistant uses a subtopic classification agent to understand the guest's initial request — whether they want to compare passes, get a recommendation, or just find general information. A date object is used to determine the time of year, as passes are only available for purchase for a limited time. If passes are no longer on sale, it will automatically route to information on lift tickets. If both passes and lift tickets are available, it asks a few clarifying questions to help customers decide which option will provide the best value for their trip.Next, the assistant hands off to the data collection agent, to determine what additional data is needed to provide a recommendation. For authenticated users, Vail Resorts provides their existing data like their home resort, average visits, age, upcoming visit resort, and peak day preferences called from a webhook. For a new non-authenticated user, it asks clarifying, easy to answer questions to fill in the gaps. There’s no one-way, static form filling here!

The generative AI’s conversational capabilities really shine during this step. My Epic Assistant is designed to chat freely back and forth, allowing guests to ask clarifying questions at any point and then return to the recommendation process without losing context. Via the instructions in the playbook, the model continually assesses if all parameters have been filled for the output in order to move on to the next step, and still allows guests to ask other questions during this process.

For instance, it’s possible that when asked if they want peak-day access, a guest might follow up with “What are peak-restricted dates this year?” My Epic Assistant will respond by invoking the website datastores that codify Vail Resorts’ extensive knowledge. My Epic Assistant then provides an answer and returns to the previous turn in conversation without losing context.

Finally, the recommendation agent takes over. Using the collected user data, it queries a structured database of all pass options to find the perfect match. The system then generates a user-friendly response explaining why an identified pass is a good fit and provides content cards with direct purchase links, simplifying the final step for the guest.

The core functionality of the pass recommendation agent is a datastore tool of a pass matrix that contains a structured file of all existing passes, their features, and their restrictions. With a broad number of options available, extensive testing was necessary to ensure the tool returned valid and appropriate options once it received all of the input parameters, and 66degrees relied on Vail Resorts’ deep domain knowledge to validate every possible outcome.

Then, prompt engineering ensures that the ultimate response presents the recommendations with detailed explanations as to why those specific passes were the best options for them. Content cards are also invoked with purchase links for those recommendations, without having to exit the playbook. For some occasions, a generative response was not appropriate, such as with specific details on the newly launched Epic Friends feature. Static responses are called via a code snippet and a conditional action, again housed within the playbook itself, simplifying the overall architecture.

Personalization made easy as a bunny slope

Thanks to the new combination of smart technology and user-focused design in the My Epic Assistant, Vail Resorts moves beyond simple customer Q&A to a truly conversational yet fully automated experience for many customer requests. It’s about providing the same concierge services resort-goers expect from an operator like Vail Resorts, yet achieving them at scale that’s impossible without AI and cloud technologies.

With pass recommendations from My Epic Assistant, Vail Resorts is gearing up every guest to receive custom recommendations for their best season yet.

Want to turn your technical black-diamond challenges into easy green groomers? Contact the experts at 66degrees and Google Cloud to discover what’s possible with the latest AI and cloud technologies.

How FM Logistic tackled the traveling salesman problem at warehouse scale with AlphaEvolve

Thu, 26 Mar 2026 16:00:00 +0000

The traveling salesman problem asks a deceptively simple question: What's the shortest route that visits every point exactly once? It's one of the hardest problems in computer science, and mathematicians have been working on it for nearly a century. It's also what FM Logistic's warehouse operators face every day in Poland.

The facility spans eight football fields. It holds over 17,700 picking locations. And across every shift, up to several dozen operators on ride-on electric trucks crisscross the floor collecting cartons, each one navigating dozens of storage locations per tour. Every unnecessary step adds up: in time, in wear on the fleet, and in delayed fulfillment.

FM Logistic, a global logistics provider operating in over 14 countries, had already optimized their routing once. Their existing model used a fast, cost-prioritized allocation logic built for real-time responsiveness. It worked well, but it made decisions step by step, which limited how well it could coordinate routes across the full warehouse. With dozens of operators working the same floor across shifts, even a small routing improvement would compound quickly.

So they turned to AlphaEvolve on Google Cloud.

Teaching an AI to write better algorithms

AlphaEvolve is an evolutionary coding agent that generates and refines algorithms autonomously using Gemini models. Rather than calculating a schedule from fixed rules, it works as a coding partner: writing new code, scoring it, and iterating until it finds a better solution than the one it started with.

The team didn't start from scratch. They gave AlphaEvolve a "seed" program: their existing algorithm, which made routing decisions one step at a time based on what looked best in the moment. This gave the agent a working baseline that already solved the problem, just not optimally. From there, AlphaEvolve used Gemini to generate variations of this code, introducing mutations and new logic to see if it could beat the human-designed original.

Measuring what good looks like

For AlphaEvolve to improve, it needs a way to measure how well each algorithm performs. FM Logistic designed a custom evaluation function using a representative dataset of 60 tours (over one hour of workforce data), letting the agent test thousands of generated algorithms against real-world conditions.

The evaluation scored every new piece of code on a primary goal: minimize the average travel distance per pick, while avoiding operational failures. The team built in specific penalties to steer the model away from unworkable solutions — things like exceeding forklift capacity, missing pending orders, assigning the same box twice, violating FIFO priority for older orders, or exceeding the computation time required for real-time operations.

The results

The new routing logic delivered immediate, measurable gains over the previous best baseline:

10.4% improvement in routing efficiency over the previous best solution.
15,000+ fewer kilometers of warehouse travel per year at full operational scale.

That efficiency gives FM Logistic room to handle larger order volumes with the same team and equipment, without adding headcount or expanding their fleet.

"Through our partnership with Google Cloud and the implementation of AlphaEvolve and Gemini, we further optimized our routing approach for fast-moving operations," said Rodolphe Bey, Group CIO at FM Logistic. "The 10.4% improvement was achieved on top of an already highly optimized baseline, where further gains are typically hard to come by. This translates directly to faster fulfillment, improved working conditions for our teams, and reduced wear on our fleet."

What the winning algorithm actually does

By running a series of experiments, each generating hundreds of candidate programs, AlphaEvolve developed a new algorithm that outperformed the previous best human-engineered one. The result is a set of clear, human-readable rules that warehouse teams can review and adjust as needs change.

The three core improvements:

Density-based starting points (Anchor selection): The previous system chose a starting mission based on the single location where the most missions overlapped. The new algorithm looks more broadly, identifying clusters of items that are close together and using those dense areas as "starting anchors" for building routes. Every tour begins with a highly efficient core.
Two-step filtering with distance simulation: To maintain real-time speed, the algorithm uses a two-stage process. First, a quick filter eliminates orders that do not fit the route's logic. Second, a precise distance simulation runs only on the best remaining candidates to find the most efficient path, without slowing down warehouse operations.
Flexible route building: If the algorithm can’t fill a truck efficiently around a specific starting point, it doesn’t force a bad route. It returns those orders to the main pool so they can be picked up by a better-fitting route later, improving efficiency across the entire warehouse.

What’s next

The Poland pilot (which is now running in production) demonstrated what evolutionary AI can do for complex routing at warehouse scale. FM Logistic is now exploring extensions — applying the algorithm to other high-volume e-commerce facilities, researching how AlphaEvolve might help optimize road transport for less-than-truckload shipments, and investigating AI-driven product placement inside warehouses to further cut travel distances.

^{This project was a collaboration between the FM logistic team including: Mateusz Klimowicz, Jarosław Urbański, Florent Martin and Alberto Brogio and the AI for Science team at Google Cloud including (but not limited to): Kartik Sanu, Laurynas Tamulevičius, Nicolas Stroppa, Chris Page, Gary Ng, John Semerdjian, Skandar Hannachi, Vishal Agarwal and Anant Nawalgaria as well asMariusz Czopiński from the Google account team as well and partners at Google DeepMind}

Inside Mercado Libre's multi-faceted Spanner architecture

Mon, 03 Nov 2025 17:00:00 +0000

Mercado Libre, the e-commerce and fintech pioneer of Latin America, operates at a staggering scale, demanding an infrastructure that's not just resilient and scalable, but also a catalyst for rapid innovation. While our use of Spanner for foundational consistency and scale is known, a deeper dive reveals a sophisticated, multi-layered strategy. Spanner is not just a database here; it's a core engine powering our internal developer platform, diverse data models, advanced analytics loops, intelligent features, and even our roadmap for next-generation AI applications.

This blog explores the technical underpinnings of how Mercado Libre leverages Spanner in concert with our internal innovations like the Fury platform, achieving significant business impact and charting a course for an AI-driven future.

The dual challenge: internet-scale operations and developer velocity

Mercado Libre faces the classic challenges of internet-scale services: keeping millions of daily financial transactions safe, making it easy for developers to build apps, and maintaining near-perfect uptime. The solution required a database powerful enough for the core and an abstraction layer elegant enough for broad developer adoption.

Fury: Mercado Libre's developer gateway

At the heart of Mercado Libre's strategy is Fury, our in-house middleware platform. Fury is designed to abstract away the complexities of various backend technologies, providing developers with standardized, simplified interfaces to build applications.

Abstraction & Standardization: Fury allows development teams to focus on business logic rather than the nuances of distributed database management, schema design for specific engines, or optimal connection pooling.
Spanner as the Reliable Core: Spanner is an always-on, globally consistent, multi-model database with virtually unlimited scale. By designating Spanner as a choice within Fury, Mercado Libre ensures that applications built on the platform using Spanner inherit its best features – they stay consistent globally, scale without breaking, and rarely go down.

Fig. 1 - Fury’s core services

Spanner – the versatile backbone

Through Fury, Spanner empowers Mercado Libre's developers with remarkable versatility. Some apps need complex transactions, others need fast lookups. Spanner handles both, which means teams can use just one system:

Relational prowess for complex transactions: For sophisticated transactional workloads like order management, payments, and inventory systems, Spanner’s relational capabilities (SQL, ACID transactions, joins) remain critical.
High-performance key-value store: Many modern applications require fast point lookups and simple data structures. While Spanner isn't Mercado Libre's default backend for typical key-value workloads, there are specific applications running large scale non-relational, KV-style workloads on the Spanner.

Spanner’s foundational architecture — TrueTime for global consistency and automated sharding for effortless scaling — makes it an ideal candidate to reliably serve both these access patterns through the Fury platform.

Handling peak demand

Mercado Libre's Spanner instances demonstrate significant processing capacity, handling around 214K queries per second (QPS) and 30K transactions per second (TPS). To manage this substantial workload, the Spanner infrastructure dynamically scales to over 400 nodes (by 30%), highlighting the robust and elastic nature of the underlying system in accommodating high-demand scenarios. This level of throughput and scalability is critical for maintaining the performance and reliability of Mercado Libre's services during its busiest times.

Fig. 2 - Diagram of the solution built with Spanner, which uses current search data to predict and recommend products that a customer is most likely to purchase.

Turning data into action

Mercado Libre builds a dynamic data ecosystem around Spanner, leveraging advanced analytics to feed insights directly back into operational systems.

They achieve real-time analytics by combining Spanner Data Boost with BigQuery Federation. Data Boost isolates analytical queries, preventing them from impacting critical transactional performance. This allows for powerful, large-scale analytics to run directly on fresh Spanner data within BigQuery, integrating seamlessly with other data sources.

Insights from BigQuery, such as customer segmentations or fraud scores, are then actioned via Reverse ETL, feeding directly back into Spanner. This enriches operational data, enabling immediate action by frontline applications like serving personalized content or performing real-time risk assessments.

Furthermore, Spanner Change Streams coupled with Dataflow drive crucial service integrations. By capturing real-time data modifications from Spanner, they establish robust pipelines. These enable loading changes into BigQuery for analytics or streaming them to services like Fury Stream for real-time consumption, ensuring low-latency data propagation and enabling event-driven architectures across their systems.

The impact: cost savings, agility, and future-proofing

The strategic adoption of Spanner, amplified by internal platforms like Fury and sophisticated data workflows, has yielded significant benefits for Mercado Libre:

Significant cost savings & low total cost of ownership: The combination of Spanner's managed nature (reducing manual sharding, maintenance, and maintenance work), efficient resource utilization, and the abstraction provided by Fury has led to a lower Total Cost of Ownership and substantial cost savings.
Business impact & agility: Developers, freed from infrastructure complexities by Fury and empowered by Spanner's versatile capabilities, can deliver new features and applications faster. The reliability of Spanner underpins critical business operations, minimizing disruptions.
Low operational overhead: Automated scaling, sharding, and maintenance in Spanner significantly reduce the human effort required to manage large-scale database infrastructure.

Building for AI: Next-generation applications on Spanner

Looking ahead, Mercado Libre is exploring Spanner to support more AI workloads.

Spanner's characteristics make it an ideal foundation:

Consistent state management: Critical for AI systems that need to maintain and reliably update their state context.
Scalable memory/knowledge store: Ability to store and retrieve vast amounts of data for AI system memory, logs, and contextual information.
Transactional operations: Enabling AI systems to perform reliable actions that interact with other systems.
Integration with analytics & Machine Learning (ML): The existing data loops and ML.PREDICT capabilities can enrich AI systems with real-time insights and intelligence.

Spanner provides the transactional foundation these sophisticated, AI applications will require.

Conclusion: A Unified, Intelligent Data Foundation

Mercado Libre's adoption of Spanner demonstrates how to use a powerful, globally consistent database not just for its core capabilities, but as a strategic enabler for developer productivity, operational efficiency, advanced analytics, and future AI ambitions. Through their Fury platform, they've simplified access to Spanner's capabilities, allowing it to serve as a flexible foundation for both relational and non-relational needs. The integration with BigQuery via Data Boost demonstrates a comprehensive approach to building an intelligent, data-driven enterprise. As Mercado Libre builds AI applications, Spanner is set to continue its role as the consistent and scalable foundation for their next wave of innovation.

Learn more

How AI can scale customer experience — online and IRL

Tue, 14 Oct 2025 21:00:00 +0000

Customer service teams at fast-growing companies face a challenging reality: customer inquiries are growing exponentially, but scaling human teams at the same pace isn’t always sustainable.

Intelligent AI tools offer a new path forward. They handle routine questions automatically so employees can focus on more complex customer service tasks that require empathy, judgment, and creative problem-solving.

LiveX AI enables businesses to build and deploy advanced AI systems that deliver natural conversational experiences at scale. These can show up as chat bots, call center agents — even 3D holographic personas in live settings.

To handle thousands of concurrent, real-time interactions with low latency requires infrastructure that is both powerful and elastic, especially when seamlessly escalating complex issues to human agents.

In this joint technical post, we'll share the technical blueprint LiveX AI uses to build and scale its intelligent customer experience systems on Google Cloud, demonstrating how the right combination of services makes this transformation possible.

Why this architecture matters: Proven ROI

This architecture delivers measurable business impact.

90%+ self-service rate for Wyze: Smart home leader Wyze deployed LiveX AI to achieve a 90%+ self-service rate, enabling their support team to focus on complex cases that require human expertise while improving the overall customer experience.
3x conversion for Pictory: The video creation platform Pictory saw a 3x increase in conversions by using LiveX AI to proactively engage and qualify website visitors.

These results are only possible through a sophisticated, scalable, and secure architecture built on Google Cloud.

Platform capabilities designed for scale

The LiveX AI platform is designed to be production-ready, enabling companies to easily deploy intelligent customer experience systems. This is possible through key capabilities, all running on and scaling with Google Cloud's Cloud Run and Google Kubernetes Engine (GKE):

AgentFlow orchestration: The coordination layer that manages conversation flow, knowledge retrieval, and task execution. It routes routine queries automatically and escalates complex issues to human agents with full context.
Multilingual by design: Built to deliver native-quality responses in over 100 languages, leveraging powerful AI models and Google's global-scale infrastructure.
Seamless integration: Connects securely to internal and external APIs, enabling the system to access account information, process returns, or manage subscriptions, giving human agents complete context when they step in.
Customizable knowledge grounding: Trained on specific business knowledge to ensure accurate and consistent responses aligned with team expertise.
Natural interface: Deployed via chat, voice, or avatar interfaces across web, mobile, and phone channels.

Figure 1: LiveX real-world 3D assistants

The technical blueprint: Building intelligent customer experience systems on Google Cloud

LiveX AI's architecture is intelligently layered to optimize for performance, scalability, and cost-efficiency. Here's how specific Google Cloud services power each layer.

Figure 2: LiveX AI customer service agent architecture on Google Cloud

The front-end layer

Managing real-time communication across web, mobile, and voice channels requires lightweight microservices that handle session management, channel integration, and API gateway services.

Cloud Run is the ideal platform for this workload. As a fully managed, serverless solution, it automatically scales from zero to thousands of instances during traffic spikes, then scales back down, so LiveX AI only pays for the computation they actually use.

The orchestration and AI engine

The platform's core, AgentFlow, manages the conversational state, interprets customer intent, and coordinates responses. When issues require human expertise, it routes them to agents with complete context. The system processes natural language input to determine customer intent, breaks down requests into multi-step plans, and connects to databases (like Cloud SQL) and external platforms (Stripe, Zendesk, Intercom, Salesforce, Shopify) so both AI and human agents have complete customer context.

Cloud Run for orchestration automatically scales based on request traffic, perfectly handling fluctuating conversational loads with pay-per-use billing.

GKE for AI inference provides the specialized capabilities needed for real-time AI:

GPU management: GKE's cluster autoscaler dynamically provisions GPU node pools only when needed, preventing costly idle time. Spot VMs significantly reduce training costs.
Hardware acceleration: Seamless integration with NVIDIA GPUs and Google TPUs, with Multi-Instance GPU (MIG) support to maximize utilization of expensive accelerators.
Low latency: Fine-grained control over specialized hardware and the Inference Gateway enable intelligent load balancing for real-time responses.

With this foundation, LiveX AI can serve millions of concurrent users during peak demand while maintaining sub-second response times.

The knowledge and integration layer

From public FAQs to secure account details, the knowledge layer provides all the information the system needs to deliver helpful responses.

The Doc Processor (on Cloud Run) builds and maintains the knowledge base in the vector database for the Retrieval-Augmented Generation (RAG) system, while the API Gateway manages configuration and authentication. For long-term storage, LiveX AI relies on Cloud SQL as the management database, while short-term context is kept in Google Cloud Memorystore.

Putting it all together

Three key advantages emerge from this design: elastic scaling that matches actual demand, cost efficiency through serverless and managed GKE services, and the performance needed for real-time conversational AI at scale.

Looking ahead: Empowering customer experience teams at scale

The future of customer service centers on intelligent systems that amplify what human agents do best: empathy, judgment, and creative problem-solving. Businesses that adopt this approach empower their teams to deliver the personalized attention that builds lasting customer relationships, freed from the burden of repetitive queries.

For teams evaluating AI-powered customer experience systems, this architecture offers a proven blueprint: start with Cloud Run for elastic front-end scaling, leverage GKE for AI inference workloads, and ensure seamless integration with existing platforms.

The LiveX AI and Google Cloud partnership demonstrates how the right platform and infrastructure can transform customer service operations. By combining intelligent automation with elastic, cost-effective infrastructure, businesses can handle exponential inquiry growth while enabling their teams to focus on building lasting customer relationships.

To explore how LiveX AI can help your team scale efficiently, visit the LiveX AI Platform.
To build your own generative AI applications with the infrastructure powering this solution, get started with GKE and Cloud Run.

From query to cart: Inside Target’s search bar overhaul with AlloyDB AI

Wed, 03 Sep 2025 16:00:00 +0000

Editor’s note: Target set out to modernize its digital search experience to better match guest expectations and support more intuitive discovery across millions of products. To meet that challenge, they rebuilt their platform with hybrid search powered by filtered vector queries and AlloyDB AI. The result: a faster, smarter, more resilient search experience that’s already improved product discovery relevance by 20% and delivered measurable gains in performance and guest satisfaction.

The search bar on Target.com is often the first step in a guest’s shopping journey. It’s where curiosity meets convenience and where Target has the opportunity to turn a simple query into a personalized, relevant, and seamless shopping experience.

Our Search Engineering team takes that responsibility seriously. We wanted to make it easier for every guest to find exactly what they’re looking for — and maybe even something they didn’t know they needed.

That meant rethinking search from the ground up.

We set out to improve result relevance, support long-tail discovery, reduce dead ends, and deliver more intuitive, personalized results.

As we pushed the boundaries of personalization and scale, we began reevaluating the systems that power our digital experience. That journey led us to reimagine search using hybrid techniques that bring together traditional and semantic methods and are backed by a powerful new foundation built with AlloyDB AI.

Hybrid search is where carts meet context

Retail search is hard. You’re matching guest expectations, which can sometimes be expressed in vague language, against an ever-changing catalog of millions of products. Now that generative AI is reshaping how customers engage with brands, we know traditional keyword search isn’t enough.

That’s why we built a hybrid search platform combining classic keyword matching with semantic search powered by vector embeddings. It’s the best of both worlds: exact lexical matches for precision and contextual meaning for relevance. But hybrid search also introduces technical challenges, especially when it comes to performance at scale.

Fig. 1: Hybrid Search blends two powerful approaches to help guests find the most relevant results

Choosing the right database for AI-powered retrieval

Our goals were to surface semantically relevant results for natural language queries, apply structured filters like price, brand, or availability, and deliver fast, personalized search results even during peak usage times. So we needed a database that could power our next-generation hybrid search platform by supporting real-time, filtered vector search across a massive product catalog, while maintaining millisecond-level latency even during peak demand.

We did this by using a multi-index design that yields highly relevant results by fusing the flexibility of semantic search with the precision of keyword-based retrieval. In addition to retrieval, we developed a multi-channel relevance framework that dynamically modifies ranking tactics in response to contextual cues like product novelty, seasonality, personalization and other relevance signals.

Fig. 2: High level architecture of the services benign built within Target

We had been using a different database for similar workloads, but it required significant tuning to handle filtered approximate nearest neighbor (ANN) search at scale. As our ambitions grew, it became clear we needed a more flexible, scalable backend that also provided the highest quality results with the lowest latency. We took this problem to Google to explore the latest advancements in this area, and of course, Google is no stranger to search!

AlloyDB for PostgreSQL stood out, as Google Cloud had infused the underlying techniques from Google.com search into the product to enable any organization to build high quality experiences at scale. It also offered PostgreSQL compatibility with integrated vector search, the ScaNN index, and native SQL filtering in a fully managed service. That combination allowed us to consolidate our stack, simplify our architecture, and accelerate development. AlloyDB now sits at the core of our search system to power low-latency hybrid retrieval that scales smoothly across seasonal surges and for millions of guest search sessions every day while ensuring we serve more relevant results.

aside_block: <ListValue: [StructValue([('title', 'Build smarter with Google Cloud databases!'), ('body', <wagtail.rich_text.RichText object at 0x7fae12ffb520>), ('btn_text', ''), ('href', ''), ('image', None)])]>

Filtered vector search at scale

Guests often search for things like “eco-friendly water bottles under $20” or “winter jackets for toddlers.” These queries blend semantic nuance with structured constraints like price, category, brand, sizes or store availability. With AlloyDB, we can run these hybrid queries that combine vector similarity and SQL filters easily without sacrificing speed or relevance.

Thanks to recent innovations in AlloyDB AI, including optimized filtered vector search and adaptive query filtering, we’ve seen:

up to 10x faster execution compared to our previous stack
product discovery relevance improved by 20%
halved the number of “no results” queries

These improvements have extended deeper into our operations. We’ve reduced vector query response times by 60%, which resulted in a significant improvement in the guest experience. During high-traffic events, AlloyDB has consistently delivered more than 99.99% uptime, providing us with the confidence that our digital storefront can keep pace with demand when it matters most. Since search is an external–facing, mission-critical service, we deploy multiple AlloyDB clusters across multiple regions, allowing us to effectively achieve even higher effective reliability. These reliability gains have also led to fewer operational incidents, so our engineering teams can devote more time to experimentation and feature delivery.

Fig 3: AlloyDB AI helps Target combine structured and unstructured data with SQL and Vector search. For example, this improved search experience now delivers more seasonally relevant styles (ie. Long Sleeves) on Page One!

AlloyDB’s cloud-first architecture and features give us the flexibility to handle millions of filtered vector queries per day and support thousands of concurrent users – no need to overprovision or compromise performance.

Building smarter search with AlloyDB AI

What’s exciting is how quickly we can iterate. AlloyDB’s managed infrastructure and PostgreSQL compatibility let us move fast and experiment with new ranking models, seasonal logic, and even AI-native features like:

Semantic ranking in SQL: We can prioritize search results based on relevance to the query intent.
Natural language support: Our future interfaces will let guests search the way they speak – no more rigid filters or dropdowns.

AlloyDB features state-of-the-art models and natural language in addition to the state-of-the-art ScaNN vector index. Google’s commitment and leadership in AI infused in AlloyDB has given us the confidence to evolve our service together with pace of the overall AI & data landscape.

The next aisle over: What's ahead for Target

Search at Target is evolving into something far more dynamic – an intelligent, multimodal layer that helps guests connect with what they need, when and how they need it. As our guests engage across devices, languages, and formats, we want their experience to feel seamless and smart.

With AlloyDB AI and Google Cloud’s rapidly evolving data and AI stack, we’re confident in our ability to stay ahead of guest expectations and deliver more personalized, delightful shopping moments every day.

Note from Amit Ganesh, VP of Engineering at Google Cloud:

Target’s journey is a powerful example of how enterprises are already transforming search experiences using AlloyDB AI. As Vishal described, filtered vector search is unlocking new levels of relevance and scale. At Google Cloud, we’re continuing to expand the capabilities of AlloyDB AI to support even more intelligent, agent-driven, multimodal applications. Here’s what’s new:

Agentspace integration: Developers can now build AI agents that query AlloyDB in real time, combining structured data with natural language reasoning.
AlloyDB natural language: Applications can securely query structured data using plain English (or French, or 250+ other languages) backed by interactive disambiguation and strong privacy controls.
Enhanced vector support: With AlloyDB’s ScaNN index and adaptive query filtering, vector search with filters now performs up to 10x faster.
AI query engine: SQL developers can use natural language expressions to embed Gemini model reasoning directly into queries
Three new models: AlloyDB AI now supports Gemini’s text embedding model, a cross-attention reranker, and a multimodal model that brings vision and text into a shared vector space.

These capabilities are designed to accelerate innovation – whether you’re improving product discovery like Target or building new agent-based interfaces from the ground up.

Learn more:

Google Cloud databases supercharge the AI developer experience
AlloyDB AI drives innovation for application developers
Discover how AlloyDB combines the best of PostgreSQL with the power of Google Cloud in our latest e-book.
Try AlloyDB at no cost for 30 days with AlloyDB free trial clusters!
Learn more about AlloyDB for PostgreSQL.

Lowe’s innovation: How Vertex AI helps create interactive shopping experiences

Tue, 29 Apr 2025 16:00:00 +0000

At Lowe's, we are always striving to make the shopping experience more enjoyable and convenient for our customers. A common challenge we’ve identified is that many shoppers visit our ecommerce site or mobile application without a clear idea of what they want but believe that they will recognize the right product when they see it.

To address this issue and enhance the shopping journey, we introduced Visual Scout — an interactive way to explore the product catalog and quickly find products of interest on lowes.com. It’s an exemplar of the ways AI recommendations are helping transform retail experiences today across many modes of communication — not just text but imagery, video, voice, and the combination of them all.

Visual Scout is designed for shoppers who value the visual aspects of products when making certain purchasing decisions. It offers an interactive experience for customers to discover a variety of styles within a product group. Visual Scout begins by presenting a panel of 10 items. Customers then indicate their preferences by “liking” or “disliking” items in the display. Based on this feedback, Visual Scout dynamically updates the panel, with items that reflect customer style and design preferences.

Here’s an example of how user feedback from a shopper looking for hanging lamps influences a discovery panel refresh.

The Visual Scout API interactively refreshes recommendation panels to reflect user feedback for currently displayed items

In this post, we will delve into the technical aspects and explore the essential technologies and MLOps practices that enable this experience.

How Visual Scout Works

When customers visit a product detail page on lowes.com, they typically have a general idea of the ‘product group’ they are looking for, but they could still have many product variants to possibly choose from. Instead of opening multiple browser windows or viewing a predefined comparison table, customers can use Visual Scout to sift through visually similar items and quickly arrive at a subset of interesting items.

For a given product page the item on that page will be treated as the “anchor item”, and this will seed the initial recommendation panel. From there, customers iteratively refine the displayed product set by providing either “like” or “dislike” feedback for individual items in the display:

"Like" feedback: If a customer selects the "more like this" button, Visual Scout replaces the two least visually similar products with items that closely match the one the customer just liked
"Dislike" feedback: Conversely, if a customer dislikes a product by clicking the 'X' button, Visual Scout replaces that product with a product that is visually similar to the anchor item

As the service updates in real time, Visual Scout provides an engaging and gamified shopping experience that encourages customer engagement and, ultimately, conversion.

Want to try it out?

To see Visual Scout in action, visit this product page and find the section titled “Discover Similar Items”. You don’t need to be logged in to an account, but be sure to select a store location in the upper left corner of the page. This helps Visual Scout nominate items available near you.

The technology behind Visual Scout

Visual Scout is supported by several Google Cloud services:

Dataproc: Batch processing jobs that compute embeddings for new items by feeding an item’s image to a computer vision model as a prediction request; the predicted values are the image’s embedding representation
Vertex AI Model Registry: Central repository for managing the lifecycle of the computer vision model
Vertex AI Feature Store: Feature management for product image embeddings, and low latency online serving
Vertex AI Vector Search: Deploys a serving index and performs vector similarity search for low latency online retrieval
BigQuery: Hosts an enterprise-wide, immutable record of item metadata (e.g., price, promotions, inventory, rating, availability in user’s selected store, restrictions, etc.)
Google Kubernetes Engine: Deploys and operates the Visual Scout application with the rest of the online shopping experience

To better understand how these components are operationalized in production, let’s review some key tasks in the following reference architecture:

Figure 2: Reference architecture for serving the Visual Scout API on lowes.com

The Visual Scout API creates a vector match request for a given item
The request first calls Vertex AI Feature Store to retrieve an item’s latest image embedding vector
Using the item embedding, Visual Scout then searches a Vertex AI Vector Search index for the most similar embedding vectors and returns the associated item IDs
For each visually similar item, product-related metadata (e.g., inventory availability) is used to filter for only items available at the user’s selected store location
The available items and their metadata are sent back to the Visual Scout API for serving on lowes.com
A daily trigger launches an update job to compute image embeddings for any new items
Once triggered, Dataproc processes any new item images, converting them to embeddings with the registered computer vision model
Streaming updates add new image embeddings to the Vertex AI Vector Search serving index
New image embedding vectors are ingested to Vertex AI Feature Store online serving nodes; vectors indexed by item ID and the timestamp of the ingestion

Low latency serving with Vertex AI

Each time items are replaced in the recommendation panel, Visual Scout relies on two Vertex AI services to do this in real time: Vector Search and Feature Store.

Vertex AI Feature Store is used to store the latest embedding representation of an item. This includes net new additions to the product catalog, as well as any new images that become available for an item. In the latter case, the previous embedding representation for an item is moved to offline storage, and the latest embedding is kept in online storage. At serving time, the Feature Store look-up retrieves the query item’s most up-to-date embedding representation from the online serving nodes, and passes this to the downstream retrieval task.

Next, Visual Scout must find, within a database of diverse items, the most similar products to the query item based on their embedding vectors. This kind of nearest neighbor search requires calculating the similarity between the query and candidate item vectors, and at this scale, this calculation can quickly become a computational bottleneck for retrieval, especially if performing an exhaustive (i.e., brute-force) search. To address this bottleneck, Vertex AI Vector Search implements an approximate search, allowing the vector retrieval to meet our low latency serving requirements.

Both of these services help Visual Scout process a high volume of requests while maintaining low-latency responses. The 99th percentile response times of approximately 180 milliseconds align with our performance expectations, and ensures a smooth and responsive user experience.

Why is Vertex AI Vector Search so fast?

Vertex AI Vector Search is a managed service providing efficient vector similarity search and retrieval from a billion-scale vector database. As these capabilities are critical to many projects across Google, this service builds upon years of internal research and development. It’s worth mentioning that several foundational algorithms and techniques are also publicly available with ScaNN, an open-source vector search library from Google Research. The purpose of ScaNN is to establish reproducible and credible benchmarking that ultimately advances research in the field. The purpose of Vertex AI Vector Search is to provide a scalable vector search solution for production-ready applications.

ScaNN primer

ScaNN provides an implementation of Google Research’s 2020 ICML paper, “Accelerating Large-Scale Inference with Anisotropic Vector Quantization”, which applies a novel compression algorithm to achieve state-of-the-art performance on nearest neighbor search benchmarks. ScaNN’s high-level workflow for vector similarity search can be described in four phases:

Partitioning: To reduce the search space, ScaNN performs hierarchical clustering to partition the index and represent its contents as a search tree, where each partition is represented by the partition’s centroids. This is usually (but not always) a k-means tree
Vector quantization: using the asymmetric hashing (AH) algorithm, this step compresses each vector into a sequence of 4-bit codes, where ultimately a codebook is learned. Its “asymmetric” because only database vectors are compressed, not the query vectors
Approximate scoring: at query time, AH creates partial-dot-product lookup-tables; uses tables to estimate <query, db-vector> dot products
Rescoring: given top-k items from approximate scoring, re-compute distances with greater precision (e.g., lower distortion or even raw datapoint)

Building an index optimized for serving

To build an index optimized for low-latency serving, Vertex AI Vector Search uses ScaNN’s tree-AH algorithm. The term “tree-AH” refers to a tree-X hybrid model consisting of (1) a partitioning “tree” and (2) a leaf searcher (in this case “AH” or asymmetric hashing). Essentially, it combines two complementary algorithms:

Tree-X, a k-means tree; a hierarchical clustering algorithm that reduces the search space by partitioning the index into a search tree, where each partition in the tree is represented by the centroid of the data points belonging to that partition.
Asymmetric Hashing (AH), a highly optimized approximate distance computation routine used to score the similarity between a query vector and the partition centroids at each level of the search tree

Figure 3: Conceptually, ‘tree-X hybrids’ combine (1) a partitioning tree and (2) a leaf searcher, where the leaf searcher is used to search and score the tree.

With tree-AH, it trains to learn an optimal indexing model that essentially defines the partition centroids and quantization codebook of the serving index. And this is even further optimized when training with an anisotropic loss function. The reason is that anisotropic loss emphasizes reducing the quantization error for vector pairs with high dot products. This makes sense because if the dot product for a <query, db-vector> vector pair is low then it is unlikely to be in the top-k, and thus the quantization error is not important. However, if a vector pair has a high dot product, we want to be even more careful about its quantization error because we want to preserve its relative ranking.

To summarize the last point:

There will be quantization error between the original vector and its quantized form
Preserving the relative ranking of vectors leads to higher recall during inference
We can be more precise about preserving the relative ranking of a subset of vectors at the expense of being less precise about preserving the relative ranking of another subset of vectors

For more details on the methods and implications of anisotropic loss, see Google Research’s blog, Announcing ScaNN: Efficient Vector Similarity Search, or the previously mentioned whitepaper.

Supporting production-ready applications

As a managed service, Vertex AI Vector Search let’s users take advantage of ScaNN performance while offering additional capabilities to alleviate operational overhead and deliver business value, including:

Real-time index updates - update indexes and metadata, query them in a matter of seconds
Multi-index deployments - deploy multiple indexes to a single endpoint (sometimes referred to as “namespacing”)
Autoscaling - ensures consistent performance at scale by automatically resizing serving nodes based on QPS traffic
Dynamic rebuilds - periodic index compaction to account for new updates; improves query performance and reliability without interpreting service
Full metadata filtering and diversity- restrict query results with strings, numerical values, allow lists, and deny lists; enforce diversity with crowding tags

Get started with Vector Search and Feature Store

If you’re looking to improve your customer experience with real-time personalization, a combination of Vertex AI’s Vector Search and Feature Store is the right choice. We continue to invest in these services because they are foundational components to many production AI workloads, and are used in many deployments across Google, and Lowe’s!

To get started with Vertex AI Vector Search, check out these additional resources:

See the Vertex AI Vector Search quickstart to learn how to create, deploy, and query an index
Read our recent blog post, What is Multimodal Search: “LLMs with vision” change businesses, to learn how Vector Search can be used for multimodal search
Access hands-on tutorials for deploying, tuning, and serving Vector Search indexes with notebook tutorials and the Vector Search documentation

To get started with Vertex AI Feature Store, check out these additional resources:

For a conceptual introduction, see Introduction to feature management in Vertex AI
Learn about our latest product features and roadmap in New Vertex AI Feature Store built with BigQuery, ready for predictive and generative AI
Access hands-on code tutorials for different use cases in our notebook tutorials

And to learn more about what happens when using these together, see Enabling real-time AI with Streaming Ingestion in Vertex AI.

The team would like to acknowledge the contributions Google Cloud retail AI specialiast Jordan Totten.

Where’s the beef? For São Paulo’s agricultural secretariat, it’s on Cloud SQL for SQL Server

Fri, 14 Feb 2025 17:00:00 +0000

When most people think of São Paulo, business and culture usually come to mind, not beef and chicken. But the state of São Paulo isn’t only home to the largest city in the hemisphere — it’s also the second largest producer of meat in a country that’s the second largest agricultural exporter in the world. Given the importance of agribusiness to Brazil’s economy, the Secretariat of Agriculture and Supply of the State of São Paulo (SAA-SP) plays a fundamental role in the development of agribusiness across the region and, by extension, the country.

With the mission of promoting the sustainable production of food, fibers and bioenergy, SAA-SP offers support to rural producers in several areas, such as technical assistance, research, agricultural defense, and access to markets. The Secretariat is also responsible for ensuring food security for the population, monitoring the quality of agricultural products and promoting nutritional education actions.

As the world’s food systems have evolved and grown more complex, organizations have looked to technology to help meet the goals for food security and sustainability. In the case of SAA-SP, the secretariat needs to securely manage increasing amounts of confidential data and ensure its critical systems are available 24/7. These systems include the Rural Environmental Registry (a mandatory electronic registry for all rural properties), and GEDAVE (a management system for animal and plant monitoring).

To give a sense of just how complex the system is, in one example, GEDAVE would handle controls for the management of poultry production, whereby each chick, after birth, needs to be transferred to a new location within 24 hours — and the entire process must be rigorously documented to ensure food safety.

If it wanted to meet the needs not only of its aging IT infrastructure but also the needs of a growing global population in need of safe, reliable food sources, the SAA-SP knew it was time to modernize some of our most important systems.

aside_block: <ListValue: [StructValue([('title', '$300 in free credit to try Google Cloud databases'), ('body', <wagtail.rich_text.RichText object at 0x7fae20786340>), ('btn_text', 'Start building for free'), ('href', 'http://console.cloud.google.com/freetrial?redirectPath=/products?#databases'), ('image', None)])]>

The growing pains:

Our team in the Department of Systems Management sits within the Information Technology Coordination organization of the SAA-SP. We’re in charge of operating GEDAVE, which is a crucial system for SAA-SP that’s responsible for controlling and monitoring animal and plant health throughout the state of São Paulo.

GEDAVE records and manages data on animal movement, plant production, use of pesticides, vaccination, pest and disease control, among other information relevant to São Paulo's agriculture. GEDAVE assists in issuing documents such as the Animal Transit Guide (GTA) and the Phytosanitary Certificate of Origin (CFO), which are essential for the trade of agricultural products.

GEDAVE's back-end was developed in Java and connected to a SQL Server database. It contains sensitive information about rural producers, such as production data, management strategies, and financial information. Previously, this database was hosted on-premises, which caused a series of issues, including:

Difficulty in keeping the database up to date: Applying patches and security updates in the on-premises environment required time and planning, resulting in periods of system unavailability, directly impacting producers who depend on SAA-SP services.
Complexity in performing regular backups: Ensuring data security with regular and reliable backups was a complex and laborious process in the on-premises environment.
Challenging high availability: Maintaining high availability of the on-premises database required investments in redundant and complex infrastructure, increasing management costs and complexity.

In addition, SAA-SP needed to ensure the 24/7 availability of these systems to help producers meet market demands, including such complex issues as quality control for export and monitoring internal production.

Data security was also crucial, as information on types of herds, vaccination strategies, pest control, among others, is highly sensitive and requires rigorous protection.

Sowing the seeds of innovation:

SAA-SP decided to modernize its data infrastructure to address these challenges, choosing Google Cloud. They felt the Google Cloud platform's high performance could ensure application availability and efficiency, while its ease of management would simplify database administration and allows the IT team to focus on other priorities.

As a first step in this modernization, SAA-SP migrated its SQL Server database to Cloud SQL for SQL Server on Google Cloud. A crucial factor in the choice was the ease of enabling high availability (HA) in Cloud SQL for SQL Server. With just a few clicks, SAA-SP configured automatic database replication and failover, ensuring service continuity in the event of failures and compliance with SLAs, without the need for complex configurations. In addition, the migration to Cloud SQL for SQL Server was carried out quickly and easily, minimizing the impact on SAA-SP's operations.

This strategic change brought a series of benefits, allowing Java applications to connect to a more modern, scalable and secure database environment.

Harvesting success:

Simplified updates: Cloud SQL for SQL Server makes it easier to apply patches and updates, minimizing downtime and ensuring that systems are always protected with the latest versions of the software.
Automated backups: The service offers automated and managed backups, ensuring data security and recovery in the event of failures.
Simplified high availability: The simplified configuration of high availability in Cloud SQL for SQL Server reduced the effort of the IT team and ensured compliance with service SLAs.
Enhanced security: With data encryption at rest and in transit, Cloud SQL for SQL Server protects SAA-SP’s confidential information from unauthorized access.
On-demand scalability: SAA-SP can adjust Cloud SQL for SQL Server resources according to demand, ensuring optimal performance of Java applications, even during peak periods.
Focus on innovation: SAA-SP’s IT team can now focus on strategic projects, such as developing new features for Java applications, instead of worrying about managing the data infrastructure.
Reduced IT costs: The migration to Cloud SQL for SQL Server eliminated the need to invest in hardware and software to maintain the on-premises database, reducing operational costs.

Cultivating a future of innovation in agriculture with Cloud SQL for SQL Server:

The migration to Cloud SQL for SQL Server was a strategic decision that allowed SAA-SP to overcome the challenges of on-premises data management and ensure the availability, security, and scalability of its critical systems. The ease of enabling high availability and the simplicity of the migration were determining factors for the success of the project.

But more than that, Cloud SQL enabled innovation at SAA-SP, opening doors to integration with generative AI for more assertive and efficient analysis and decision-making. For example, SAA-SP is leveraging the power of Gemini with Looker to provide C-level executives with real-time data insights hosted on Cloud SQL, facilitating data-driven decisions.

Furthermore, SAA-SP is empowering its customers with Gemini Database, allowing them to harness AI to enhance database performance and maintenance.

SAA-SP plans to continue modernizing its infrastructure and services, undertaking:

Migration to microservices: Launch an updated version of the microservices-based application to increase the flexibility, scalability and capacity of the system.
Data analysis with generative AI: Enable the use of generative AI to perform predictive analysis and obtain real-time insights from Cloud SQL for SQL Server data, assisting in strategic decision-making for the agricultural sector.
Data management with Gemini: Use Gemini to facilitate data management and analysis, extracting relevant information and simplifying access to complex data.

SAA-SP’s move towards intelligent management of operations, coupled with the advancements in analysis, has consolidated SAA-SP's position as a reference in technology and innovation in the agricultural sector, driving the development of agribusiness across São Paulo and serving as a beacon for those around the world.

Get Started:

Discover how Cloud SQL for SQL Server can enhance your application performance and ensure uninterrupted availability.
Read more on how others like Ford and Visual Research are modernizing their workloads with Cloud SQL for SQL Server, resulting in high performance and cost reduction.
Ready to unlock the power of Cloud SQL? Start a free trial today!

How L’Oréal's tech accelerator built its end-to-end MLOps platform

Thu, 23 Jan 2025 17:00:00 +0000

Technology has transformed our lives and social interactions at an unprecedented speed and scale, creating new opportunities. To adapt to this reality, L'Oréal has established itself as a leader in Beauty Tech, promoting personalized, inclusive, and responsible beauty accessible to all, under the banner "Beauty for Each, powered by Beauty Tech."

This convergence of Beauty Tech is evident in augmented beauty products, smart devices, enhanced marketing, online and offline services, and digital platforms, all powered by information and communication technologies, data, and artificial intelligence. L'Oréal is committed to developing innovative solutions that elevate the beauty experience and contribute to a future where beauty is accessible, sustainable, and caters to the diverse needs and aspirations of individuals worldwide.

L'Oréal, the world’s largest cosmetics company, has for years leveraged AI to enhance digital solutions for its employees and provide personalized experiences for customers. In this blog, we will describe how L'Oréal’s Tech Accelerator built a scalable and end-to-end MLOps platform using Google Cloud. This platform accelerates the deployment of AI models, enabling the team to rapidly innovate.

aside_block: <ListValue: [StructValue([('title', '$300 in free credit to try Google Cloud AI and ML'), ('body', <wagtail.rich_text.RichText object at 0x7fae1e296760>), ('btn_text', 'Start building for free'), ('href', 'http://console.cloud.google.com/freetrial?redirectPath=/vertex-ai/'), ('image', None)])]>

Our MLOps vision and requirements

To accelerate AI initiatives and optimize product development, L'Oréal Tech Accelerator sought to build a reusable, secure, and user-friendly Machine Learning Operations (MLOps) platform on Google Cloud. This platform aims to:

Streamline workflows and enhance collaboration, reducing friction between teams and accelerating time to market.
Ensure security and best practices that promote consistent, well-documented processes to minimize errors.
Enable rapid adoption through an intuitive platform that requires minimal training.

This approach fosters a more cohesive and efficient development environment, ultimately leading to higher quality products and greater agility in responding to evolving business needs.

Overview of L'Oréal’s MLOps platform

To understand the Tech Accelerator’s MLOps platform, let's break down its key components. Here's a simplified view of the process:

Labeled data preparation: Labeled data is gathered from various sources, including BigQuery, Google Cloud Storage, on-premise systems, and data lakes. It’s then processed and stored in a centralized location (such as BigQuery or Google Cloud Storage) to prepare it for training ML models
Training pipeline development: The team uses the Kubeflow SDK to define the flow and logic for the training pipeline. This pipeline automates the process of training the ML model.
Run training pipeline: The training pipeline is executed, generating a trained model artifact. This artifact is stored as a pickle file embedded in a Python library for easy access and deployment.
Prediction pipeline development: Using the Kubeflow SDK again, the team creates a prediction pipeline that utilizes the trained model to generate inferences on new data.
Run predict pipeline: The prediction pipeline is executed, generating inferences that are stored in BigQuery, Google Cloud Storage, or a data lake.
Validate trained model: The inference results from the prediction pipeline are used to evaluate the performance of the trained model. This involves calculating key accuracy metrics like F1-score and precision.
Push prediction pipeline to production: Up to this point, all pipeline components have been developed, tested, and validated manually by data scientists or ML engineers, and a new model version (or versions) has been created. The next step is to push the new version of the prediction pipeline, incorporating the new model version, to production. This deployment leverages development best practices, such as CI/CD pipelines.

The MLOps platform leverages DevOps principles to ensure a robust and efficient development lifecycle. This involves separating the ML development process into four distinct environments (i.e. Google Cloud projects):

DataOps: This environment provides a centralized repository for storing and managing all data assets, including labelled training data, model artifacts, and pipeline components. This ensures data consistency and accessibility throughout the ML workflow. Additionally, in this environment, the training pipeline is run to create the new version of the models.
Development: A dedicated space for testing new versions of prediction pipelines orchestrating multiple models. This environment allows for evaluating computation speeds, data coherence, end-to-end integration, and other performance aspects.
Staging: This environment mirrors the production setup, enabling rigorous testing and validation of the business expectations and requirements. By using staging data that closely resembles real-world data, potential business issues of prediction pipelines can be identified and addressed early on.
Production: The live environment where validated prediction pipelines and new versions of the models are deployed to generate real-time/batch predictions for L'Oréal's Tech Accelerator applications and services, delivering value to end-users.

This structured approach, with its clear separation of environments, promotes efficient collaboration, minimizes risks, and ensures a smooth transition from development to production, ultimately enabling L'Oréal's Tech Accelerator to deliver high-quality AI-powered beauty experiences. Note that, to further optimize efficiency and reduce costs, the training pipeline is executed only once within the DataOps environment. The resulting trained model is then deployed across the other environments. This eliminates the need to retrain the model in each environment, resulting in a significant cost reduction (up to 3x).

The figure above illustrates the relationship among the multiple environments and the required infrastructure. Notable points:

Model training pipelines output Python packages that embed the trained models.
The CI/CD pipeline outputs Kubeflow Pipelines (KFP) pipeline definitions and Docker images related to their components.
There are two distinct operational blocks: "Training" for creating new models and "Inference" for generating predictions.

Diving Deeper: Key Components of the MLOps Platform

At the core of the platform's operation is KFP. To understand its role, let's define what Vertex pipelines are:

“A pipeline is a definition of a workflow that composes one or more components together to form a computational directed acyclic graph (DAG). At runtime, each component execution corresponds to a single container execution, which may create ML artifacts. Pipelines may also feature control flow.”
— Kubeflow Documentation

In this section, we'll focus on how L'Oréal Tech Accelerator builds and manages the two main operational building blocks: "Training" and "Inference."

Training pipeline

The training pipeline architecture is designed for efficiency and reproducibility. Here's how it works:

Pipeline Definition and Components: The pipeline's definition is fetched from a KFP artifact registry, while the container images that execute individual pipeline steps are retrieved from a Docker artifact registry. These artifacts are created and managed by a CI/CD pipeline, ensuring version control and consistency (as described in the “MLOps platform overview” section).
Model Training and Packaging: Once a new training pipeline run completes, the newly trained model is packaged into a Python library for easy deployment and integration.
Model Registry: This packaged model is then pushed to a Python artifact registry, creating a centralized repository of trained models. This allows for easy versioning, sharing, and deployment of models across different environments.

Inference pipeline

The inference pipeline follows a similar architecture to the training pipeline, ensuring consistency and efficiency in model deployment. Here's how it works:

Pipeline Definition and Components: The inference pipeline's definition, defined using KFP, is retrieved from an artifact registry. Similarly, the Docker images containing the necessary components for the pipeline are fetched from another artifact registry.
CI/CD Integration: These pipeline definitions and Docker images are created and deployed by the CI/CD pipeline, ensuring that the inference pipeline is always up-to-date and uses the latest validated components.

The modularity and dependency challenge

Traditional ML pipelines often rely on a single, shared codebase for their definition. This can lead to challenges when multiple teams need to collaborate and contribute to the pipeline's development. Having all these teams work on the same codebase can create friction and slow down the development process due to:

Merge conflicts: When multiple teams edit the same files simultaneously.
Integration challenges: Ensuring the different components developed by separate teams work together seamlessly.
Version control complexities: Managing different versions and updates of the pipeline.
Deployment bottlenecks: Coordinating deployments when different teams need to make changes.

For example (see figure above), if two teams are working on separate models (Model 1 and Model 2) within the same codebase, and one model's pipeline fails, it can prevent the other model's inference pipeline from running. This creates a single point of failure that can disrupt the entire system.

To address this, a more modular and independent approach to pipeline development is needed, where individual teams can work on their components without affecting others.

The figure above illustrates the ML pipeline definition and infrastructure for the example and issue explained previously

How we solved it

L’Oréal Tech Accelerator solution uses KFP artifact registry to enable a modular approach to pipeline development. This allows the creation of independent sub-pipelines, each with its own codebase and CI/CD pipeline. This separation offers significant benefits:

Independent Development: Teams can work autonomously on their sub-pipelines without interfering with each other’s progress or deployments. This reduces friction and accelerates development cycles.
Isolated Testing and Versioning: Each sub-pipeline can be tested and versioned independently, ensuring that changes in one component don’t inadvertently affect others.
Increased Agility: This modularity enables teams to quickly adapt and update their sub-pipelines without impacting the overall system.

Furthermore, L'Oréal's Tech Accelerator introduces an additional codebase that acts as an orchestrator. This orchestrator assembles the individual sub-pipelines into a cohesive workflow, using the output artifacts of each sub-pipeline as building blocks. This approach combines the benefits of independent development with the power of a unified pipeline.

The figure above illustrates the ML pipeline definition and infrastructure of the solution

Example: Code snippet of an aggregated pipeline

The following code snippet demonstrates the simplicity of using an aggregation module to combine multiple prediction pipelines and models from different teams. This orchestration layer allows for seamless integration of individual components into a unified workflow.

code_block: <ListValue: [StructValue([('code', '# Fetching artifact pipeline from registry\r\n\r\nartifact_ml_model_1_pipeline = RegistryClient(host=KFP_REGISTRY_URL).download_pipeline(\r\n package_name=’ml_model_1_pipeline’,\r\n tag=1.0.0\r\n)\r\n\r\nartifact_ml_model_2_pipeline = RegistryClient(host=KFP_REGISTRY_URL).download_pipeline(\r\n package_name=’ml_model_2_pipeline’,\r\n tag=1.0.0\r\n)\r\n\r\n# Load artifact pipeline\r\n\r\nml_model_1_pipeline = kfp.components.load_component_from_file(artifact_ml_model_1_pipeline)\r\nml_model_2_pipeline = kfp.components.load_component_from_file(artifact_ml_model_2_pipeline)\r\n\r\n# Pipeline usage\r\n\r\n@pipeline\r\ndef aggregated_pipeline():\r\n ml_model_1_pipeline(params={…})\r\n ml_model_2_pipeline(params={…})'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7fae18194310>)])]>

Conclusion & Next steps

L'Oréal's modular MLOps platform, built on Google Cloud, has significantly boosted efficiency and agility in the AI development process. By empowering teams to work independently on their respective ML models, L'Oréal's Tech Accelerator has accelerated development, improved collaboration, and enhanced the quality and reliability of its systems.

While the current platform offers significant advantages, the team continues to optimize it.

One focus area is addressing the challenges of large model artifacts, which can increase Docker image sizes and slow down pipelines. L'Oréal's Tech Accelerator is exploring solutions like on-demand model downloading and API-driven inference to mitigate this and remain at the forefront of Beauty Tech innovation.

^{The authors would like to thank and acknowledge the following contributors to this blog: Kerebel Paul-Sirawit, DevOps & Cloud Lead, L’Oréal, Dr. Sokratis Kartakis, Generative AI Blackbelt, Google, Christophe Dubos, Principal Architect, Google.
Opening image credits: Ben Hassett / Myrtille Revemont / Helena Rubinstein pour L’Oréal}

How inference at the edge unlocks new AI use cases for retailers

Mon, 13 Jan 2025 17:00:00 +0000

For retailers, making intelligent, data-driven decisions in real-time isn’t an advantage — it's a necessity. Staying ahead of the curve means embracing AI, but many retailers hesitate to adopt because it’s costly to overhaul their technology. While traditional AI implementations may require significant upfront investments, retailers can leverage existing assets to harness the power of AI.

These assets, ranging from security cameras to point-of-sale systems, can unlock store analytics, faster transactions, staff enablement, loss prevention, and personalization — all without straining the budget. In this post, we’ll explore how inference at the edge, a technique that runs AI-optimized applications on local devices without relying on distant cloud servers, can transform retail assets into powerful tools.

In this blog, we will explore how retailers can leverage inference at the edge to enhance their operations and customer experience, including how retailers can leverage the power of cloud infrastructure, AI, and edge hardware powered by Intel Xeon processors with Google Distributed Cloud.

aside_block: <ListValue: [StructValue([('title', 'Try Google Cloud for free'), ('body', <wagtail.rich_text.RichText object at 0x7fae12edddf0>), ('btn_text', 'Get started for free'), ('href', 'https://console.cloud.google.com/freetrial?redirectPath=/welcome'), ('image', None)])]>

How retailers can build an AI foundation

Retailers can find assets to fuel their AI in all corners of the business. You can unlock employee productivity by transforming your vast repository of handbooks, training materials, and operational procedures into working assets for AI.

Digitized manuals for store equipment, human resources, loss prevention, and domain-specific information can also be combined with agent-based AI assistants to provide contextually aware “next action assistants”. By extending AI optimized applications from the cloud to the edge, retail associates can now ask their AI assistant, “What do I do next?” with a detailed and fast response tailored to the retail associate's question.

Edge processing power decision point: CPU vs GPU

Next, we’ll explore the critical decision on the right hardware to power your applications. The two primary options are CPUs (Central Processing Units) and GPUs (Graphics Processing Units), each with its own strengths and weaknesses. Making the informed choice requires understanding your specific use cases and balancing performance requirements, bandwidth, and model processing with cost considerations. Consider this chart to guide your decision-making process, especially when choosing between deploying at a regional DC or at the edge.

Decision matrix (chart):

Feature	CPU	GPU	Use cases (examples)
Cost	Lower	Higher	Basic analytics, people counting, simple object detection
Performance	Required; Good for general-purpose tasks	Optional; Good for parallel processing	Complex AI, video analytics, high-resolution image processing, ML model training
Power consumption	Lower	Higher	Remote locations, small form-factor devices
Latency	Moderate	Lower (for parallel tasks)	Real-time applications, immediate insights
Deployment location	Edge or Regional DC	Typically Edge, but feasible in Regional DC	Determined by latency, bandwidth, and data processing needs

Key decision criteria for retail decision makers

Complexity of AI models: Retail use case focused AI models, like basic object detection, can often run efficiently on CPUs. More complex models, such as those used for real-time video analytics or personalized recommendations with large datasets, typically require the parallel processing power of GPUs.
Data volume and velocity: If you're processing large amounts of data at high speed, a GPU may be necessary to keep up with the demand. For smaller datasets and lower throughput, a CPU may suffice.
Latency requirements: For use cases requiring ultra-low latency, such as real-time fraud detection, GPUs can provide faster processing, especially when located at the edge, closer to the data source. However, network latency between the edge and a regional DC might negate this benefit if the GPU is located regionally.
Budget: GPUs usually have a higher price tag than CPUs. Carefully consider your budget and the potential ROI of investing in GPU-powered solutions before making a decision. Start with CPU-based solutions where possible and upgrade to GPUs only when absolutely necessary.
Power consumption: GPUs generally consume more power than CPUs. This is an important factor to consider for edge deployments, especially in locations with limited power availability. This is less of a concern if deploying at a regional DC where power and cooling are centralized.
Deployment location: The proximity of the processing power to the data source has major implications for latency. Deploying at the edge (in-store) minimizes latency for real-time use cases. Regional DCs introduce network latency, making them less suitable for applications requiring immediate action. However, certain tasks requiring heavy compute but not low latency (e.g., nightly inventory analysis) might be better suited for a regional DC where resources can be pooled and managed centrally.

Remember, not all AI and ML require new investments in emerging technology. Many AI/ML based use cases can produce the desired outcome without using a GPU. For example, consider visual inspection for storage analytics and fast check out referenced in the Google Distributed Cloud Price-a-Tray interactive game. The inference is performed at 5FPS, while the video stream continues to run at 25FPS. The bounding boxes are then drawn on top of the returned information rather than having one system perform the video stream, detection and bounding boxes. This enables more efficient use of the CPU since many of the actions in this example can be split across cores and threads.

But there are cases when GPUs do make sense. When very high precision is required, GPUs are often needed as the drop in fidelity to quantize a model may reduce the quality beyond acceptable thresholds. In the example of tracking an item, if millimeter movement accuracy is required, 5FPS would not be sufficient on a reasonably fast moving item and a GPU would likely be required.

There is a middle between GPUs and CPUs—the world of speciality accelerators. Accelerators come in the form of peripherals to a system or as special instruction sets to a CPU. CPUs are being manufactured with advanced matrix multiplication math assisting tensor manipulation on-chip, greatly improving performance of ML and AI models. One concrete example is running models compiled for OpenVINO. In addition, Google Distributed Cloud (GDC) Server and Rack editions utilize Intel Xeon processors, an architecture designed to be more flexible, supporting matrix math improving the performance of ML models on CPU over traditional ML model service serving.

Bring AI to your business

By tapping into the power of existing infrastructure and deploying AI at the edge, retailers can deliver modern customer experiences, streamline operations, and unlock employee productivity.

Learn more about how to transform your retail brand with Google Distributed Cloud.

Empowering retailers with AI for commerce, marketing, supply chains, and more

Sun, 12 Jan 2025 13:00:00 +0000

Google Cloud’s mission is to accelerate every organization’s ability to digitally transform its business and industry — and a key part of doing that is with our ISV and service partners, who possess critical industry knowledge and technical expertise. To provide customers with the most advanced ecosystem of solutions across industries, we’ve enabled these partners to easily build and scale products on our platform. Many are deeply engaged with our AI technology to deliver new and novel AI solutions directly to our customers and theirs.

Today, at the annual National Retail Federation (NRF) conference, we wanted to highlight more than 20 ISV and services partners that are utilizing Vertex AI, Gemini models, and other Google Cloud technologies to empower retail businesses with the tools they need to transform how employees work and shoppers engage with their brands.

Generative AI has already had a significant impact on the retail industry by enabling businesses to run more personalized marketing campaigns, increase sales via improved search capabilities, and enhance customer service experiences through more accurate and tailored resolutions. In many cases, AI agents are helping these businesses move beyond predictive capabilities to performing tasks autonomously.

At NRF, we’re excited to showcase the breadth of our ecosystem of retail partners and spotlight the ways they are enabling customer success using technology from Google Cloud.

aside_block: <ListValue: [StructValue([('title', 'Try Google Cloud for free'), ('body', <wagtail.rich_text.RichText object at 0x7fae207a1a60>), ('btn_text', 'Get started for free'), ('href', 'https://console.cloud.google.com/freetrial?redirectPath=/welcome'), ('image', None)])]>

Transforming marketing with AI-powered data

AI is helping retailers get significantly more value from business data, enabling them to create personalized campaigns at scale, increase ROI with data-driven insights, and build more predictive and advanced audience segments. Partners are using Vertex AI, Gemini models, and BigQuery to let customers unlock the true potential of their data to optimize revenue and more effectively grow their businesses.

Eagle Eye delivers its AI-powered omnichannel personalization solution, built on Vertex AI, with built-for-retail algorithms to generate personalized promotions at scale that drive loyalty and customer engagement across channels.
LiveRamp provides a data collaboration platform that allows companies to enrich, activate, and analyze customer data while protecting brand and consumer trust
Revieve offers multiple solutions tailored for beauty retailers and brands that provide real-time consumer interactions, next gen AI, conversational AI, and data-informed product discovery.
Revionics’ price optimization suite utilizes Gemini and Vertex AI to power conversational analytics that enable customers to engage with their retail data using natural language search, such as “which competitor changes prices most frequently” and "which products are priced higher than competitors.”

Optimizing unified commerce experiences

Unified commerce experiences equip retailers with a more holistic view of front- and back-end systems to have complete visibility of the customer, inventory, and orders across all retail channels. With Google Cloud technology like BigQuery and embedded ML, partners are helping customers enhance decision-making processes and create stronger brand loyalty and revenue growth.

BigCommerce uses Google Cloud AI within BigAI Product Recommendations, which enables brands to offer shoppers real-time, personalized recommendations and can boost conversion and average order value.
Bloomreach uniquely integrates customer and product data within its real-time AI solution, enabling more personalized marketing, product discovery, advertising content, and conversational shopping experiences.
commercetools is a global leader in composable commerce and empowers businesses to customize, scale, and optimize shopping experiences with solutions that help retailers reduce risks and costs, and expand growth through exceptional customer experiences.
Everseen Vision AI platform and applications reduce retail shrink, improve inventory accuracy, enhance customer service, and provide data-driven insights, contributing to retailers’ ROI and a streamlined shopping experience.
Quantum Metric provides a digital analytics platform that enables businesses to more easily monitor, troubleshoot, and optimize their customers’ digital journeys while leveraging gen AI to enhance user retention, conversion rates, and much more.
Shopify is the leading global commerce company with a platform engineered for speed, customization, reliability, and security for businesses of any size, and a better experience for consumers everywhere they shop.

Creating sustainable supply chains

AI-powered tools for supply chains and logistics are enabling retailers to drive more sustainable and efficient operations, scale automation, and reduce their carbon footprint across the entire value chain. Partners are leveraging Vertex AI and BigQuery to extend these capabilities to retailers, with industry-leading analytics and predictive capabilities that can help optimize business performance.

345 Global is a cloud-based platform that enables customers to optimize store planning, merchandising, sales, and marketing functions within a single, integrated solution.
Impact Analytics helps retailers and consumer goods businesses make better decisions and improve profitability with a platform that uses predictive analytics and machine learning to optimize various aspects, such as forecasting demand, managing supply chains, and enhancing merchandise planning, pricing, and promotions.
Manhattan empowers retailers to unify point of sale, order management, inventory, fulfillment, and customer service with supply chain execution — optimizing operations, enabling real-time decisions, and driving growth.
o9 Solutions unlocks measurable results by transforming disconnected planning processes, reducing value leakage, and enabling smarter, integrated, and more efficient planning decisions.

aside_block: <ListValue: [StructValue([('title', 'Our 2025 AI trends for retail and consumer goods'), ('body', <wagtail.rich_text.RichText object at 0x7fae207a1be0>), ('btn_text', 'Read them now.'), ('href', 'https://cloud.google.com/resources/ai-trends-retail?utm_source=cgc-blog&utm_medium=blog&utm_campaign=FY24-Q4-global-ENT30703-website-dl-ai-trends-report-retail-cpg-2025&utm_content=-&utm_term=-'), ('image', None)])]>

Enhancing physical store operations

Physical stores and in-person shopping experiences remain vital to retailers. AI is helping these businesses improve how they operate in a variety of ways, whether it’s enhancing how merchandising assistants support customer requests or deploying machine vision to detect and resolve low-inventory challenges.

NCR Voyix enables retailers to deliver a seamless and personalized omnichannel shopping experience while providing real-time, data-driven insights into shopper behavior and store performance, which helps optimize operations and supports long-term growth.
Standard.ai offers solutions that let retailers optimize performance through computer vision with capabilities, such as multi-camera tracking to enable high-resolution understanding of shopper behaviors and store performance.
VusionGroup helps retailers maximize efficiency and improve store performance with solutions that can optimize critical functions, such as intelligent pricing and promotions, real-time shelf monitoring, in-store digital advertising, and more.
Zebra offers new integrated hardware and software solutions that leverage AI and machine learning to help retailers transform workflows through improved inventory, connected frontline workers, and intelligent automation.

Enabling customer success with services partners

Google Cloud relies on its services partners to provide customers with the expertise and support needed to plan, deploy, and optimize AI projects. Many of these partners have launched services specifically for retailers and are continuing to demonstrate their proven ability to help customers transform with AI and other Google Cloud technology at NRF.

Accenture and its ai.RETAIL solution provide customers with the technology needed to transform operations, deploying AI and edge computing to improve consumer experiences, personalize marketing, enhance employee productivity, and more.
Deloitte offers a real-time Associate Productivity solution for intelligent task management and improving in-store operations, a Demand Planning solution to enhance inventory productivity and on-shelf availability, and a Customer Data Enrichment solution for better customer insights and personalized marketing.
Publicis Sapient applies Google Cloud AI for its Content Supply Chain offering, which helps businesses optimize the content lifecycle, and its Retail Media Accelerator, which enables retailers to identify new revenue streams and increase ROI throughout the marketing lifecycle.
Tredence brings unified data models and AI/ML accelerators together with its gen AI-powered Category Performance Advisor, which provides real-time prescriptive recommendations for retail organizations to stay ahead of market trends, improve efficiency, and drive measurable growth.
Slalom provides retail businesses with a multimodal AI discovery solution that uses BigQuery, Vertex AI, and Gemini to help customers solve product discovery challenges and initiate automated workflows for delivery and warranty information.

How to build dynamic web experiences with Conversational Agents

Fri, 10 Jan 2025 17:00:00 +0000

If you have a website, it’s table stakes to build engaging experiences that are effective at retaining existing customers, and attracting new ones. Users want tailored content, but traditional website development tools struggle to keep up with the demand for dynamic, individualized journeys. With Google Gemini and Conversational Agents (Dialogflow CX), you can now build websites that dynamically adapt their content based on what your users are looking for.

In this blog post, you will learn how to:

Create dynamic web pages that respond to user’s intents using Conversational Agents
Use function tools to bridge the gap between conversation intent and web content display

What is a Conversational Agents function tool?

A Conversational Agent function tool is a feature that allows your chatbot to interact with external systems and trigger actions based on user conversations. In this article, we use it to:

Detect user intents from natural language input
Map those intents to specific function tool
Dynamically update the UI based on the conversation flow

aside_block: <ListValue: [StructValue([('title', '$300 in free credit to try Google Cloud developer tools'), ('body', <wagtail.rich_text.RichText object at 0x7fae20315c70>), ('btn_text', 'Start building for free'), ('href', 'http://console.cloud.google.com/freetrial?redirectPath=/welcome'), ('image', None)])]>

Let’s take an example: Retail chatbot

While everyone can benefit from these features, retailers in particular can benefit from building dynamic web pages with Conversational Agents. We'll use a retail chatbot use case to demonstrate this tool. Here’s the workflow:

Step 1: Create a function tool

Set up a new Playbook function tool called Load-Swag-Content with the following input/output schemas in YAML format.

code_block: <ListValue: [StructValue([('code', '# Input format\r\nproperties:\r\n url:\r\n type: string\r\n description: the URL for the Swag\r\nrequired:\r\n - url\r\ntype: object\r\n\r\n# Output format\r\nnull'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7fae180ffdf0>)])]>

Your console should look something like this:

Step 2: Set up a playbook steering agent

Set up a main steering playbook to call the function tool Load-Swag-Content.

Step 3: Create examples to drive Playbook agent behavior.

In this example, when a user asks about “Backpack”, the Playbook agent will call the function tool by passing a backpack related URL as an argument to the web client.

More information on the web client in the next step.

Step 4: Write web client JavaScript function

This client-side Javascript function receives the URL from the Load-Swag-Content function tool and updates the HTML iframe accordingly.

code_block: <ListValue: [StructValue([('code', 'function loadURL(url) {\r\n console.log("URL received: " + url.url);\r\n document.getElementById(\'myIframe\').src = url.url;\r\n }'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7fae180ff760>)])]>

We are using HTML iframe to demonstrate the function calling and parameter passing capabilities. The same concept works across different web frameworks and applications, and developers can be as creative as they want to build custom logic.

Step 5: Register the function tool

Register the Playbook function tool using registerClientSideFunction, which will map the Load-Swag-Content tool with the JavaScript function loadURL.

code_block: <ListValue: [StructValue([('code', 'const toolId = "YOUR_TOOL_ID"\r\n\r\nconst dfMessenger = document.querySelector(\'df-messenger\');\r\n\r\ndfMessenger.registerClientSideFunction(toolId, "Load-Swag-Content", loadURL)'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7fae12f61be0>)])]>

You can get the toolId from the browser URL bar at the Playbook function tool page.

Step 6: Integrate Dialogflow messenger

Finally, embed the Dialogflow messenger integration with your web client.

code_block: <ListValue: [StructValue([('code', '<df-messenger\r\n location="YOUR_REGION"\r\n project-id="YOUR_PROJECT_ID"\r\n agent-id="YOUR_AGENT_ID"\r\n language-code="en"\r\n max-query-length="-1">\r\n <df-messenger-chat-bubble\r\n chat-title="VA driven Web Content"\r\n expanded="true"\r\n chat-width="320"\r\n chat-height="480">\r\n </df-messenger-chat-bubble>\r\n</df-messenger>'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7fae12f613d0>)])]>

Sample front end source code

This is a front end sample code. You need to update configuration such as YOUR_REGION, YOUR_PROJECT_ID, YOUR_AGENT_ID, YOUR_TOOL_ID, and custom JavaScript function.

code_block: <ListValue: [StructValue([('code', '<HTML>\r\n <body>\r\n <iframe id="myIframe" src="" width="100%" height="600px"></iframe>\r\n \r\n <link rel="stylesheet" href="https://www.gstatic.com/dialogflow-console/fast/df-messenger/prod/v1/themes/df-messenger-default.css">\r\n <script src="https://www.gstatic.com/dialogflow-console/fast/df-messenger/prod/v1/df-messenger.js"></script>\r\n <df-messenger\r\n location="<YOUR_REGION>"\r\n project-id="<YOUR_PROJECT_ID>"\r\n agent-id="<YOUR_AGENT_ID>"\r\n language-code="en"\r\n max-query-length="-1">\r\n <df-messenger-chat-bubble\r\n chat-title="VA driven Web Content"\r\n expanded="true"\r\n chat-width="320"\r\n chat-height="480">\r\n </df-messenger-chat-bubble>\r\n </df-messenger>\r\n <style>\r\n df-messenger {\r\n z-index: 999;\r\n position: fixed;\r\n --df-messenger-font-color: #000;\r\n --df-messenger-font-family: Google Sans;\r\n --df-messenger-chat-background: #f3f6fc;\r\n --df-messenger-message-user-background: #d3e3fd;\r\n --df-messenger-message-bot-background: #fff;\r\n bottom: 16px;\r\n right: 16px;\r\n }\r\n </style>\r\n \r\n <script>\r\n function loadURL(url) {\r\n console.log("URL received: " + url.url);\r\n document.getElementById(\'myIframe\').src = url.url;\r\n }\r\n \r\n const toolID = "<YOUR_TOOL_ID>";\r\n \r\n const dfMessenger = document.querySelector(\'df-messenger\');\r\n dfMessenger.registerClientSideFunction(toolID, "Load-Swag-Content", loadURL);\r\n \r\n const default_url = {"url":"https://www.example.com/v1/inventory.html"};\r\n loadURL(default_url);\r\n dfMessenger.sendQuery(\'Hi\');\r\n \r\n </script>\r\n </body>\r\n </HTML>'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7fae12f614f0>)])]>

Demo web page

Let's look at a demo use case for a virtual swag assistant. The customer is greeted at the start of the chat.

When the customer wants to find out more about a Fleece Jacket, the page is dynamically updated to display relevant information.

Next steps

To learn more about Conversational Agent Function tools, check out the following resources and enhance your customer experience with real-time intent-based dynamic web pages.

Get started with Conversational Agent by following the tutorial here
Function Tool Documentation
Conversational Agents best practices

How retailers are accelerating AI into production with NVIDIA and Google Cloud

Thu, 09 Jan 2025 13:00:00 +0000

Retailers have always moved quickly to connect and match the latest merchandise with customers' needs. And the same way they carefully design every inch of their stores, the time and thought that goes into their IT infrastructure is now just as important in the era of omnichannel shopping.

As retail organizations increasingly adopt AI foundation models and other AI technologies to improve the shopping journey, robust infrastructure becomes paramount. Retailers need to be able to develop AI applications and services quickly, reliably, robustly, and affordably, and with support from Google Cloud and NVIDIA, leading companies are already accelerating their time to market and achieving scalable costs as they move AI from pilots into production.

Google Cloud has worked with NVIDIA to empower retailers to boost their customer engagements in exciting new ways, deliver more hyper-personalized recommendations, and build their own AI applications and agents; we’ve also integrated prebuilt generative AI agents for customer service to drive immediate savings. With the NVIDIA AI Enterprise software platform available on the Google Cloud Marketplace, retailers can streamline AI development and deployment through scalable NVIDIA infrastructure running on Google Cloud.

aside_block: <ListValue: [StructValue([('title', 'Try Google Cloud for free'), ('body', <wagtail.rich_text.RichText object at 0x7fae207e45b0>), ('btn_text', 'Get started for free'), ('href', 'https://console.cloud.google.com/freetrial?redirectPath=/welcome'), ('image', None)])]>

Now, retailers can also leverage NVIDIA NIM microservices, part of NVIDIA AI Enterprise and available on Google Kubernetes Engine (GKE) to deploy generative AI models at scale, optimize inference, and handle large volumes of inquiries at reduced costs.

Retail customers and partners are combining Google Cloud with NVIDIA AI Enterprise to unlock AI transformation at scale.

Reduce costs and enhance customer satisfaction: LiveX AI stands at the cutting edge of generative AI technology, building custom, multimodal AI agents that can deliver truly human-like customer experiences. Google Cloud and LiveX AI collaborated to help jumpstart LiveX AI’s development, using GKE and NVIDIA AI Enterprise. In a matter of three weeks, LiveX AI and Google Cloud worked together to deliver a custom solution for its client, resulting in a reduction in customer support costs by up to 85%.

“NVIDIA’s software on Google Cloud brings two of the best technology leaders together. NVIDIA’s easy-to-use NIM microservices, available on Google Cloud, are secure and reliable, and help deploy high-performance AI model inference more quickly and affordably. NVIDIA NIM microservices and GPUs on GKE accelerated LiveX AI Agent’s average answer/response generation speed by 6.1x, enabling real-time, human-like interactions for customer support, shopping assistance, and product education, boosting growth, retention and customer experience.” – Jia Li, Co-Founder, Chief AI Officer, LiveX AI

Improve responsiveness: AI practices like text embeddings and vector databases help retailers make more relevant recommendations by using more data, but this can also slow the experience down. The in-house engineering and data science organization at a top-5 U.S. grocer collaborated with Google and NVIDIA to optimize models for better performance.

By using NVIDIA AI Enterprise software’s performance and caching improvements in its Vertex AI endpoint, the grocer cut inference time from several seconds to just 100 milliseconds — without changing the model. This now makes large-scale, real-time personalization possible. Learn more about the benefits of combining Google Cloud's Vertex AI platform and NVIDIA AI Enteprise software.
In-store analytics and innovation: AI is advancing how brick and mortar stores understand customer engagement, creating new opportunities to personalize the shopper journey. Standard.ai is accelerated by NVIDIA Metropolis, also available with NVIDIA AI Enterprise on the Google Cloud Marketplace, giving retailers and consumer goods precise visualization of customer journeys and creating actionable insights analyzing factors in real time, such as dwell time, shopper orientation, proximity, and engagement with products, ads, and high-impact zones.

“The NVIDIA Metropolis platform and DeepStream software development kit have enabled us to seamlessly deploy our video pipelines across Google Cloud data centers and on-prem GPUs, and, in combination with model optimizations through the NVIDIA TensorRT ecosystem of application programming interfaces, we have cut our image preprocessing time to one-third, significantly reducing our infrastructure footprint." – David Woollard, Chief Technology Officer, Standard.ai

aside_block: <ListValue: [StructValue([('title', 'Our 2025 AI trends for retail and consumer goods'), ('body', <wagtail.rich_text.RichText object at 0x7fae207e48e0>), ('btn_text', 'Read it now.'), ('href', 'https://cloud.google.com/resources/ai-trends-retail?utm_source=cgc-blog&utm_medium=blog&utm_campaign=FY24-Q4-global-ENT30703-website-dl-ai-trends-report-retail-cpg-2025&utm_content=-&utm_term=-'), ('image', <GAEImage: Retail ROI hero>)])]>

Accelerate AI transformation

Influenced by the rapid advancements of AI, the retail landscape is evolving faster than ever. For retailers looking to stay on the cutting edge, the collaboration between Google Cloud and NVIDIA continues to offer access to the latest in AI models, infrastructure, platforms that ensure scalability, and development tools all in an environment that’s built on responsible AI practices and best-in-class security.

Get started now with NVIDIA AI Enterprise on Google Cloud to maximize your AI investments and scale across your enterprise.

Accelerate retail media success with EPAM and Google Cloud

Thu, 31 Oct 2024 16:00:00 +0000

Retail media networks, a type of advertising platform that allows retailers to sell ad space on their digital channels to third-party brands, are nothing new. But they are about to change dramatically in the coming year. Consumers are increasingly privacy-conscious and desire more personalized ad recommendations.

EPAM and Google Cloud have been working on retail media solutions for a long time, providing you with data and insights to give you more comprehensive and refined views of your audiences, better measurements, and improved buyer experiences. Together, we have been active participants in the development of this promising “third wave” of digital media, and our experience tells us that the coming year is going to be significant.

Companies that leverage first-party data most efficiently and use AI and gen AI will see a ROI in retail-media. That’s why we are excited to bring EPAM’s Retail Media Orchestration Toolkit to market now, helping retailers of most sizes capitalize on the promise of the coming year, no matter how mature their retail media operations. With the Retail Media Orchestration Toolkit, you gain customized, in-house retail media operations supported by Google and Google Cloud’s industry-leading digital advertising capabilities, and orchestrated with EPAM’s deep retail expertise.

With Google Cloud’s AI and gen AI tools and expertise, you can supercharge your first-party data with a level of insight that was previously out of reach — and gain a significant advantage over your competitors.

aside_block: <ListValue: [StructValue([('title', 'Try Google Cloud for free'), ('body', <wagtail.rich_text.RichText object at 0x7fae18118c70>), ('btn_text', 'Get started for free'), ('href', 'https://console.cloud.google.com/freetrial?redirectPath=/welcome'), ('image', None)])]>

Retail media profits remain elusive

While many retailers are aware of the revenue potential of first-party data and have implemented retail media operations, they still struggle to mature these initiatives. Common roadblocks to maximizing retail media profits include:

Infrastructure unsuited to fulfill the increasing demand for data-driven insights: There are hundreds of retail media networks, so advertisers have an abundance of choice in selecting a network for hosting a campaign. Brands are eager to invest advertising spend in networks that provide detail-rich, data-driven insights, however, many retailers struggle to provide the depth of insights that advertisers seek to justify their retail media advertising spend. Retailers are first and foremost sellers of products — few are retail media experts. And the scale involved for many retailers magnifies the difficulty.

Inability to provide fast, accurate measurements: Providing closed-loop measurement of campaign performance — and particularly omnichannel measurement across disparate physical and digital customer interactions — requires a level of retail media technology, expertise and orchestration that few retailers possess.

Lack of technology and resources to provide data clean rooms: Customer data drives retail media. The more comprehensive and in-depth the data, the greater the advertising success. Given that detailed customer data is also often sensitive information, it’s important to protect that data to adhere to industry ethics, maintain customer goodwill and, quite often, adhere to regulatory requirements. Data clean rooms provide a safe environment to use and share customer data among multiple legitimate participants. But the technological challenges involved in maintaining a data clean room are significant, and many retailers have neither the resources nor the expertise to do so.

Difficulties in standardizing workflows and data: Most retail media networks are made up of a patchwork of multiple independent software vendors (ISVs). They utilize their own processes, procedures and reporting formats. The result is a never-ending stream of incoming reporting data that must be translated to match in-house formats before it can be relayed to advertisers. Many retailers attempt to handle this by manually managing incoming data, resulting in increased staff, degraded performance when reporting to customers, and deflated retail media profits.

The Retail Media Orchestration Toolkit

Today, retailers can implement in-house, custom solutions for retail media, just as mega-retailers like Albertsons, Kroger, Tesco and Walmart have done.

The Retail Media Orchestration Toolkit was developed under Google Cloud’s Industry Value Network (IVN) initiative in partnership with EPAM and Google Cloud, and leveraging ISV solutions such as Moloco. The Toolkit lets retailers leverage their data to support their retail media operations and serve their advertising clients.

EPAM has in-depth knowledge of retail media operations, earned through years of experience in developing custom, Google-Cloud-powered, in-house solutions for some of the world’s largest retailers. Google Cloud makes it possible to design and implement custom retail media solutions, offering a holistic, end-to-end platform and solutions for audience capabilities, measurement, media execution and innovation. Based on an innovative cloud-based data foundation known as Google Cloud Cortex Framework, EPAM's Toolkit helps customers better utilize their data, wherever it is stored, including first-party data from tools like Google Ad Manager, Google Search Ads 360, Display & Video 360 and others — making a true in-house, custom retail media solution a realistic option. As part of Google Cloud’s Industry Value Network, the solution also leverages ISV solutions to provide a comprehensive and repeatable solution through the pre-built connectors and accelerators.

Retailers that deploy the Retail Media Orchestration Toolkit gain:

A fully owned, customized, in-house retail media solution that can easily be scaled as needed
The ability to simplify, standardize and automate retail media operations
Fully automated processes that replace costly, error-prone manual processes
Omnichannel measurement capabilities so they can provide proof of campaign performance and ROI delivered for advertisers
Better informed, data-fueled decision making for maximizing campaign performance across different and disparate platforms
Guidance in developing a solution that serves their current needs while leveraging the potential of emerging technologies such as AI/machine learning (ML)

Retailers implementing the EPAM and Google Cloud retail media solution are already experiencing significant, quantifiable benefits, including:

15-20% increase in retail media revenue and advertiser demand
2x increase in campaign performance
40% time savings
12% reduction in retail media operations costs

The four phases of retail media success

Designing and implementing an in-house solution with the Retail Media Orchestration Toolkit consists of four phases:

Multichannel measurement: Merge, standardize, automate, analyze and visualize multichannel campaign delivery data and transactional data.
Advanced omnichannel measurement: Track, report and analyze data from user-level interactions across channels and platforms.
Audiences: Create high-value predictive audiences based on your transactional data using custom segmentation models deployed within your own cloud environment and syndicated to your retail media partners.
Insights: Use analytics to generate and enhance new revenue streams based on consumer and brand insights.

This phased design was developed to benefit retailers, regardless of their level of retail media maturity. Your solution can be installed and configured to meet your specific needs to deliver almost immediate benefits — and, when you’re ready, you can leverage the insight-boosting value of Google Cloud’s AI and ML capabilities.

Get ready to grow your retail media operations

Retail media is young enough that few, if any, organizations have achieved true maturity in their retail media operations. Even mega-retailers that have developed their own in-house retail media solutions have some tweaking to do: streamlining workflows, developing full and complete automation of retail media operations, and fully leveraging emerging capabilities with AI and ML are just some areas that typically lack technological maturity.

But where does your organization stand? How far are you from fully maximizing the potential of retail media within your organization? And what steps must you take to reach that point?

A maturity assessment provides answers to those questions. Just two to three sessions with key stakeholders in your organization will provide us with a high-level understanding of your retail media operation. And, based on that knowledge, we’ll develop a tailor-made action plan for you, including:

A multi-year roadmap, personalized to your business and maturity level, that covers technologies, processes and partnerships
A retail media profit and loss forecast that highlights key dependencies
A resource and investment framework that supports your accelerated growth

Quite simply, your maturity assessment provides a roadmap for progressing from where you are to where you want to be, illuminating a pathway to maximizing the potential of retail media for your organization.

Learn more: Discover how EPAM and Google Cloud’s solution empowers retailers to maximize revenue and gain a competitive edge.

The modern marketer’s strategic advantage: AI-powered data clean rooms

Mon, 26 Aug 2024 16:00:00 +0000

Businesses across all industries crave data to better understand their customers and drive sales. Imagine a major consumer packaged goods brand that primarily sells through a large retailer. This brand could gain valuable insights by understanding the key actions / high-value assets (HVAs) customers take on the retailer's website before making a purchase. Although this makes good business sense, retailers are hesitant to share sensitive customer data, making collaboration difficult.

Boards, CEO’s and CFO’s are turning to CMO’s (and marketing orgs) to get an answer. This brings us to the key points, on what modern marketers are truly looking for from data:

Get detailed level insights: Safely analyze data from different sources without compromising privacy
Use those insights for smarter decisions: Use powerful AI tools to uncover hidden patterns and opportunities
Drive business success: Fuel growth with targeted marketing and personalized customer experiences

The single thread connecting the above three areas is the data clean room that offers a secure, privacy-compliant solution, empowering modern marketers to unlock valuable insights from collaborative data analysis across various industries, driving strategic decision-making and business growth.

Google BigQuery data clean Room: the secure solution

BigQuery data clean rooms, introduced in 2023, offers a secure environment for sharing, collaborating, and analyzing sensitive data, all while leveraging the benefits of the BigQuery ecosystem.

How it works and its architecture

BigQuery data clean rooms is a specialized application of Analytics Hub, a platform within BigQuery for secure data sharing and exchange. Analytics Hub enables organizations to build a data ecosystem where datasets are shared in-place, granting providers control and visibility into data usage.

Leveraging Analytics Hub and BigQuery's serverless architecture, BigQuery data clean rooms establish a secure environment for multi-party collaboration. Data remains in its original location, allowing participants to run queries and share aggregated results, ensuring data privacy.

Behind the scenes: the architecture

At its core, BigQuery serves as the data platform where data contributors and subscribers store their datasets. Google Cloud BigQuery is a fully managed, serverless data warehouse that enables scalable and cost-effective analysis of massive datasets. It stands out for its decoupled architecture, separating compute and storage, allowing independent scaling for optimal performance and cost efficiency.

It leverages the concept of shared datasets from Analytics Hub, allowing the clean room owner to contribute their dataset along with specific egress and analysis rules. These rules dictate what kind of outputs are permissible from the clean room, ensuring data privacy. You can refer to the Google Cloud Documentation for a detailed understanding of the architecture.

Industry use cases

Data clean rooms are transforming businesses across industries. Let's look at a few use cases.

Use case 1: measuring new customer acquisition from digital advertising

A company runs a digital advertising campaign across various platforms to attract new customers or re-engage lapsed ones. Once the campaign concludes, the ad platform data (impressions, clicks, etc.) is brought into a data clean room.

Within this secure environment, the company can combine advertising campaign data with their own internal customer data. This allows them to match ad interactions (like clicks) with actual customer conversions. The clean room ensures that sensitive customer information remains private and is only used for aggregated analysis. The company can then see key metrics, like how many new customers were acquired through the campaign, the cost per acquisition, and the overall return on ad spend. These insights help them evaluate the campaign's success and make informed decisions for future advertising strategies.

Use case 2: retailer-CPG collaboration

When retail media networks collaborate with their consumer packaged goods (CPG) brand partners, BigQuery data clean rooms can unlock new and valuable insights. Through this collaboration, a CPG company can evaluate the effectiveness of its advertising campaigns conducted on the retailer's platform, specifically for audiences that overlap between the two entities. The CPG company gains insights into the impact of its campaigns on the retailer's platform, allowing for more informed decision-making and optimization of marketing strategies.

CPG data: CPG offers data on their existing audience data (1p).
Retailer data: Retailer possesses data indicating which customers made purchases.
Data clean room: A secure and privacy-preserving environment known as the data clean room enables CPG and Retailer to match hashed customer IDs. This allows them to determine whether the targeted customers went on to purchase the advertised products.

CPG can evaluate the effectiveness of their ads and enhance their campaigns. Simultaneously, Retailer can demonstrate the worth of their advertising platform to CPG partners.

Use case 3: retailer-publisher collaboration

A retailer could collaborate with a publisher, like a streaming service. The retailer brings its loyalty data and mobile data, while the streaming service contributes its engagement data. The data clean room acts as a secure, neutral environment where these datasets can be combined and analyzed without either party directly accessing the other's raw data.

The retailer can understand the viewing habits of its loyalty program members and identify potential new customers. Meanwhile, the streaming service can gain insights into subscribers' shopping behaviors and personalize content recommendations. Both can benefit from combined data analysis, gaining competitive intelligence and identifying market trends and customer behavior across platforms.

Use case 4: retailer-manufacturer collaboration

A retailer can collaborate with a manufacturer within a data clean room by sharing its sales and inventory data, while the manufacturer shares its product data.

The combined data allows them to uncover insights and generate actionable recommendations. This can lead to optimized product assortments, strategic pricing, and targeted marketing campaigns.

Beyond marketing: internal secure collaboration

It's worth noting that data clean rooms can be used for various internal collaboration use cases, enabling organizations to leverage sensitive data across internal teams while upholding strict privacy standards. By anonymizing or pseudonymizing information, teams can collaborate effectively without compromising individual privacy.

Use cases

HR analytics: HR departments can partner with data science teams to analyze employee data, identify trends in performance and turnover, and develop predictive models for talent retention. Data clean rooms ensure sensitive employee information remains protected throughout the analysis process.
Employee engagement: Internal communications teams can analyze employee sentiment through surveys and social media data while preserving anonymity. This empowers organizations to understand employee perspectives and identify areas for improvement without compromising individual privacy.

Data clean rooms facilitate secure internal collaboration across various departments, enabling data-driven decision-making while safeguarding sensitive information. This fosters a culture of trust and compliance, empowering organizations to unlock the full potential of their data without compromising privacy.

So what are the actionable strategies for modern marketers?

Data clean rooms empower businesses to:

Unlock insights: Extract actionable intelligence from data while maintaining privacy and security
Fuel innovation: Enable data-driven decisions that enhance customer experiences and drive growth
Foster collaboration: Break down silos and enable secure data sharing

For modern marketers, AI-powered data clean rooms are a strategic advantage. By identifying use cases, establishing data-sharing agreements, leveraging AI tools, and monitoring results, they can harness the power of data to drive their businesses forward. Read more details about how BigQuery data clean rooms work and explore the architecture. Your data team can get started today with a free trial of BigQuery.

Shiseido: building a data analysis platform using BigQuery for 80% cost savings

Thu, 25 Jul 2024 16:00:00 +0000

The cosmetics industry is one of the industries that has been strongly affected by the COVID-19 pandemic, as people’s habits changed — both in how the shop and how they want to show up, whether that's at work, to a party, or just out and about. Even as our lives have found a new normal of get-togethers and hybrid work, the market has continually evolved since 2020.

Shiseido, one of Asia and the world's largest beauty brands has responded by shifting our business model to focus on each consumer, in order to accumulate profits over the medium- to long-term, rather than increasing sales in the short-term.

To do that, we need to understand each consumer like never before. And to do that, we had to understand our data at a level we never had before.

I joined Shiseido in early 2022, as Data Analytics Group Manager in the Customer Strategy & Planning (CS&P) department, and this responsibility fell under my purview. I oversee a team that builds strategies, creates business plans, and deploys them to distribution and sales sites.

As a starting point, we decided it was imperative to improve our data utilization process.

Shifting from an inefficient data analysis environment to BigQuery

Previously, Shiseido did not have expertise to manage data in-house. Therefore, we had to outsource data processing and analysis. We were also outsourcing the data processing services on a project basis, which led to segmenting workloads; that segmentation caused is to frequently duplicate our data processing work.

This also burdened our development team, with new requests constantly being made. We were in a situation where we had 20 to 30 inefficient database servers running in parallel. The costs and time required for analysis-processing started to grow beyond what was realistic.

aside_block: <ListValue: [StructValue([('title', 'Next is now'), ('body', <wagtail.rich_text.RichText object at 0x7fae1df839a0>), ('btn_text', 'Register'), ('href', 'https://cloudonair.withgoogle.com/events/next-tokyo-24'), ('image', <GAEImage: Next24_Blog_blank_2-02>)])]>

In order to improve this situation, we decided to build a data analysis platform that could be used organization-wide. We set three key goals for our new data analysis platform, which were faster processing, reduced costs, and a structure to streamline operations and customization. After conducting proof of concept (POC) on all major cloud platforms, we ultimately adopted Google Cloud.

In particular, we found that BigQuery offered attractive processing performance that came with cost reductions. From the perspective of data utilization, the many features related to AI were also attractive.

System configuration with ease of operation and future expandability in mind

This diagram shows the system configuration of our new data analysis platform built on Google Cloud.

Yoko Nishio, Platform Development Manager (Data Engineering) at BrainPad Inc., who was in charge of development, explained the points unique to Shiseido:

“One key feature is that we deliberately separated the application layer and the infrastructure layer, and adopted a configuration in which both layers can be accessed through a shared virtual private cloud.”

In the future, our line-of-business units will use the application layer, while the infrastructure layer will be used for the information systems of the entire Shiseido Group. The latter will be independently operated and maintained by Shiseido Interactive Beauty, Shiseido’s group company that leads its digital transformation. To do this, Shiseido Group must meet high-security regulations when building the system.

Splitting the system infrastructure into two parts allows for more versatility and heightened security. If an employee uses Google Cloud in a project, they can simply connect to the infrastructure layer via the virtual private cloud in order to share the project internally, and in a secure manner. As for the application layer, we proactively selected managed services that are easier to handle, as they will be managed by line-of-business teams.

During the development, the first priority was to launch the system on time. To achieve this, the existing data processing method was retained and and migrated. Nishio says that this was the most difficult part of the development.

Seamless data migration enabled by SQL and DataFlow

The biggest challenge to executing on time was migrating the existing processes for the data processing area.

The stored procedures that were running on the existing SQL Server were large-scale, dynamic SQL modules with about 3,000 rows. In other words, they were quite complex. Rather than rebuild them, we prioritized the schedule and decided to migrate approximately 80 of the existing 130 processes in the processing area to BigQuery.We first used Google Cloud’s migration tool in order to convert the existing processes in the stored procedures, but due to the complexity of the existing structure, we were not able to automatically convert the processes and had to handle it manually. Also, the existing environment had a loose collation, where it did not have the ability to distinguish between the half-sized and full-sized letters used on Japanese keyboards. Because of that, it was necessary to take measures on the source data side to obtain equivalent results in BigQuery, where data is strictly differentiated.Data migration in the existing environment was just as difficult. Because the data was siloed, it was harder to organize and select tables when consolidating to BigQuery. On the positive side, we migrated over 100TB of data easily thanks to SQL and Dataflow. In particular, SQL Server and Cloud SQL were highly compatible, which allowed us to carry out the migration with zero import errors or value discrepancies.

Enhancing functionality with multi-cloud support and AI/ML

Following the successfully delivered POC in July 2022, requirements were defined in November, and development began in December. By the end of June 2023, we had begun full-scale operations of our data analysis platform.

The results exceeded our expectations. By consolidating our previously disparate servers into Google Cloud, we reduced the weight of storage and improved processing efficiency, resulting in an 80% cost reduction and a 90% reduction in processing time. We’re steadily expanding our in-house operations, which has allowed the team to move away from our dependence on outsourced companies. This has had the benefit of adding more clarifications to the workload.

Having optimized the data processing aspect of our existing environment, we’re now working toward a second phase. This is designed to further refine processing performance, accelerate the provision cycle by adding external database linkage and automating validation checks, and democratize data analysis by integrating with Looker, Google Cloud's business analytics platform.

We have also begun working to improve analysis through various AI/ML measures using Vertex AI.

As our third phase, we plan to go multi-cloud by integrating our data from Google Cloud and other storage clouds owned by the Shiseido IT department. We have high hopes for BigQuery Omni, which allows us to cross-utilize data held in other clouds. We are also paying attention to topic-generation AI. For example, we have begun creating a system for coupon distribution content that automatically generates text and images based on customer classification.

A year-long development process, in review

Our interactions with the support team over the past year of development have left a lasting impression on us. In developing our platform, the Google Cloud team provided generous support for analysis that went beyond infrastructure. We believe that by leveraging Google Cloud, we will be able to comprehensively upgrade our data analytics.I would also like to take on the challenge of changing the image of Shiseido. In a not-so-distant future, we’ll have a greater presence in our customers’ lives, and not just be seen as a company that sells cosmetics products in the storefront. For instance, wouldn’t it be interesting if we can provide a service in which a customer wakes up in the morning to wash their face, and through the image taken by the mirror of that customer, a communication would start between that person and Shiseido, where Shiseido would provide the optimal lotion through the dedicated device.It may sound like a distant goal, but I think it’s crucial for us to start thinking in a broader perspective in order for Shiseido to continue to grow in the long term. We understand the need to deliver more value to our customers as soon as possible, and we are committed to working towards that.

Create ecommerce experiences with commercetools and Google Cloud Application Integration

Fri, 24 May 2024 16:00:00 +0000

The world of digital commerce is rapidly evolving. To stay competitive, businesses need the ability to integrate their commerce systems with a wide array of applications and services. Furthermore, customers expect targeted and personalized experiences. Google’s AI and data solutions create an opportunity to deliver a personalized experience for customers within your ecommerce solutions. This can be done by implementing AI powered search capabilities, product recommendation, AI chatbots, content creation with AI and more. As organizations start to implement these capabilities, being able to connect Google Cloud services to commercetools, a cloud-native, headless commerce platform, offers significant value.

Google Cloud’s Application Integration is an Integration-Platform-as-a-Service (iPaaS) solution, offering tools to streamline data connectivity and management across various applications and services. We recently launched the commercetools connector (currently in public preview) to empower users to accelerate the integration between their ecommerce platform and Google Cloud solutions as well as other platforms available in the library of 80+ connectors. By leveraging the commercetools connector to integrate commercetools data with Google Cloud services such as BigQuery and Vertex AI, users can unlock a wide range of analytics and machine learning capabilities within their ecommerce. For example, users can use commercetools data as a model's prompt in order to generate personalized insights or leverage the commercetools data fed into BigQuery to create and execute machine learning models with BigQuery ML.

In this blog we will explore different functionalities of the commercetools connector and some of the use cases where you can leverage it to improve your customer experience.

Connect your commercetools platform with Google Cloud

commercetools provides a composable commerce platform, giving 450+ companies the components required to run outstanding shopping experiences across digital and physical touchpoints. commercetools equips some of the world's largest businesses with tools to future-proof digital offerings, reduce risks and costs, and drive revenue growth.

The commercetools connector can unlock a new level of integration, bringing numerous benefits:

Simplified integrations: Establish connections between your commercetools platform and the rich ecosystem of connectors, tasks and services available on Google’s Application Integration. The pre-built connectors make it easier to integrate with Google first party services such as BigQuery, Vertex AI, Pub/Sub, Cloud Spanner etc. as well as third-party systems for which we have connectors. You can let Application Integration handle the integrations for you while using data mapping tasks to easily address data transformation needs.This translates to faster time-to-market and reduced development costs.
Enhanced data flow: Enable powerful data analysis and machine learning for your ecommerce using Google Cloud services like BigQuery and Vertex AI with the pre-built connectors and the vertex AI prediction task. Use the integration tasks to orchestrate and transform the data flows between these applications. You can use the integration flows you design to drive business insights, intelligent product recommendations, personalized promotions, and more.
Accelerated innovation: Reduce the technical overhead of building and maintaining custom integrations for your ecommerce platform. Using application Integration to build your integrations between commercetools and Google Cloud with the drag-and-drop interface frees up developer resources and lets you focus on creating innovative customer experiences.

The commercetools connector facilitates a number of use cases, such as:

1. commercetools data to BigQuery:
The commercetools connector helps organizations transfer real-time data from commercetools to BigQuery in order to leverage their customer data and unlock insights to improve customer experience and increase conversion rate.

The following example shows how you can push product data from commercetools to BigQuery. The integration is triggered by an API call, although we could use any of the available triggers, including a scheduler, a pub/sub message and more. Then, Application Integration fetches product data from commercetools and proceeds to an iterative processing loop. This loop will call a sub integration for every product that will collect and store all the products in BigQuery.

With the commercetools data in BigQuery you can make informed and data-driven decisions combining data from your ecommerce and other diverse data sources.

2. On-demand AI-driven Insights:
Power your ecommerce by generating on demand predictions with Vertex AI. You can use events to trigger a prediction and get a real-time answer from your Vertex AI model, whether this is generating a reply to a text prompt, generating a product recommendation or predicting the demand of a product.

In the example below we use a pub/sub message to trigger a Vertex AI prediction using the Vertex AI task in Application Integration. We extract the commercetools product data and use it as the prompt for a Vertex AI model to generate product descriptions for the ecommerce products. We could take this a step further by updating the product in commercetools with the AI generated product description.

An integration flow of this kind helps you to connect your ecommerce platform data to extract AI insights directly from Vertex AI.

3. Ecommerce personalization:
Users are now connected to many devices and reach several digital interaction points before making a decision. For example, buyers tend to search both on websites and search engines before buying. This means that delivering personalized experiences across different channels has become increasingly important. With commercetools we can achieve an omnichannel ecommerce strategy by keeping data consistent across channels. The commercetools connector helps you to take this strategy one step further by integrating commercetools with customer data platforms (CDPs) and recommendation engines on Google Cloud and creating a centralized hub of customer data to fuel consistent personalization across different channels.

By combining the power of these two leading platforms, businesses gain the tools to create exceptional customer experiences that drive growth and success. Get started now: Find commercetools available in the Google Cloud Marketplace, visit the commercertools and Google Partnership page, and read more about Application Integration and the commercetools connector.

Streamline digital commerce with the Integrated Commerce Network from Kin + Carta

Wed, 07 Feb 2024 17:00:00 +0000

At Google Cloud, we’re always looking for ways to help our retail, consumer packaged goods (CPG), and B2B and B2C customers along their digital commerce transformation, no matter the starting point. Whether you’re an enterprise manufacturer who needs to modernize and deliver a consumer-like commerce experience to your B2B customers, or a leading CPG company that is delivering new commercial models, the ideal commerce solution is one that can incorporate multiple best-in-class solutions into a composable but tightly integrated whole. There’s a need for enterprises to combine their data with powerful design, and create the moments that matter for their clients. The result is greater customer loyalty, with a personalized experience, and ultimately, increased profitability. Finding the right independent software vendors (ISVs) to meet your needs and connecting them to create a holistic solution can be challenging.

That’s why we’re pleased to announce the Integrated Commerce Network, a pre-integrated digital commerce solution from a curated group of our digital commerce ISV partners and delivered by systems integrator partner Kin + Carta. The Integrated Commerce Network is part of Google Cloud’s industry value network (IVN) strategy, and makes building an end-to-end digital commerce solution to support ongoing growth and innovation much easier.

What is the Integrated Commerce Network?

The Integrated Commerce Network brings together three essential pillars of modern commerce: a commerce personalization platform (Bloomreach), a truly cloud-native composable platform (commercetools), and an advanced digital experience and analytics platform (Quantum Metric). Kin + Carta, a digital transformation consultancy, brings deep expertise working with these solution providers and Google Cloud. Kin + Carta will develop a technical accelerator that enables faster deployment of these solutions on Google Cloud and facilitates data analysis in BigQuery. Collectively, the Integrated Commerce Network aims to deliver a modern commerce architecture that allows businesses to focus on innovation and differentiation for their customers.

The Integrated Commerce Network is part of Google Cloud’s IVN initiative and is the first such solution developed for digital commerce. IVN solutions combine expertise and offerings from systems integrators (SIs), ISVs, and content partners to create comprehensive, differentiated, repeatable, and high-value solutions that accelerate time-to-value and reduce risk for customers. By pre-integrating partner solutions on Google Cloud, the Integrated Commerce Network, an IVN solution for digital commerce, minimizes the need for clients to build bespoke solutions for common challenges. Each component is curated by layering focused solutions provided by our best-in-class, built-on Google Cloud SaaS partners, tailored to an enterprise’s specific challenges — whether they be retailers, CPG brands, manufacturers, healthcare or more.

The new Integrated Commerce Network solution from Kin + Carta and built on Google Cloud packages powerful capabilities from our partners including:

Bloomreach - an AI-powered commerce personalization platform that seamlessly integrates with Google Cloud BigQuery. Powered by a customer data engine, Bloomreach drives commerce growth with multichannel marketing automation and intelligent product discovery solutions. It unifies real-time customer and product data so businesses understand what customers really want.
commercetools - a cloud-native, component-based, tech-agnostic composable platform that gives you all the components you need to build and run outstanding shopping experiences across all digital and physical touchpoints. It delivers improved scalability, high availability and uptime, and reduced total cost of ownership (TCO) compared to on-premises solutions.
Quantum Metric - a digital experience analytics platform that offers in-depth customer understanding, quantified and tied to core business objectives. It does so by providing real-time analytics, allowing you to monitor, diagnose, and optimize critical digital journeys. With Quantum Metric, you can easily discover when, where, and why customers struggle along their journey, reducing guesswork and time to resolve your customers’ digital friction so you can create compelling and easy-to-use websites and apps.
Kin + Carta - a global digital transformation consultancy and a Premier Partner with Google Cloud. With 20+ years of experience in commerce delivery, Kin + Carta's experts are able to drive beyond customer expectations to build omnichannel experiences that drive revenue and convert customers from anonymous to advocates.

How it works

Independently, Bloomreach, commercetools, and Quantum Metric provide extensive API connectivity to each other as well as numerous third-party solutions. Each partner enhances the others by leveraging shared data to add value.

Bloomreach can use commercetools data to curate personalized purchasing experiences, customizing ads, emails, and mobile notifications. Additionally, it transforms behavioral analytics from Quantum Metric into valuable insights, fostering customer loyalty, satisfaction, and conversion.

commercetools can augment its robust commerce engine with AI-powered site search, intelligent product and content recommendations, and 360-degree content delivery from Bloomreach. You can also optimize the conversion funnel in commercetools by identifying areas where potential customers are abandoning their digital commerce journey with data from Quantum Metric.

The Quantum Metric platform can be used to identify what parts of a customer’s digital experience need improving, quantify the impact of that experience, and inform decisions around redesign, replatforming and more.

Finally, these critical components for modern digital commerce are integrated and deployed by Kin + Carta. Kin + Carta is building a technical accelerator to speed deployment of Bloomreach, commercetools, and Quantum Metric solutions as part of the Integrated Commerce Network and consolidate data analysis via BigQuery.

Built on Google Cloud

Because these tools are built and run on Google Cloud, customers can take advantage of all the benefits that Google Cloud brings to the table, including speed, scale, and security; advanced AI and ML tools; and commitment to open-source. In addition, enterprises can take advantage of our Industry Solutions team of experts and their leadership experience at top retailers, CPG brands, and manufacturers across the globe.

With BigQuery, businesses are empowered to leverage their customer data and unlock insights to improve customer experiences and boost revenue. When data from commercetools, Bloomreach and Quantum Metric is combined in BigQuery, businesses can uncover new insights into:

Customers’ propensity to purchase
Pricing optimizations
Customer churn avoidance
Marketing campaign optimization to understand customer segmentations, retargeting, and promotions

BigQuery also offers built-in machine learning which enables data analysts to operationalize ML models at scale. Further supercharge your data, allowing your commerce organizations to better uncover, summarize and make predictions into customer purchase behaviors.

The Integrated Commerce Network brings the best of Google Cloud’s partner-forward approach to an easy-to-adopt packaged solution for digital commerce transformation. Thanks to our partners, commerce organizations from industry incumbents to scrappy newcomers can take advantage of a tightly integrated, end-to-end solution that improves the customer experience and commercial outcomes while establishing a solid foundation for future growth and innovation.

Learn more about the Integrated Commerce Network delivered by Kin + Carta and built on Google Cloud.

^{Special thanks to the Google Cloud ISV Partner team, including Frank Napoletano, ISV Sales Specialist, George Keller, Partner Engineer, ISV, Tina Feng Liu, Global ISV Partner Marketing, and Liz Seidner Davidoff, Global ISV Partner Marketing, for contributing to this post.}

Looker Studio brings powerful explorations, fresher data and faster filtering

Thu, 16 Nov 2023 17:00:00 +0000

Looker Studio supports self-serve analytics for ad hoc data, and together with Looker, contributes to the more than 10 million users who access the Looker family of products each month. Today, we are introducing new ways for analysts to provide business users with options to explore data and self-serve business decisions, expanding ways all our users can analyze and explore data — leading to faster and more informed decisions.

Introducing personal report links

Business users often leverage shared dashboards from data analysts, which contain key company metrics and KPIs, as a starting point and want to explore beyond the curated analysis to arrive at more specific insights for their own data needs. The introduction of personal reports in Looker Studio enables this activity, delivering a private sandbox for exploration so users can self-serve their own questions and find insights faster – without modifying the original curated report.

Whether you share a report link in group chats or direct messages, an individual copy is created for each user that opens it so that everyone gets their own personal report.

Personal Looker Studio reports are designed to be ephemeral, meaning you don’t need to worry about creating unwanted content, but if you land on valuable insights that you want to keep, you can save and share these reports with new links, separate from the original report you built from.

You can learn more about how personal reports work and how to use them in our Help Center.

Looker Studio Personal Link Visual

Automated report updates

Your analysis and insights are only as good as the freshness of your reports. Looker Studio users can now enable their reports to auto-refresh data at a predefined cadence, so critical business decisions are based on current and updated information.

To learn more about how auto-refresh works, including details on how it works with cache, presentation mode, and existing data freshness settings, visit our Help Center.

Looker Studio Auto refresh feature Visual

Faster filtering in reports

Quick filters enable powerful exploration to slice data and uncover hidden patterns and insights within the context of your report. Quick filters don’t affect other users’ views, so whether you are exploring in a shared or personal report, your unique view is only shared once you are ready. The filter bar also gives you a complete picture of whether applied filters originate from interactive cross-chart filtering or quick filters.

Learn more about how to add quick filters in reports in our Help Center.

Looker Studio Quick filters and filter bar feature Visual

Pause updates

Configuring multiple filters and charts for exploration can quickly add to the query volume, even with presence of a cache. We’ve heard from analysts that they want better control over running queries, so they can optimize query volume and, thus, query costs.

We have added the ability to pause updates, giving you the flexibility to fully configure chart elements like fields, filters, parameters, sorting, and calculated formulas before running any data updates. You can then simply resume updates to see the updated data. Pausing updates does not prevent any style changes, so you can continue to modify design elements and other detailed styles and formatting without running a single query. Learn more about this feature in our Help Center.

The new pause report updates feature in Looker Studio has meaningfully improved the report creation experience. Asset producers can build and test reports without wasting database resourcing waiting for data to reload. Caroline Bollinger
BI Tooling Product, Wayfair

View underlying data

Data accuracy is one thing — being able to see its detail is another. As analysts configure charts to build reports and design information hierarchy, previewing the underlying data is important for understanding context and seeing what data is available and its structure so you can make the best decisions about what to include in your analysis. It’s also handy when troubleshooting or customizing your reports.

This feature allows analysts to preview all the data that appears in a chart, including the primary dimensions, breakdown dimensions, and metrics. Learn more about how to view underlying data in our Help Center.

Looker Studio Data preview feature Visual

With this collection of updates, Looker Studio users can now easily know the data they share is up-to-date, inspect it in detail, rapidly create filters, and share personal links to reports. The goal remains, as always, to empower users to make smart and impactful decisions based on their enterprise data. To stay on top of all our latest features, view our release notes. Access Looker Studio for free and learn more about Looker Studio Pro.

No more double vision: How Miinto improved its customer experience using Vertex AI Vision

Mon, 06 Nov 2023 17:00:00 +0000

The fashion world is notoriously fast-paced, and it can be hard to keep up with the latest trends. At Miinto, we bring over 1,000 of the world's best boutiques together in one place. We strive to offer the most customer-centric fashion platform on the planet, where users can find the best selection of premium, luxury and hand-picked local brands, specially curated to each individual user’s tastes and needs.

It can be a challenge for independent stores to reach a wider audience, because selling online can be expensive and time-consuming. Miinto helps these boutiques get their stock exposed to hundreds of thousands of global visitors every day, giving them an important new revenue stream at the click of a button.

Miinto’s data platform can help stores make crucial business decisions, such as choosing the right stock for upcoming seasons.The most important thing we do is give these boutiques the opportunity to reach new markets. And we couldn’t do this without Google Cloud.

We moved to Google Cloud in 2021, a tough year for the retail industry. We made the change for a number of reasons. First, our original technology stack was quite fragmented, which made us less efficient. Second, we were operating on a local cloud platform, and its infrastructure was slowing us down. We wanted to rapidly increase our pace of development. Ecommerce is a seasonal business, driven by big sales events such as Black Friday and the holiday season, that put a lot of stress on our infrastructure. We needed more speed and scalability, and Google Cloud was a good fit for our company.

Putting an end to duplication

We now use Google Cloud as our infrastructure and hosting environment for most of our ecosystem. We run 65 services on Compute Engine virtual machines, utilizing Terraform to build infrastructure and Ansible to provision configurations and deployments. This has allowed us to scale up and down to meet traffic peaks.

One particularly thorny challenge that Google Cloud helped us solve: inventory duplication.

Inventory duplication happens when different sellers add the same product. These duplicates need to be detected and merged, or the platform will display multiple versions of the same items to our customers. Duplicates can affect sales, as most of our customers are landing directly on the product detail page, and can’t actually see that their size is available under the duplicate. Which is, of course, a poor customer experience. Before migrating to Google Cloud, our Data Perfection team needed to manually look for a product in the database to check if the product already existed which created a bottleneck and consumed a lot of time. We needed to find a way to improve the process and bring down handling time. The solution? Vertex AI Vision.

Saving valuable time and money with Vision API

Vision API product search (which is a feature of Discovery AI for Retail) is connected directly to our arrival service, which processes incoming data from our partners. We then convert that data into products. And since we are working with a large number of sellers who often offer the same products, we need to identify if the product already exists in the database. If it is already there, then we merge the new stock with the existing product. This is where Vision AI product search comes into play, as it helps us match the same products based on image similarity.

As mentioned above, we used to only be able perform manual searches of the same product to merge the new stock. Manual operations are, of course, time consuming, ineffective in meeting demand, and often resulted in the creation of duplicates. Consequently, a separate post-production process was needed to address those duplicates, which used a lot of resources. All of this resulted in longer processing times, which impacted our time to market, and was a factor in lower conversion rates.

Vision AI allows us to enhance the product creation process, so whenever we have new products, we streamline them through our quality gates to ensure that we are offering the best customer experience. Given the number of products we have on the platform, one of our most important factors is product processing time. Increasing the speed of processing time by just a few seconds could see big business gains.

To further this cause, we built our own visual search service (VSS), which acts as a supportive service within the product creation process. VSS does two main things. Its primary job is to index products that are similar to the one coming from our partners. This is done based on pictures of the products. It then synchronizes product images between our product centers. When new products come in — which we call an arrival — VSS is responsible for uploading each product and its main image to Google Cloud Storage, and this is sent for indexing with Vision API product search. We then start the manual processing step. At the same time, VSS is querying Vision API product search which uses its pretrained image embeddings to get information about similar products, displaying potential duplicates. Our product center operators can then see if it is a duplicate. Once they confirm it, the products are merged.

Boosting conversion rates

Since implementing Vision AI, the efficiency of the arrival creation process has increased by more than 40%. The biggest benefits for Miinto are a shorter time to market and lower operating costs. We’ve found in testing that, among the five top matches returned, the duplicate image is returned 100% of the time. Of the five top matches, the duplicate is found in the very first result 98% of the time. The tests also revealed that the matching is done in less than a second. As a result, we can move computing resources to other processes based on these cost and time savings. The net result is improved our customer experience: our conversion rates have improved by up to 20% when we merge duplicates into one product.

We started working with Google Cloud during some really challenging times for the fashion industry, and we knew we needed to adapt to stay ahead. We can’t imagine how we would have got through these turbulent times if we hadn’t migrated to Google Cloud. It would have felt like mission impossible. But since our move, we’ve seen what Google Cloud can help us achieve, and we’re exploring how its AI solutions could help us grow even more.

How TEKsystems Global Services is helping retail brands break down data silos through platform modernization

Wed, 01 Nov 2023 16:00:00 +0000

Retail brands face the challenges of data duplication and data silos as businesses and technology stacks grow — while consumer expectations remain high. These challenges not only increase the complexity of data management but can also lead to a rise in overall costs. By leveraging platform modernization services within Google Cloud, such as BigQuery, BigLake, and Dataplex, retail brands can help drive faster analytics, avoid unauthorized data access risks and reduce overall security and governance challenges. Let’s break down what retail brands experience and how Google Cloud can help overcome these challenges.

Data silos can increase complexities and lower productivity

Data silos refers to compartmentalization of information that arises when different departments within a retail organization manage their data independently. For example, marketing, sales, supply chain, logistics, and customer service teams often utilize distinct systems to track and store their data. This compartmentalization, although convenient initially, can lead to added complexities as processes grow and evolve with the growth of data volumes and business needs.

The retail industry is a mammoth data machine reliant on a well-oiled data analytics engine at its heart. The data flows in from different sources where it is then stored in different systems and in different formats. This data is also subject to change from time to time that can make cross departmental collaboration difficult and error prone. Here are how these challenges can impact retail businesses:

Limited visibility and insights: Data silos prevent retailers from having a comprehensive view of their customers, operations, and overall business performance. This makes it difficult to gain insights into customer behavior, identify trends, and make informed decisions.
Inefficient operations: Data silos can lead to inefficiencies in various operational areas, such as inventory management, supply chain optimization, and customer service. When data is scattered across different systems, it increases the risk of data inconsistencies.
Limited marketing reach: Data silos prevent retailers from effectively targeting and personalizing their marketing campaigns. Without a unified view of customer data, it's challenging to understand customer preferences, interests, and purchase patterns.
Limited access control: Data silos can make it difficult to implement and enforce consistent access control policies. This can increase the risk of unauthorized access to sensitive data.

The most efficient way to address concerns about data silos and security while working with retail data from various sources is to build a data mesh.

What is a Data Mesh?

A data mesh is a new approach to data architecture that emphasizes autonomy, governance, and scalability. It is designed to address the challenges of traditional data architectures, which are often monolithic and inflexible.

A data mesh is a decentralized, scalable, and modular architecture for managing data. It is designed to address the challenges of data silos, which can make it difficult to find, use, and share data across an organization.

In a data mesh, data is owned and managed by domain experts, who are responsible for ensuring that the data is accurate, complete, and up-to-date. The data is then published to a central catalog, where it can be discovered and used by other parts of the organization.

Building a secure data mesh with Google Cloud

A data mesh works well with an analytics lake house. An analytics lake house is an architecture that combines the benefits of a data lake and a data warehouse. It provides a centralized repository for data, while also allowing for the use of analytical tools and processes that are typically associated with data warehouses.

The data mesh can provide a way to manage the data in the lake house, while the lake house can provide a place to store and analyze the data. The key benefits of the lake house architecture in the retail industry are:

Improved customer experience: Customers will no longer need to reach out to multiple different departments to get complete information on questions spanning department boundaries as the data will be easily accessible by all departments as long as they have jurisdiction over it.
Improved efficiency at lower costs: Retailers will no longer need to bear extra expenses of storing duplicate data and moving data across teams/platforms.
Securing data without silos: With fine-grained access control, data stewards can ensure data is kept secure with access being provided in accordance with the principle of least privileges.
Improved data governance: With Dataplex, data stewards and administrators can build a logical data mesh and governance layer to implement governance, validation and cataloging of data from a single interface.

Retail organizations often work with a centralized data lake where data from different sources such as sales, marketing, inventory, supply chain, etc., come in. This data often comes in unprocessed. From there, different analytics teams fetch the data, process it and then generate reports for business users.

BigQuery enables data stewards to import data into native tables called Capacitor. They can grant access to the data at various asset levels. For example, the finance team would not have access to HR data and the supply chain team would not have access to sales data. For organizations in a multi-cloud architecture, BigLake also enables the creation of BigQuery assets on data residing outside of Google Cloud in AWS and Azure.

What about data residing in Cloud Storage? Data in this object store is neither easily queryable nor suitable for running analytics on directly. BigLake solves this problem by allowing the creation of BigLake tables on Cloud Storage data. This avoids data duplication and movement. Finally, you can apply fine-grained access controls on these tables to securely share these assets with different analytics teams while the data stays in place.

Dataplex plays the governing role in building this data mesh by organizing data in Cloud Storage and BigQuery into a hierarchy of lakes, zones, and assets. Within each lake, you can create zones for subcategories such as region or business unit. You can also separate zones into raw and curated data, do data validation and auto-discovery to flag pattern changes for incoming data that can potentially break downstream processes.

Once the data mesh is built out, analysts can use BigQuery to run their analytics on popular open source engines for things like sales predictions, supply-chain analysis, or inventory estimations. The Google Cloud services discussed in this article integrate seamlessly with other Google Cloud data portfolio services like Dataflow and Dataproc, allowing users to build robust and powerful data pipelines.

Google Cloud can simplify data management, enabling retail brands to exceed consumer expectations. Using these platform modernization services within Google Cloud, businesses can not only avoid data duplication, but they can also reduce costs and down silos without compromising on security.

To learn more about how TEKsystems Global Services is helping businesses take advantage of Google Cloud, please visit www.TEKsystems.com.