Compliance

Expanding GKE posture: Policy Controller violations now in Security Command Center

Wed, 27 Sep 2023 16:00:00 +0000

Customers using Kubernetes at scale need consistent guardrails for how resources are used across their environments to improve security, resource management, and flexibility. Customers have told us that they need an easy way to apply and view those policy guardrails, so we launched the Policy Controller dashboard and added support for all GKE environments.

We received further feedback from Security Administrators that policy and compliance violation reports for GKE should be available alongside security insights from across their Google Cloud estate. To address this, we are excited to announce a fully managed integration to surface Policy Controller (CIS Kubernetes Benchmark v1.5.1 and PCI-DSS v3.2.1) violations in Security Command Center (SCC) .

SCC is our built-in security and risk management solution for Google Cloud. It helps discover misconfigurations, vulnerabilities, and compliance errors that can leave cloud assets exposed to attack.

Policy Controller can help you audit or enforce fully programmable policies for your GKE cluster resources that act as "guardrails," and prevent changes from violating security, operational, or compliance controls. Policy Controller can help accelerate your application modernization efforts by helping developers release code quickly and safely. Here are some examples of policies that you can audit or enforce with Policy Controller:

All container images must be from approved repositories
All pods must have resource limits
Resources running on my fleet of clusters should be CIS-K8s benchmark-compliant
Resources running on my fleet of clusters should be NIST-800 framework-compliant
Resources running on my fleet of clusters should be PCI-DSS benchmark-compliant

Integrating Policy Controller with SCC

Policy Controller violations are available in SCC for all Policy Controller users. Benefits of this integration include:

Increased visibility and transparency: With SCC integration, you can get organization-wide visibility into your platform and workload violations from a single dashboard. This can lead to improved security and compliance posture and reduced risk for your organization.
Ease of use: Fully managed integration means no additional build or operational overhead. It is available out-of-the-box.
On-by-default: The integration will be on-by-default for all Policy Controller and Security Command Center users.
Improved efficiency and decision-making: Integrated violations and compliance reporting provides data to inform decision making for taking the right steps to meet desired security, governance, and compliance standards.

Get started with Policy Controller and SCC integration

For existing Policy Controller and Security Command Center users, you do not need to do anything. Policy Controller violations will automatically show up in your SCC findings tab.

View Policy Controller violations from SCC Findings tab

View Finding Details for Policy Controller violations

Each Policy Controller assessment is visible alongside the other assessments SCC offers and mapped to the relevant compliance control on the SCC vulnerabilities page.

View Policy Controller findings from SCC Vulnerabilities tab

We continue to invest in building out fully managed Policy features for GKE and GKE Enterprise, focusing on ease-of-use, out-of-the-box content, and a more integrated Google Cloud experience. To get started with Policy Controller, simply install Policy Controller and try applying a policy bundle to audit your fleet of clusters against a standard such as the CIS Kubernetes benchmark. To get started with SCC today, visit the Google Cloud console and our quickstart guide.

Policy Controller dashboard: Now available for all Anthos and GKE environments

Tue, 16 May 2023 16:00:00 +0000

Customers using Kubernetes at scale need consistent guardrails for how resources are used across all their environments to improve their security, resource management and flexibility, and to achieve faster time-to-market and better operational efficiency.

At the same time, customers tell us they need an easy way to apply and view those policy guardrails, including enforcement status, violations and remediation recommendations for their fleet of clusters. In January 2023, we launched the Policy Controller dashboard along with out-of-the-box policy bundles. And now, the Policy Controller dashboard is available for all Google Kubernetes Engine (GKE) and Anthos environments (i.e., on-premises, multi-cloud and attached clusters) and includes a powerful flow to help remediate violations.

Policy Controller lets you enforce fully programmable policies for your cluster resources that act as "guardrails" and prevent any changes from violating security, operational, or compliance controls. Policy Controller can help accelerate your application modernization efforts by helping developers release code quickly and safely. Here are some examples of policies that you can audit or enforce with Policy Controller:

All container images must be from approved repositories
All ingress hostnames must be globally unique
All pods must have resource limits
All namespaces must have a label that lists a point-of-contact
Resources running on my fleet of clusters should be CIS-K8s benchmark-compliant
Resources running on my fleet of clusters should be PCI-DSS benchmark-compliant

The Policy Controller dashboard

With the Policy Controller dashboard, meanwhile, platform and security admins can:

Get an at-a-glance view for the state of all the policies applied to the cluster fleet, including enforcement status (dryrun, warn, or enforced)
Easily troubleshoot and resolve policy violations by referring to opinionated recommendations for each violation
Get visibility into the compliance status of cluster resources

The Policy Controller dashboard is designed to be user-friendly and intuitive, making it easy for users of all skill levels to manage and monitor violations for their fleet of clusters, with a centralized view of policy violations that they can act on if necessary.

The circular “donuts” on the dashboard above show you the state of policies across all environments, including Policy Controller’s overall install status, any clusters that are in violation, and the total policies applied across the fleet of clusters. Enforcement action shows if you are just auditing your resources against policies or enforcing the policies at admission time on your clusters.

The bottom part of the Policy Controller dashboard page shows the coverage for each policy bundle for your fleet, breaking it down into percentage of resources that are compliant or in violation with this bundle:

A fully grayed-out bar implies that the bundle is not applied to this fleet of clusters. If the bar is partially grayed-out, it means that the bundle has not been applied to one or more clusters in the fleet.
If one or more bundles are applied, overall compliance for your resources is calculated against the bundle. The blue portion of the bar shows the resources that are in compliance with the policy, while orange represents the violations.

For each bundle, you can click on the violations link, which will take you to the detailed Violations tab depicted in the image below.

The new Violations tab makes it easy to view and understand policy violations in your environment.

The Violations tab lets you filter violations based on things like cluster, bundle, or resource type. Furthermore, you can now group violations by constraint and namespace to help different user personas do their jobs more efficiently. From either one of the groupings, users can click on the constraint link which will show the Constraint view as shown below.

The Constraint details tab shows the remediation action and constraint yaml.

The Constraint details tab describes the constraint, and shows the recommended action to fix the violations against the constraint and affected resources. You can also view the constraint’s YAML as it exists on your clusters. The affected resources tab lists all the resources that are in violation of the constraint with a detailed error message as shown in the image below.

The Affected Resources tab showing the resources in violation.

Get started today

We continue to invest in building out fully managed Policy features for GKE and Anthos, focusing on ease-of-use, out-of-the-box content, and a more integrated Google Cloud experience. To get started with Policy Controller, simply install Policy Controller and try applying a policy bundle to audit your fleet of clusters against a standard such as the CIS Kubernetes benchmark. You can also try Policy Controller to audit your cluster against the Policy Essentials bundle. Stay tuned to learn more about the latest features in Policy Controller in our next blog post, and we hope you join us at Next ’23 to learn more.

Harden your Kubernetes clusters and monitor workload compliance at scale with new PCI DSS policy bundle

Wed, 08 Feb 2023 17:00:00 +0000

PCI DSS is a set of network security and business best practices guidelines adopted by the PCI Security Standards Council to establish a “minimum security standard” to protect customers’ payment card information. Google Cloud undergoes a third-party audit to certify individual products against the PCI DSS at least annually, and customers can build off of these attestations to measure their applications’ compliance. This blog can help you evaluate your new and existing applications for PCI DSS compliance.

Policy Controller enables the enforcement of fully programmable policies for your clusters. A Policy bundle is an out-of-the-box set of constraints that are created and maintained by Google Cloud. Policy bundles help audit your cluster resources against Kubernetes standards, industry standards, or Google Cloud-recommended best practices. Many policy bundles are available now, and they can be easily used by a new or existing user as-is without writing a single line of code. You can also view the status of Policy bundle coverage and compliance for your fleet of clusters using Policy Controller dashboard.

The PCI DSS v3.2.1 Policy bundle

Security administrators from your organization can view the alignment for your applications with the PCI DSS requirements by viewing the violations for the PCI DSS Policy bundle. Each constraint in the PCI DSS bundle also has the PCI DSS control number listed which can be mapped back to PCI requirements, these mappings may be used during the compliance reporting, as needed. More information on how to view the list of violations is covered in the next section in this blog.

The PCI DSS v3.2.1 Policy bundle includes policies focusing on the following areas:

Secure networks and systems

Ensures requirements for a firewall by requiring all apps to contain a specified audit label.
Ensures requirements for network-controls by requiring all apps to contain a specified annotation.
Requires that every namespace defined in the cluster has a NetworkPolicy.
Requires a valid app.kubernetes.io/managed-by= label on RoleBinding resources.
Restricts the creation of resources using a default service account.
Restricts pods from using the default namespace.

Secure systems and applications

Enforce all PeerAuthentications cannot overwrite strict mTLS.
Requires the presence of an anti-virus daemonset.
Enforce the presence and enablement of Anthos Config Management.
Enforce Cloud Armor configuration on BackendConfig resources.

Strong access control and monitoring

Restricts the use of basic-auth type secrets.
Ensures consistent and correct time on Nodes by ensuring the usage of Container-Optimized OS as the OS image.

Using the PCI DSS v3.2.1 Policy bundle

The PCI DSS v.3.2.1 Policy bundle can be installed on Anthos Cluster(s) with Policy Controller v1.14.0 or higher. The policies included are configured in "audit" mode by default, so they do not impact any of your existing or new workloads. You can apply Policy bundles using kubectl (demonstrated just below), kpt, or Config Sync.

1. Install and initialize the Google Cloud CLI, which provides the gcloud and kubectl commands used in these instructions. If you use Cloud Shell, Google Cloud CLI comes pre-installed.

2. Install Policy Controller on your Anthos cluster with the referential constraints and the Policy Controller Constraint Template Library enabled.

3. Save the following YAML manifest to a file as policycontroller-config.yaml. The manifest configures Policy Controller to watch specific kinds of objects.

Note: If you already have an existing Config in the gatekeeper-system namespace, you must include all previous customization settings to preserve your changes.

code_block: <ListValue: [StructValue([('code', 'apiVersion: config.gatekeeper.sh/v1alpha1\r\nkind: Config\r\nmetadata:\r\n name: config\r\n namespace: "gatekeeper-system"\r\nspec:\r\n sync:\r\n syncOnly:\r\n - group: "apps"\r\n version: "v1"\r\n kind: "DaemonSet"\r\n - group: "networking.k8s.io"\r\n version: "v1"\r\n kind: "NetworkPolicy"'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f3aa99ff4c0>)])]>

4. Apply the policycontroller-config.yaml manifest:

code_block: <ListValue: [StructValue([('code', 'kubectl apply -f policycontroller-config.yaml'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f3aa99ffd00>)])]>

4. Preview the policy constraints with kubectl

code_block: <ListValue: [StructValue([('code', 'kubectl kustomize https://github.com/GoogleCloudPlatform/acm-policy-controller-library.git/anthos-bundles/pci-dss-v3.2.1'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f3aa99ff850>)])]>

5. Apply the policy constraints with kubectl:

code_block: <ListValue: [StructValue([('code', 'kubectl apply -k https://github.com/GoogleCloudPlatform/acm-policy-controller-library.git/anthos-bundles/pci-dss-v3.2.1'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f3aa99fffa0>)])]>

The output will be similar to the following:

code_block: <ListValue: [StructValue([('code', 'asmpeerauthnstrictmtls.constraints.gatekeeper.sh/pci-dss-v3.2.1-asm-peer-authn-strict-mtls created\r\nk8sblockcreationwithdefaultserviceaccount.constraints.gatekeeper.sh/pci-dss-v3.2.1-block-creation-with-default-serviceaccount created\r\nk8sblockobjectsoftype.constraints.gatekeeper.sh/pci-dss-v3.2.1-block-secrets-of-type-basic-auth created\r\nk8senforcecloudarmorbackendconfig.constraints.gatekeeper.sh/pci-dss-v3.2.1-enforce-cloudarmor-backendconfig created\r\n...'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f3aa99ff220>)])]>

6. Verify that policy constraints have been installed and check if violations exist across the cluster:

code_block: <ListValue: [StructValue([('code', 'kubectl get constraint -l policycontroller.gke.io/bundleName=pci-dss-v3.2.1'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f3aa99ff5e0>)])]>

The output is similar to the following:

code_block: <ListValue: [StructValue([('code', 'NAME ENFORCEMENT-ACTION TOTAL-VIOLATIONS\r\nasmpeerauthnstrictmtls.constraints.gatekeeper.sh/pci-dss-v3.2.1-asm-peer-authn-strict-mtls dryrun 0\r\n\r\nNAME ENFORCEMENT-ACTION TOTAL-VIOLATIONS\r\nk8sblockcreationwithdefaultserviceaccount.constraints.gatekeeper.sh/pci-dss-v3.2.1-block-creation-with-default-serviceaccount dryrun 0\r\n\r\nNAME ENFORCEMENT-ACTION TOTAL-VIOLATIONS\r\nk8sblockobjectsoftype.constraints.gatekeeper.sh/pci-dss-v3.2.1-block-secrets-of-type-basic-auth dryrun 0\r\n…'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f3aa99fff10>)])]>

In order to remediate the violations, we recommend that you update your resource(s) yaml — some guidelines are included here. Each violation will also include steps to fix the violation, which can be viewed both from CLI and the Policy Controller dashboard.

Viewing the PCI DSS Policy bundle violations on Policy Dashboard

Violations on the cluster can also be viewed in the UI using the Policy Controller Dashboard.

The Policy Controller dashboard

Monitoring the cluster(s) for PCI DSS Policy Bundle violations

The PCI DSS policy bundle by default has its enforcement action set to dryrun, which is the configuration for Policy Controller to show you violations without blocking or aborting any resources. This gives you the ability to audit your clusters, share any violations with workload owners and collaborate on fixing critical security issues.

All policy violations are automatically recorded in Cloud Logging and can be found by applying these filters in the Logs Explorer:

code_block: <ListValue: [StructValue([('code', 'resource.type="k8s_container"\r\nresource.labels.namespace_name="gatekeeper-system"\r\nresource.labels.pod_name:"gatekeeper-audit-"\r\njsonPayload.process: "audit"\r\njsonPayload.event_type: "violation_audited"\r\njsonPayload.constraint_name:*\r\njsonPayload.constraint_namespace:*'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f3aa99ff610>)])]>

You can also set up log based alerts using Cloud Monitoring for whenever policy violations occur to get notified.

Policy Controller includes the metrics related to policy usage such as number of constraints, constraint templates, audit violations detected just to name a few (see list of metrics exposed). These metrics can be exported to cloud monitoring and/or prometheus at install time (blog, docs). You can also set up alerts based on metrics.

Conclusion

Policy Controller enables the enforcement of both Google created and maintained Policy bundles and custom policies for your cluster which prevent changes to the Kubernetes API from violating security, operational, or compliance controls. Optionally, Policy Controller can also be used to analyze configuration for compliance before deployment to your Kubernetes cluster.

Get started today

The easiest way to get started with Anthos Policy Controller is to install Policy Controller and try out some of the other Google created and maintained Policy bundles:

Apply policy bundles and monitor policy compliance at scale for Kubernetes clusters

Mon, 23 Jan 2023 17:00:00 +0000

As more enterprise customers are adopting a hybrid and multi cloud strategy, centralized security and governance become increasingly important as workloads are distributed across the environments. Anthos is our cloud-centric container platform to run modern applications anywhere consistently and at scale. Anthos Config Management (ACM) automates policy and security for Kubernetes clusters and is comprised of Config Sync, Config Controller, and Policy Controller. Config Sync reconciles the state of clusters with one or more Git repositories. Config Controller is a hosted service that allows administrators to manage Google Cloud Platform (GCP) resources in a declarative fashion. This blog covers the enhancements we have brought to the Policy Controller component.

As a key component of ACM, Policy Controller enables the enforcement of fully programmable policies for your clusters. These policies act as "guardrails" and prevent any changes from violating security, operational, or compliance controls. Policy Controller can help accelerate your application modernization efforts by helping developers release code quickly and safely.

We are thrilled to announce the launch of our new built-in Policy Controller Dashboard, a powerful tool that makes it easy to manage and monitor the policy guardrails applied to your Fleet of clusters.

With Policy Controller Dashboard, Platform and Security Admins can:

Get an at-a-glance view for the state of all the policies applied to Fleet of clusters including enforcement status (dryrun or enforced)
Easily troubleshoot and resolve policy violations by referring to opinionated recommendations for each violation
Get visibility into compliance status of the cluster resources

Policy Controller Dashboard is designed to be user friendly and intuitive, making it easy for users of all skill levels to manage and monitor violations for their fleet of clusters. It allows you to have a centralized view of Policy violations and take action if necessary.

The Anthos Policy Controller dashboard

The dashboard can also show you which of your resources are affected by a specific policy, and can make opinionated suggestions on how to fix the problem.

Identifying resources affected by vulnerabilities

Introducing Policy Bundles

Policy bundle is an out-of-the-box set of constraints that are created and maintained by Google. The bundles help audit your cluster resources against kubernetes standards, industry standards, or Google recommended best practices.

Policy bundles are available now, and can be easily used by a new or existing user as-is i.e. without writing a single line of code. Users will view the status of Policy bundle coverage for the fleet from the Policy Controller dashboard i.e. if you have 4 clusters in your fleet and you have applied the PCI DSS 3.2.1 bundle on all 4 clusters then the dashboard will show a 100% coverage for your fleet. In addition to coverage, the dashboard will also show the overall state of compliance for each bundle for the entire fleet of clusters.

Following policy bundles are available now with Anthos:

PCI DSS 3.2.1: Helps audit your cluster resources against the PCI-DSS 3.2.1 industry standard
CIS Kubernetes Benchmark 1.5.1 : Helps audit your cluster resources against the CIS Kubernetes Benchmark, a set of recommendations for configuring Kubernetes to support a robust security posture.
PSS Baseline: Helps audit your cluster resources against the PSS - Baseline
PSS Restricted: Helps audit your cluster resources against the PSS - Restricted
PSP: Helps audit your cluster resources against Pod Security Policies
Policy Essentials: Helps audit your cluster resources against Google recommended best practices for containerized workloads
Anthos Service Mesh Security : Helps audit your cluster for recommended Anthos Service Mesh best practices

Get started today

The easiest way to get started with Anthos Policy Controller is to just install Policy controller and try applying a policy bundle to audit your fleet of clusters against a standard such as CIS benchmark.

You can also Try Policy controller to audit your cluster against Policy Essentials bundle.

CISO Survival Guide: Vital questions to help guide transformation success

Mon, 09 Jan 2023 17:00:00 +0000

Part of being a security leader whose organization is taking on a digital transformation is preparing for hard questions – and complex answers – on how to implement a transformation strategy.

In our previous CISO Survival Guide blog, we discussed how financial services organizations can more securely move to the cloud. We examined how to organize and think about the digital transformation challenges facing the highly-regulated financial services industry, including the benefits of the Organization, Operation, and Technology (OOT) approach, as well as embracing new processes like continuous delivery and required cultural shifts.

As part of Google Cloud’s commitment to shared fate, today we offer tips on how to ask the right questions that can help create the conversations that lead to better transformation outcomes for your organization. While there often is more than one right answer, a thoughtful, methodical approach to asking targeted questions and maintaining an open mind about the answers you hear back can help achieve your desired result. These questions are designed to help you figure out where to start and where to end your organization’s security transformation. By asking the following questions, CISOs and business leaders can develop a constructive, focused dialogue which can help determine the proper balance between implementing security controls and fine-tuning the risk tolerance set by the executive management and the board of directors.

aside_block: <ListValue: [StructValue([('title', 'Hear monthly from our Cloud CISO in your inbox'), ('body', <wagtail.rich_text.RichText object at 0x7f3aaa173eb0>), ('btn_text', 'Subscribe today'), ('href', 'https://inthecloud.withgoogle.com/google-cloud-ciso-newsletter/signup.html'), ('image', None)])]>

To start the conversation, begin by asking:

What defines our organization’s culture?

How can we best integrate the culture with our security goals?

CISOs should ask business leaders:

What makes a successful transformation?

What are the key goals of the transformation?

What data is (most) valuable?

What data can be retired, reclassified, or migrated?

What losses can we afford to take and still function?

What is the real risk that the organization is willing to accept?

Business leaders should ask CISOs and the security team:

What are the best practices for protecting our valuable data?

What is the business impact of implementing those controls?

What are the top threats that we need to address?

CISOs and business leaders should ask:

Which threats are no longer as important?

Where could we potentially use spending for more cost-effective controls such as firewalls and antivirus software?

What benefits do we get from refactoring our applications?

Are we really transforming, or lifting and shifting?

How should we perform identity and access management to meet our business objectives?

What are the core controls needed to ensure enterprise-level performance for the first workloads?

CISOs and risk teams should ask:

How can we use the restructuring of an existing body of code to streamline security functions?

How should we monitor our security posture to ensure we are aligned with our risk appetite?

Business and technical teams should ask:

What’s our backup plan?

What do we do if that fails?

Practical advice and the realities of operational transformation

Some organizations have been working in the cloud for more than a decade and have already addressed many operational procedures, sometimes with painful lessons learned along the way. If you’ve been operating in the cloud securely for that long, we recognize that there’s a lot to be gained from understanding your approaches to culture, operational expertise, and technology.

However, there are still many organizations that have not thought through how they will operate in a cloud environment until it’s almost ready – and at that point, it might be too late. If you can’t detail how a cloud environment will operate before its launch, how will you know who should be responsible for maintaining it?

Who are the critical stakeholders, along with those responsible for engineering and maintaining specific systems, who should be identified at the start of the transformation? There are likely several groups of stakeholders, such as those aligned with operations for transformation, and those focused on control design for cloud aligned with operations.

If you don’t have the operators involved in the design phase, you’re destined to create clever security controls with very little practical value because those tasked with day-to-day maintenance most likely won’t have the expertise or training to effectively operate these controls.

This is complicated by the fact that many organizations are struggling to recruit and retain resources with the right skills to operate in the cloud. We believe that training current employees to learn new cloud skills, and giving them the time away from other responsibilities, can help build skilled, diverse cloud security teams.

If your organization continually experiences high turnover in security leadership and skilled staff, it’s up to you to navigate your culture to ensure greater consistency. You can, of course, choose to supplement internal knowledge with trusted partners – however, that’s an expensive strategy for ongoing operational cost.

We met recently with a security organization that turns over skilled staff and leadership every two to three years. This rate of churn results in a continual resetting of security goals. This particular team joked that it’s like “Groundhog Day” as they constantly re-evaluate their best security approaches yet make no meaningful progress. This is not a model to emulate.

Many security controls fail not because they are improperly engineered, but because the people who use them – your security team – are improperly trained and insufficiently motivated. This is especially true for teams with high turnover rates and other organizational misalignments. A security control that blocks 100% of attacks might be engineered correctly, but if you can’t efficiently operate it, the effectiveness of the control will plummet to zero over time. Worse, it then becomes a liability because you incorrectly assume you have a functioning control.

In our next blog, we will highlight several proven approaches that we believe can help guide your security team through your organization’s digital transformation.

To learn more now, check out:

Detect spoofing exceptions in financial markets with Google Cloud and GTS

Tue, 29 Nov 2022 17:00:00 +0000

Regulatory Surveillance of Trading Activity with Google Cloud

The purpose of regulatory surveillance is to verify market fairness and protect against unethical trading behavior. Since regulatory reporting requirements have local differences, global financial institutions often need to operate in a large number of jurisdictions, meet specific variations of the local regulations of each, conduct internal surveillance, and justify their firm's compliance approaches. Additionally, with the rapid increase in low-latency co-location trading networks, the speed at which orders are entered to the market and transactions are executed has gone from seconds to microseconds, while the volume has simultaneously exploded. Thus, these firms require robust recording and surveillance solutions to monitor and audit trading activity for potential market manipulation and make this available to internal stakeholders and regulators. This blog focuses on one such solution that is built on Google Cloud, using SQL analytics on BigQuery.

Market surveillance is getting more complex as deceptive trading practices increase with higher trading volume and the emergence of new asset classes. One such deceptive practice is “spoofing,” which largely revolves around the intent (or lack thereof) of traders to place bonafide orders, devoid of any premeditation to cancel a particular order before or at the time of its placement. “Flashing” is a form of market spoofing — where market participants exhibit a pattern of submitting orders that are not intended to be fulfilled, but rather only to move ("improve") the market to benefit a subsequent order on the other side of the market. The “flashed” orders are short-lived orders, which are canceled quickly after being entered and before getting executed. In the world of electronic trading, order entry is fully automated and typically happens within milliseconds or microseconds.

Large financial institutions have solved this by implementing in-house surveillance systems that operate in their local data centers. Although on-premises solutions achieve the desired result of detecting and reporting such activities, one challenge is their long implementation and testing cycles. Another is the costly and rigid infrastructure, often overprovisioned to serve the peak times of the reporting cycles and sitting largely unused the remainder of the time. Hence, these institutions need mechanisms for efficiently operating the surveillance processes, and use the data for value-add activities.

This blog post’s focus — a cloud-native pipeline that fits to help meet cost-saving targets, operational efficiencies and demanding evolving regulatory requirements — collaboratively developed by GTS Securities LLC, Strike Technologies LLC (its technology provider) and Google Cloud, is one such mechanism.

GTS is a leading global, electronic market maker that combines market expertise with innovative, proprietary technology. As a quantitative trading firm continually building for the future, GTS leverages the latest in artificial intelligence systems and sophisticated pricing models to bring consistency, efficiency, and transparency to today’s financial markets. GTS accounts for 3-5% of daily cash equities volume in the U.S. and trades over 30,000 different instruments globally, including listed and OTC equities, ETFs, futures, commodities, options, fixed income, foreign exchange, and interest rate products. GTS is the largest Designated Market Maker (DMM) at the New York Stock Exchange, responsible for nearly $12 trillion of market capitalization.

Detecting flashing exceptions

GTS modeled its flashing detection analytics solution on Google Cloud after its post-trade surveillance system, a large-scale simulation framework that marries the firm’s bidirectional trading order data with the high resolution market-data feeds disseminated by the exchanges into a uniform stream. The stream is analyzed by numerous surveillance reports to perform a fully automated regulatory compliance review.

Datasets

Market Data

Public high resolution tick data disseminated by the exchanges

A unidirectional data stream of quotes, trades, symbol trading status, etc.

These per-exchange feeds are collected, normalized and merged to a per-symbol uniform stream
For the purposes of this surveillance, the market-wide best bid and ask price (NBBO - National Best Bid / Offer) at any given time are used

Order Data

Proprietary order activity of the market participant under review

A bidirectional data stream initiated by the market participant

A simplified life cycle of a single order is illustrated below:

Flashing detection

This surveillance is designed to capture the manipulative practice of placing orders with no trading intention. Rather, these orders are placed on the market to induce a favorable movement.

A flashing activity is composed of a “flash” and “take” events:

Flash

Entry of short-lived orders on one side of the market
These orders are not meant to be executed (filled) just to show an artificial interest on the market, as such they are canceled a short period after being entered

Note that the flashed order lifespan is a parameter of the surveillance that should be configured to fit the trading profile of the firm under review
For demonstration purposes we used 500ms (though in practice this can happen over a much smaller timespan)

Market-movement

Other market participants react to the flashed order/s, moving the price towards the indicated direction, i.e., causing the NBBO to “improve”

Take

Placement of an order/s on the other side of the market that gets executed against other market participants that joined the market at the improved price
Note that the take order can be “staged” on the market prior to the initiation of the flashing order or entered after it
Furthermore, the take event should be in proximity to the Flash event

For demonstration purposes we configured this parameter to 10 seconds

As indicated above, flashing detection requires marrying the order activity of the market participant under review with the public market data, essentially synchronizing two large and discrete nanosecond resolution datasets. In addition to the technical challenge of joining these datasets, we recognize that a trading platform might introduce a small latency when processing large market-data volumes. Further, there might be slight clock differences between the trading-platform and surveillance system’s view of the market-data. To compensate for this difference, the surveillance system employs a windowing mechanism when reviewing the NBBO (National Best Bid and offer) near the take event. The NBBO window length is a configurable parameter of the surveillance and should be set based on the specifics of the trading firm under review. For demonstration purposes we used a 1000ms window. Note that as is the case for discrete data, there might not be any NBBO update within the window. In such cases we carry forward the last NBBO update prior to the window.

Finally, any sequence of events that fit the scenario outlined above are flagged as potential exceptions to be reviewed by the compliance officer of the trading firm.

Modernizing this solution using SQL Analytics on Google Cloud

To implement the above points, the collaboration applies data analytics best practices to a financial services problem. This solution can enable efficient, flexible data processing — but also supports the organizational processes that enable reliable reporting and minimize fire drills.

At the highest level, the steps are executed in containers, which models the macros steps — e.g., load data / execute transform / run data quality. Cloud Composer (Apache Airflow) is the tool that helps orchestrate containers to run BigQuery SQL jobs. As discussed in the next section in detail, these queries are defined using DBT models instead of raw SQL.

DBT is the conversion tool that codifies the regulatory reporting transformations. It runs the SQL logic which implements the surveillance rules required by the regulator as per the rulebook. With DBT, models are parameterised into reusable components and coupled with BigQuery to execute the SQL models. It fits nicely into the stack and helps in creating a CI/CD pipeline that feeds normalized data to BigQuery.

BigQuery is a critical solution component because it can be cost-efficient, offer minimal operational overhead, and solve this high-granularity problem. By cheaply storing, rapidly querying and joining huge, granular datasets, BigQuery helps provide firms with a consistent source of high-quality data. This single source serves multiple report types.

The entire infrastructure in Google Cloud is deployed using Terraform, which is an open-source ‘Infrastructure as a code’ tool. It helps in storing configuration in declarative forms, Kubernetes yaml, etc; which helps promote portability, repeatability, and enable the platform to scale.

Solution overview

First, the pipeline must process sparse data from the two feeds, that is the public NBBO market data feed disseminated by the exchanges and proprietary order activity of the market participant under review. The source data is ingested to BigQuery, which supports running regulatory reporting workloads that exhibit frequent spikes in demand. This serverless infrastructure helps in processing complex queries within seconds and to address the scaling problem of legacy implementations.

Second, the pipeline must evaluate the order lifecycle, whether an order was entered and canceled within a short timeframe and whether this induced a market movement, which was later capitalized on. These conditions are compiled as reporting rules and expressed using ANSI SQL. As discussed above, we have leveraged an open source framework (DBT) to create reporting rules as code (i.e., modules) that can be released to regulated institutions. These reporting rules are in the form of DBT models that bundle together documentation (e.g., fields definitions) and technical schema. This framework also can remove the need for explicit maintenance by automatically building SQL DAGs of the models.

Third, the pipeline must execute these reporting rules for the order events and check for fluctuations in the market. A huge number of simple SQL queries may need to be spawned for generating the final reports. Hence, an orchestration capability can coordinate this processing, which takes place in a number of services. This is supported by running Cloud Composer workflows on Kubernetes. The output of the processing is saved back to BigQuery.

The end result is the report generation, which is supported by the analytics and reporting layer, that allows users to explore the data and work with it. This data can be accessed by Business Intelligence tools such as Looker or Google Sheets. If required, data can also be egressed on prem for analysis.

Give it a try

This approach is available to help you meet your organization’s reporting needs. Please review our user-guide, whose Tutorials section provides a step-by-step guide to constructing a simple data processing pipeline that can maintain quality of data, auditability, and ease of change and deployment, and also supports the requirements of regulatory reporting.

CISO Survival Guide: How financial services organizations can more securely move to the cloud

Mon, 14 Nov 2022 17:00:00 +0000

It’s not just children and adults who face excitement and nervousness on the first day of school. The first day in the cloud can be daunting for financial services organizations, too.

Chief Information Security Officers must lead the cloud security component of their organization’s digital transformation, a complicated task beset by many questions that the members of our Google Cybersecurity Action Team can help answer. We want to help you move into the brave new world of digital transformation and build engaged, robust cybersecurity teams as you go because there is no “one size fits all” approach to cloud security.

We’ve worked with many financial services organizations in the middle of their transformations. Some want to revolutionize how their organizations achieve their cybersecurity goals. Others want to have minimal viable security controls for Day 1 launches. Each organization has its own operational and technological needs, its own funding sources, and its own risk appetites, all of which can fundamentally influence security strategy.

We’re here to offer our real-world knowledge and experiences from Google’s Office of the Cloud CISO to help you move boldly – and more securely – to the cloud. We do this as part of our commitment to operate in a shared fate model that helps our customers achieve the best possible security outcomes. We strongly believe that secure organizations make for a more secure world.

First come the questions, so many questions

Many times, we go into customer organizations as they are on the cusp of moving to the cloud and hear questions such as:

I’ve never done this before, what do I need to worry about first?
How do we make sure we don’t move our technical and cyber debt to the cloud?
What are the key threats that I need to pay attention to?
What on-premises baggage am I going to be left with?
How do I organize my team to best address the things that we need to focus on?

What becomes apparent from these conversations is that technology and security leaders use moving to the cloud as an opportunity to transform their businesses. This is an excellent plan. However, just because technical and cyber debt were not created intentionally does not mean that they can be wished away. It takes a concerted effort to reduce risk by building on solid fundamentals and leveraging the advantages of the cloud to pay down that debt.

These areas of concern and the strategies for addressing them can be categorized around your organization and its operations, technology, and people – and your CISO leadership.

Teach your organization to think cloud

Recently, security teams have been organizing around security compliance models such as the NIST cybersecurity framework. While this provides a foundation to discuss security disciplines and general security posture, it doesn’t necessarily provide the best way to organize your security team for optimal impact.

In addition, most of these frameworks were developed before cloud was widely adopted in regulated industries. We now have more specialized knowledge and tools to more effectively serve specialized cases and verticals.

As use of the cloud becomes more prevalent, frameworks need to evolve and adapt to new threats and a new operating environment with rapid business changes and agile IT . Fundamentally, digital transformation is about organizational change management. A key component of preparing for digital transformation is guiding the people in your organization to evolve beyond on-premises mindsets to adopt new ones.

In our discussion on how CISOs need to adapt their mental models for cloud security, we noted that security during and after a digital transformation should focus on how network and endpoint security, detection and response, data security, and identity and access management (IAM) function in the cloud — and how taking advantage of those differences can help you build a more resilient security posture.

The right questions can drive security changes

One key question to ask yourself when making strategic and tactical decisions is: Why am I implementing this security control?

Digital transformation provides an excellent opportunity to re-examine your team (because culture comes first in cloud transformation) and lead the way to changes that address your organization’s go-forward strategies when it comes to firewalls, antivirus software, applications, data protection, your overall security and risk postures, and your backup plans. Changing technical controls first rarely leads to success.

Your organization needs to have a clear vision and set objectives to determine how to most effectively achieve its security goals. Most of the time this means that CISOs and their teams have to reach outside their comfort zone and work with technology, business, and other partners to achieve success. If your organization goes down the path of “it’s always been done this way on-premises,” your cloud transformation is more apt to be inefficient and ultimately block the business from achieving agility and security.

At the September conference Measuring Cyber Risk in the Financial Services Sector hosted by MIT and the Federal Reserve Board, an audience member posed an important question to the panel: Why do cyber insurers ask if I have file integrity monitoring installed?

This kind of question from cyber insurers is indicative of the mindset that should evolve with the digital transformation process. We want to be open to new opportunities to rethink practices and architecture. File integrity in a vacuum means very little to the overall risk reduction of your organization. Depending on their objective, cyber insurers could have asked a different set of questions, such as: How do you ensure that critical payment data is not altered in the transaction flow? And how do you ensure that software running in production is authorized and not altered?

Both questions could be answered with file integrity monitoring. However, answering a question on a cyber insurer’s questionnaire provides little to no value. It’s a check-the-box exercise that doesn’t provide a measurable security benefit. Cloud provides the same opportunities to rethink standard controls and generate better security and business outcomes.

As you begin implementing security in the cloud, keep in mind what your organization’s ideal security posture should be and come to an agreement with stakeholders (including business and IT leaders) about how you can set and achieve your goals. The first steps offer an invaluable “pressure test” for your organization – and take comfort in the fact that very few CISOs get it right on the first try. That’s why you should be adaptable, be open to change, and work to minimize organizational strife as much as possible.

We will continue this discussion in the next blog focused on the realities of starting the operational transformation.

To learn more now, check out our podcast on CISO frustrations, successes, and lessons learned, and our guidance report on cloud security transformations as well as our white paper on building operational resilience in financial services. Review Google Cybersecurity Action Team site for additional papers and other guidance.

View policy enforcement metrics for ACM Policy Controller

Fri, 23 Sep 2022 16:00:00 +0000

Policy Controller enables the enforcement of fully programmable policies for your clusters. These policies act as "guardrails" and prevent any changes from violating security, operational, or compliance controls at admission time, and post admission, using continuous audit.

Through ongoing conversations with platform and security administrators, we have received feedback about increasing visibility into how the policies are applied i.e. enforced or audited across Anthos or GKE clusters.

With the Anthos Config Management (ACM) 1.12.0 onwards, we have made it easier to export and visualize Policy Controller metrics.

Policy Controller Metrics

Policy controller includes the metrics related to policy usage such as number of constraints, constraint templates, audit violations detected just to name a few (see list of metrics exposed).

Exporting the metrics

Policy Controller uses OpenCensus to create and record metrics related to its processes and policy usage. Policy Controller can be easily configured to export these metrics to Prometheus and/or Cloud Monitoring at the install time. Default setting for exporting metrics for Policy controller will export the metrics to both Prometheus and Cloud monitoring.

Viewing the metrics

These metrics are exported to the customer's Cloud Monitoring project in Prometheus format. As a result, customers can view these metrics in the Cloud Monitoring UI or query them via the Cloud Monitoring API using either PromQL (the de-facto query language for Kubernetes metrics) or MQL (Google's proprietary metrics query language).

There is also a newly added cloud monitoring dashboard to view your metrics. This dashboard can be further edited to meet your business or operational needs.

This dashboard can be imported from within Cloud Console.

Login to Cloud Console and click on the hamburger (collapsed) menu and click on More Products to expand the list of products in the menu.
Select Monitoring > Dashboards and then click the Sample Library tab on the page.This will show all the samples available by category.
Select Anthos Config Management from the list.
Check Policy Controller from the list and click Import.
Confirm that you want to import the dashboard.
This will create a new dashboard.
You can view by clicking on the Dashboards menu item and then selecting the newly created Policy Controller dashboard from the list.

Pricing

These metrics are available at no additional cost to our customers.

Alerting on the metrics

You can create alerting policies in Cloud Alerting so you are notified in case something needs your attention.

Third Party integration

Any third party observability tool can ingest these metrics using Cloud Monitoring API. If you are using Grafana dashboards all you have to do is point it to the Cloud Monitoring API for it to work.

Next steps

Introducing fine-grained access control for Cloud Spanner: A new way to protect your data in Spanner

Thu, 15 Sep 2022 15:00:00 +0000

As Google Cloud’s fully managed relational database that offers unlimited scale, strong consistency, and availability up to 99.999%, Cloud Spanner powers applications of all sizes in industries such financial services, gaming, retail, and healthcare. Today, we’re excited to announce the preview of fine-grained access control for Spanner that lets you authorize access to Spanner data at the table and column level. With fine-grained access control, it’s now easier than ever to protect your transactional data in Spanner and ensure appropriate controls are in place when granting access to data.

In this post, we’ll take a look at Spanner’s current access control model, examine the use cases of fine-grained access control, and look at how to use this new capability in your Spanner applications.

Spanner’s access control model today
Spanner provides access control with Identity and Access Management (IAM). IAM provides a simple and consistent access control interface for all Google Cloud services. With capabilities such as a built-in audit trail and context-aware access, IAM makes it easy to grant permissions at the instance and database level to Spanner users.

The model for IAM has three main parts:

Role. A role is a collection of permissions. In Spanner, these permissions allow you to perform specific actions on Spanner projects, instances, or databases. For example, spanner.instances.create lets you create a new instance, and spanner.databases.select lets you execute a SQL select statement on a database. For convenience, Spanner comes with a set of predefined roles such as roles/spanner.databaseUser, which contains the permissions spanner.databases.read and spanner.databases.write, but you can define your own custom roles, too.
IAM principal. A principal can be a Google Account (for end users), a service account (for applications and compute workloads), a Google group, or a Google Workspace account that can access a resource. Each principal has its own identifier, which is typically an email address.
Policy. The allow policy is the collection of role bindings that bind one or more principals to individual roles. For example, you can bind roles/spanner.databaseReader to IAM principal user@abc.xyz.

The need for more robust access controls
There are a number of use cases where you may need to define roles at a level that is more granular than the database-level. Let’s look at a few of these use cases below.

Ledger applications
Ledgers, which are useful for inventory management, cryptocurrency, and banking applications, let you look at inventory levels and apply updates such as credits or debits to existing balances. In a ledger application, you can look at balances, add inventory, and remove inventory. You can’t go back and adjust last week’s inventory level to 500 widgets. This corresponds to having SELECT privileges (to look at balances) and INSERT privileges (to add or remove inventory), but not UPDATE or DELETE privileges.

Analytics users
Analytics users often need SELECT access to a few tables in Spanner database, but should not not have access to all tables in the database. Nor should they have INSERT, UPDATE, or DELETE access to anything in the database. This corresponds to having SELECT privileges on a set of tables – but not all tables – in the database.

Service accounts
A service account is a special type of Google account intended to represent a non-human user that needs to authenticate and be authorized to access data from Google Cloud. Each Spanner service account likely needs to have its own set of privileges on specific tables in the database. For example, consider a ride-sharing application that has service accounts for drivers and passengers. Likely the driver service account needs SELECT privileges on specific columns of the passenger’s profile table (e.g., user’s first name, profile picture, etc.), but should not be allowed to update the passenger’s email address or other personal information.

The basics of fine-grained access control in Spanner
If you’re familiar with role-based access control in other relational databases, you already are familiar with the important concepts of fine-grained access control in Spanner. Let’s review the model for fine-grained access control in Spanner:

Database Privilege. Spanner now supports four types of privileges: SELECT, INSERT, UPDATE, and DELETE. SELECT, INSERT, UPDATE and DELETE privileges can be assigned to tables, and SELECT, INSERT, and UPDATE can be applied to tables or columns.
Database Role. Database roles are collections of privileges. For example, you can have a role called inventory_admin that has SELECT and INSERT privileges on the Inventory_Transactions table and SELECT, INSERT, UPDATE, and DELETE privileges on the Products table.

Because Spanner relies on IAM for identity and access management, you need to assign database roles to the appropriate IAM principals by managing conditional role bindings.

Let’s look at an example. Suppose we want to set up IAM principal user@abc.xyz with fine-grained access to two tables: Inventory_Transactions and Products. To do this, we’ll create a database role called inventory_admin and grant this role to user@abc.xyz.

Step 1: Set up the IAM principal as a Cloud Spanner fine-grained access user
Until today, if you wanted to grant database-level access to an IAM principal, you’d grant them either the roles/spanner.databaseUser role, or some privileges that are bundled in that role. Now, with fine-grained access control, you can instead grant IAM principals the Cloud Spanner Fine-grained Access User role (roles/spanner.fineGrainedAccessUser).

Cloud Spanner Fine-grained Access User allows the user to make API calls to the database, but does not confer any data access privileges other than those conferred to the public role. By default, the public role does not have any privileges, and this role only grants access to make API calls to the database. To access data, a fine grained access user must specify the database role that they want to act as.

Step 2: Create the database role
To create a role, run the standard SQL CREATE ROLE command:
CREATE ROLE inventory_admin;

The newly created database role can be referenced in IAM policies via the resource URI: projects/<project_name>/instances/<instance_name>/databases/<database_name>/databaseRoles/inventory_admin. Later on, we’ll show how to configure an IAM policy that allows a specific IAM principal permission to act as this database role.

Step 3: Assign privileges to the database role
Next, assign the appropriate privileges to this role:

code_block: <ListValue: [StructValue([('code', 'GRANT SELECT, INSERT\r\nON TABLE Inventory_Transactions\r\nTO ROLE inventory_admin;\r\n\r\nGRANT SELECT, INSERT, UPDATE, DELETE\r\nON TABLE Products\r\nTO ROLE inventory_admin;'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f3aa85a2e80>)])]>

While you can run these statement individually, we recommend that you issue Cloud Spanner DDL statements in a single batch:

Step 4: Assign the role to an IAM principal
Finally, to allow user@abc.xyz to act as the database role inventory_admin, grant Cloud Spanner Database Role User to user@abc.xyz with the database role as a condition. To do this, open the database’s IAM Info Panel and add the following conditions using the IAM condition editor:

resource.type == “spanner.googleapis.com/DatabaseRole” &&
resource.name.endsWith(“/inventory_admin”)
You can also add any other conditions to further restrict access to this database role, such as scheduling access by time of day, day of week, or with an expiration date.

Transitioning to fine-grained access control
When you’re transitioning to fine-grained access control, you might want to assign both roles/spanner.databaseUser and roles/spanner.fineGrainedAccessUser to an IAM principal. When you’re ready to switch that IAM principal over to fine-grained permissions, simply revoke the databaseUser role from that IAM principal.

Using the role as an end user
When an end user logs into Spanner, they can access the database using the role they’ve been granted, through the Google Cloud console or gcloud commands. Go, Java, Node.js and Python client libraries are also supported, with support for more client libraries coming soon.

Learn more
With fine-grained access control, you can set up varying degrees of access to your Spanner databases based on the user, their role, or the organization to which they belong. In preview today, fine-grained access control is available to all Spanner customers at no additional charge.

To get started with Spanner, create an instance, try it out with a Spanner Qwiklab, or create a free trial instance
To get started with fine-grained access control in Spanner, check out About fine-grained access control or access it directly from the Write DDL statements in the Google Cloud console
To get started with Spanner, create an instance or try it out for free, or take a Spanner Qwiklab

Announcing policy guardrails for Terraform on Google Cloud CLI preview

Wed, 18 May 2022 16:00:00 +0000

Terraform is a popular open source Infrastructure as Code (IaC) tool today and is used by organizations of all sizes across the world. Whether you use Terraform locally as a developer or as a platform admin managing complex CI/CD pipelines, Terraform makes it easy to deploy infrastructure on Google Cloud.

Today, we are pleased to announce gcloud beta terraform vet, which is a client-side tool, available at no charge which enables policy validation for your infrastructure deployments and existing infrastructure pipelines. With this release, you can now write policies on any resource from Terraform’s google and google-beta providers. If you're already using Terraform Validator on GitHub today, follow the migration instructions to leverage this new capability.

The challenge

Infrastructure automation with Terraform increases agility and reduces errors by automating the deployment of infrastructure and services that are used together to deliver applications.

Businesses implement continuous delivery to develop applications faster and to respond to changes quickly. Changes to infrastructure are common and in many cases occur often. It can become difficult to monitor every change to your infrastructure, especially across multiple business units to help process requests quickly and efficiently in an automated fashion. As you scale Terraform within your organization, there is an increased risk for misconfigurations and human error. Human authored configuration changes can extend infrastructure vulnerability periods which expose organizations to compliance or budgetary risks. Policy guardrails are necessary to allow organizations to move fast at scale, securely, and in a cost effective manner - and the earlier in the development process, the better to avoid problems with audits down the road.

The solution

gcloud beta terraform vet provides guardrails and governance for your Terraform configurations to help reduce misconfigurations of Google Cloud resources that violate any of your organization's policies.

These are some of the benefits of using gcloud beta terraform vet:

Enforce your organization's policy at any stage of application development
Prevent manual errors by automating policy validation
Fail fast with pre-deployment checks

New functionality

In addition to creating CAI based constraints, you can now write policies on any resource from Terraform’s google and google-beta providers. This functionality was added after receiving feedback from our existing users of terraform validator on github. Migrate to gcloud beta terraform vet today to take advantage of this new functionality.

Primary use cases for policy validation

Platform teams can easily add guardrails to infrastructure CI/CD pipelines (between the plan & apply stages) to ensure all requests for infrastructure are validated before deployment to the cloud. This limits platform team involvement by providing failure messages to end users during their pre-deployment checks which tell them which policies they have violated.

Application teams and developers can validate their Terraform configurations against the organization’s central policy library to identify misconfigurations early in the development process. Before submitting to a CI/CD pipeline, you can easily ensure your Terraform configurations are in compliance with your organization’s policies, thus saving time and effort.

Security teams can create a centralized policy library that is used by all teams across the organization to identify and prevent policy violations. Depending on how your organization is structured, the security team (or other trusted teams) can add the necessary policies according to the company’s needs or compliance requirements.

Getting started

The quickstart provides detailed instructions on how to get started. Let’s review the simple high-level process:

1. First, clone the policy library. This contains sample constraint templates and bundles to get started. These constraint templates specify the logic to be used by constraints.

2. Add your constraints to the policies/constraints folder. This represents the policies you want to enforce. For example, the IAM domain restriction constraint ensures all IAM policy members are in the “gserviceaccount.com” domain. See sample constraints for more samples.

code_block: <ListValue: [StructValue([('code', 'apiVersion: constraints.gatekeeper.sh/v1alpha1\r\nkind: GCPIAMAllowedPolicyMemberDomainsConstraintV2\r\nmetadata:\r\n name: service_accounts_only\r\n annotations:\r\n description: Checks that members that have been granted IAM roles belong to allowlisted\r\n domains.\r\nspec:\r\n severity: high\r\n match:\r\n target: # {"$ref":"#/definitions/io.k8s.cli.setters.target"}\r\n - "organizations/**"\r\n parameters:\r\n domains:\r\n - gserviceaccount.com'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f3aa7193670>)])]>

3. Generate a Terraform plan and convert it to JSON format
$ terraform show -json ./test.tfplan > ./tfplan.json

4. Install the gcloud component, terraform-tools
$ gcloud components update
$ gcloud components install terraform-tools

5. Run gcloud beta terraform vet
$ gcloud beta terraform vet tfplan.json --policy-library=.

6. Finally, view the results. If you violated any policy checks, you will see the following outputs.

Pass:

code_block: <ListValue: [StructValue([('code', '[]'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f3aa6dad910>)])]>

Fail: The output is much longer, here is a snippet:

code_block: <ListValue: [StructValue([('code', '[\r\n{\r\n "constraint": \r\n… \r\n\r\n"message": "IAM policy for //cloudresourcemanager.googleapis.com/projects/PROJECT_ID contains member from unexpected domain: user:user@example.com",\r\n…\r\n]'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f3aa6dada90>)])]>

Feedback

We’d love to hear how this feature is working for you and your ideas on improvements we can make.

Implementing HKMA’s Secure Tertiary Data Backup (STDB) on Google Cloud

Tue, 03 May 2022 00:00:00 +0000

In this post, we will discuss how Google Cloud can be used as a backup storage solution to support the objectives of the Secure Tertiary Data Backup (STDB) guideline developed by the Hong Kong Monetary Authority (HKMA) and Hong Kong Association of Banks. We will focus on the storage location and its surrounding security controls. The solution should work with any backup tool that can configure Google Cloud Storage as the data storage destination.

To help satisfy STDB requirements, the data backup destination or storage location should be able to fulfill the following requirements:

Immutable and controlled

1. Data should be immutable by the data writer. For instance, the backup software should not have the ability to update or delete the data once it is written to the storage

Secure and verifiable

2. All data transfer should go through a private and encrypted connection

3. All data at rest must be encrypted and by keys that are managed by the data owner

4. All operations on the backup location, both by the owner and solution provider must be logged and auditable

Air-gapped

5. The backup target is “air-gapped” or disconnected from the data source / on-prem network when backup is not taking place

Limit the network connection by disconnecting the data source and backup storage as much as possible.
Grant read and write-only access rights based on the principle of least privilege and through role AND time-based access control.

6. To minimize the surface of attack, the number of components in the solution should be minimized while multiple layers of access control should be enforced to restrict network connection and resource access control.

High performance

7. Storage location should provide instantaneous (milliseconds) access to the backed up data when needed, enabling rapid recovery of data.

Survivable

8. The backup destination should be highly durable and available.

9. The storage location should have capacity to support a long history of backups to ensure a clean copy is in place. This is especially critical for recovery from ransomware attacks.

10. The solution must be designed for resiliency against attacks.

Optionally, the following requirements can also be considered:

11. All interactions against the backup target should be accessible via API only. In other words, the inability to access the user interface should not interfere with the backup operation and the management of the solution.

Design Details

1. To help ensure all data transfer to and from Google Cloud Storage is private and secured, an Interconnect (recommended for production) or VPN (sufficient for dev or small bandwidth requirements) should be established between on-premise and Google Cloud. IPSec tunnel can optionally be used to encrypt the connectivity.

To support the air-gapped requirement, either disconnect the Interconnect/VPN from the on-premise router or at the Cloud Router, or both. This disconnection can be done programmatically through gcloud CLI or Terraform. Combining it with Cloud Schedule, on-premise cron job, or any scheduler to disconnect the Interconnect at a specified time or on a scheduled and recurring basis.

2. To protect against data exfiltration risks, Google Cloud supports the concepts of VPC Service Control, which allows a security perimeter to be established around a project, network, and/or services so that network communication is constrained within such perimeter. This perimeter provides an additional layer of security protection that complement role based access control provided by Identity and Access Management (IAM), and more importantly, this protection can work with managed services that do not have a fixed IP/port range, which is something that a network firewall can protect with ease.

Using an example in the diagram above, a client or VM may carry sufficient IAM role to access the Cloud Storage, but because the Cloud Storage is protected by VPC Service Control, such access will still be blocked as the is not configured within the security perimeter.

3. To achieve the principle of least privilege and to further segregate access control, two or more IAM roles with fine grained permissions can be defined.

In this case a “Writer” IAM role having storage.objects.create permission would be granted to the backup software, so that it does not have permission to manipulate the bucket and objects already written to Cloud Storage.
To further support the air-gapped requirement, implement time-based controls on the “Writer” IAM role. For example, defining a conditional role binding using date/time attribute results in the write permission being only enabled at a specified time or on a scheduled and recurring basis.
For the break glass scenario, create an “Admin” IAM, say containing storage.objects.delete permission, and manage this IAM role in a PAM system (e.g. CyberArk).
IAM Recommendation - Google Cloud has a built-in IAM policy intelligence that uses machine learning to predict access needs, so administrators may grant the right levels of access to avoid over-granting. The tool shows a list of recommendations, which includes recommending users to revoke unused roles (e.g. revoke a role if it hasn’t been accessed for over N days), or provision restricted roles to replace the overly broad ones.

4. In order for the backup software to copy data into Cloud Storage, it must first have the “Writer” IAM role, and be added into the perimeter by whitelisting its private IP. The data transfer would stay private between on-premise data centers and Google Cloud.

5. Data written to the Cloud Storage is encrypted by default. Also customers can enable Customer Managed Encryption Keys to have more control over the keys used to encrypt data, e.g. Rotating the key, or backing the key with Cloud HSM key. It is actually possible to disable or destroy a key such that Cloud Storage can no longer decrypt the objects, rendering the object unretrievable.

Archive storage class is used to achieve ultra low cost and highly durable storage for the backup objects. Despite being an Archive class storage, it can still support instantaneous retrieval and requires no thawing, allowing instant recovery of data when it is needed.

6. Access and maintenance of the storage is logged and auditable, including operations done by Google. If User Interface access (i.e. Google Cloud Console) is not desired, customers can block access at the corporate proxy but still allow backup process and management of the service to take place programmatically using gcloud CLI or client libraries.

Security Controls

Fundamental security controls:

Data Encryption by default for at rest using customer managed encryption key.
Fine grained access control using IAM supporting time-based control of permissions.
Define Security Perimeter to protect against data exfiltration risks (in addition to IAM).
Private connectivity through a secure/dedicated communication channel with end-to-end network encryption for data transport.
Centralized logging for administrator/user activity monitoring and auditing. Access transparency ensures all operations done by Google are also logged and auditable.
Security Command Centre to help detect misconfigurations, abnormal activity, such as service account abuse and data exfiltration.

Advanced security features:

Context Aware Access for contextual access control (e.g. who, where, device, etc.) and automatically prompt for re-authentication if required.
DLP protection integrated with Cloud Storage, which can automatically detect and flag the existence of PII using machine learning.

Mapping features into STDB goals

Immutable and controlled

Security Perimeter (VPC Service Controls)
VPC Service Controls delivers an extra layer of control with a defense-in-depth approach for multi-tenant services that helps protect service access from both insider and outsider threats. It enforces a security perimeter with VPC Service Controls to isolate resources of multi-tenant Google Cloud services by reducing the risk of data exfiltration or data breach.
Enabling the bucket lock feature can provide immutable storage on Cloud Storage. Once enabled, all objects in the bucket can only be deleted or replaced once their age is greater than the retention period. This feature is recommended to be used in conjunction with Detailed Audit Logging mode when seeking various compliance requirements.

Secure and verifiable

IPsec over Interconnect
It provides customers a managed solution to encrypt their traffic over Interconnect, so that all data transfer go through a private and encrypted connection.
Default Encryption at rest
Cloud Storage encrypts your data on the server side, before it is written to disk, at no additional charge. Customer managed encryption keys (CMEK) and Customer supplied encryption keys (CSEK) are also supported as a Server-side encryption which acts as an additional encryption layer on top of the standard Cloud Storage encryption.
Cloud IAM manages resource permissions by creating granular access control policies to resources based on attributes like device security status, IP address, resource type, and date/time.
Cloud Audit Logs with Cloud Storage help you answer the questions, "Who did what, where, and when?" by generating Admin Activity logs and Data Access logs. With Detailed Audit Logging mode enabled, data access logs will contain detailed request and response information including query parameters, path parameters, and request body parameters.
Access Transparency
Provides near real-time logs when Google administrators access your content based on support tickets. Access Approval allows users to approve/dismiss requests for access by GCP administrators working to support your service.

Enable corporate proxy to block UI/console access to Cloud Storage.
IAM policy intelligence that uses machine learning to predict access needs, so administrators may grant the right levels of access to avoid over-granting.

Data Integrity
Google Cloud Storage uses MD5 hash (or ETag) or CRC32c to validate integrity of the data. In the case of composite objects, CRC32c is used. When supplying an object's expected MD5 or CRC32C hash in an upload request, Cloud Storage will only create the object if the provided hash matches the value Cloud Storage calculates. Likewise, a download integrity check can be performed by hashing downloaded data on the fly and comparing the results to the server-supplied hashes. If gsutil is used for data transfer, the above data integrity checking is handled automatically.

Restoration Validation

A regular data restoration drill is recommended to ensure a successful recovery and a clean backup image is in place. Both restoration drill and actual restore can be performed by running the gsutil command to get the data/files back. Or if any backup software is used, performing a data restore operation using the backup software can also serve the same purpose.

Air-gapped

Use of date/time attributes in Cloud IAM to enforce time-based controls when accessing a given resource. You can grant temporary access to a project that starts and stops at a specified time.
Simple architecture that sits in a Google-privately owned and managed network (without any traffic in the public network) with a full stack of unique security controls including VPC Service Controls, default encryption, Cloud IAM with time-based control, access transparency, etc.

High performance

For the 4 Storage Classes of Google Cloud Storage, all 4 of them (including the lowest cost tier of Archive Storage) have the same SLA with extremely low retrieval latency - time to first byte is typically in tens of milliseconds.

Object Lifecycle Management

To optimize storage costs, it is possible to transition to a lower cost storage class when the backup artifact is no longer current. For instance, using Object Lifecycle Management, users can configure a rule to downgrade the storage class of objects older than 365 days to Coldline Storage.

Survivable

Archive Storage offers ultra low-cost, highly-durable, highly available archival storage. For data accessed less than once a year, Archive is a cost-effective storage option for long-term preservation of data.
Archive Storage Class description

Unlimited storage with no minimum object size.
Worldwide accessibility and worldwide storage locations.
Low latency (time to first byte typically tens of milliseconds).
High durability (99.999999999% annual durability).
Geo-redundancy if the data is stored in a multi-region or dual-region.
Typical monthly availability.
99.95% in multi-regions and dual-regions and 99.9% in regions.
A uniform experience with Cloud Storage features, security, tools, and APIs.

Automatic data risk management for BigQuery using DLP

Thu, 14 Apr 2022 16:00:00 +0000

Protecting sensitive data and preventing unintended data exposure is critical for businesses. However, many organizations lack the tools to stay on top of where sensitive data resides across their enterprise. It’s particularly concerning when sensitive data shows up in unexpected places – for example, in logs that services generate, when customers inadvertently send it in a customer support chat, or when managing unstructured analytical workloads. This is where Automatic Data Loss Prevention (DLP) for BigQuery can help.

Data discovery and classification is often implemented as a manual, on-demand process, and as a result happens less frequently than many organizations would like. With a large amount of data being created on the fly, a more modern, proactive approach is to build discovery and classification into existing data analytics tools. By making it automatic, you can ensure that a key way to surface risk happens continuously - an example of Google Cloud's invisible security strategy. Automatic DLP is a fully-managed service that continuously scans data across your entire organization to give you general awareness of what data you have, and specific visibility into where sensitive data is stored and processed. This awareness is a critical first step in protecting and governing your data and acts as a key control to help improve your security, privacy, and compliance posture.

In October of last year, we announced the public preview for Automatic DLP for BigQuery. Since the announcement, our customers have already scanned and processed both structured and unstructured BigQuery data at multi-petabyte scale to identify where sensitive data resides and gain visibility into their data risk. That’s why we are happy to announce that Automatic DLP is now Generally Available. As part of the release we’ve also added several new features to make it even easier to understand your data and to make use of the insights in more Cloud workflows. These features include:

Premade Data Studio dashboards to give you more advanced summary, reporting, and investigation tools that you can customize to your business needs.

Easy to understand dashboards give a quick overview of data in BQ

Finer grained controls to adjust frequency and conditions for when data is profiled or reprofiled, including the ability to enable certain subsets of your data to be scanned more frequently, less frequently, or skipped from profiling.

Granular settings for how often data is scanned

Automatic sync of DLP profiler insights and risk scores for each table into Chronicle, our Security Analytics platform. We aim to build synergy across our security portfolio, and with this integration we allow analysts using Chronicle to gain immediate insight into if the BQ data involved in a potential incident is of high value or not. This can significantly help to enhance threat detections, prioritizations, and security investigations. For example, if Chronicle detects several attacks, knowing if one is targeting highly sensitive data will help you prioritize, investigate, and remediate the most urgent threats first.

Deep native integration into Chronicle helps speed up detection and response

Managing data risk with data classification

Examples of sensitive data elements that typically need special attention are credit cards, medical information, Social Security numbers, government issued IDs, addresses, full names, and account credentials. Automatic DLP leverages machine learning and provides more than 150 predefined detectors to help discover, classify, and govern this sensitive data, allowing you to make sure the right protections are in place.

Once you have visibility into your sensitive data, there are many options to help remediate issues or reduce your overall data risk. For example, you can use IAM to restrict access to datasets or tables or leverage BigQuery Policy Tags to set fine-grained access policies at the column level. Our Cloud DLP platform also provides a set of tools to run on-demand deep and exhaustive inspections of data or can help you obfuscate, mask, or tokenize data to reduce overall data risk. This capability is particularly important if you’re using data for analytics and machine learning, since that sensitive data must be handled appropriately to ensure your users’ privacy and compliance with privacy regulations.

How to get started

Automatic DLP can be turned on for your entire organization, selected organization folders, or individual projects. To learn more about these new capabilities or to get started today, open the Cloud DLP page in the Cloud Console and check out our documentation.

Verifying the security and privacy controls of Google Cloud: 2021 CCAG customer pooled audit

Tue, 05 Apr 2022 16:00:00 +0000

Earning the role as our customers’ most trusted cloud requires commitment to ongoing transparency, collaboration and assurance. Our products regularly undergo independent verification, achieving certifications or attestations of compliance against global regulatory requirements, frameworks, and guidelines. At Google Cloud we work closely with our customers, their regulators, and appointed independent auditors who want to verify the security and privacy of our platform. One example of how the Google Cybersecurity Action Team supports customers’ risk management efforts is our annual audit with the Collaborative Cloud Audit Group (CCAG). In 2020, faced with the global COVID-19 pandemic and the demands for teleworking, Google Cloud swiftly enabled customer audits in completely remote settings. 2021 brought additional challenges for organizations globally with an increased number of cybersecurity threats, data breaches, software supply chain attacks, as well as rapidly evolving data transfer and privacy requirements. These heightened challenges emphasized the importance for customers to rigorously assess the controls that enable them to independently operate and confidently protect their data and applications in the cloud. So, not surprisingly, the scope of the 2021 CCAG pooled audit included, among others, the measures ensuring Google keeps customer data secure and private, effectively manages threats and vulnerabilities, and ensures transparency and accountability across the full software supply chain.

CCAG is an initiative of 50 leading European financial institutions and insurance companies who depend on cloud infrastructure and technologies to deliver innovative solutions and experiences for their customers. For the third year in a row, the CCAG audit of Google Cloud enabled the group members to manage the risks associated with outsourcing material workloads and satisfy strict national and EU regulatory obligations.

Hamidou Dia, VP for Solutions Engineering in Google Cloud, spoke about how critical verifiable transparency is to earning and retaining customer trust:

“Our customers recognize the need to secure their global workforce, applications, and data across all platforms, which includes understanding cloud service providers’ relevant policies and controls,” said Dia. “Successful enterprise trust partnerships require transparency, along with access to information, premises and experts, all of which help our customers rapidly complete their risk management and due diligence.”

The pooled audit executed by CCAG is a great example of customers working together to efficiently deploy their resources and gain detailed information and assurances of Google Cloud’s trust posture. The annual engagement lasts approximately six months and is a comprehensive assessment of the design and the effectiveness of Google Cloud security and privacy controls.

“This year we primarily focused on testing Google’s infrastructure security, cryptographic and data privacy controls, and supply chain management. An audit of the scale performed by CCAG requires extensive preparation and resources on both sides. To satisfy the participating members’ individual risk assessments, as well as meet regulatory compliance requirements, we reviewed a large number of policies, processes, technical documents, and test samples”, said Christina Hepp, Divisional Head IT, Operations & Sourcing Group Audit, Commerzbank. “Google Cloud teams pulled together subject matter experts across the organization and secured leadership support to help us successfully complete the audit.”

Verifying the security and privacy controls of the platform through pooled audits is one way Google Cloud maintains the commitment to being the industry’s most trusted cloud. We continue to partner with customers to meet their evolving regulatory compliance requirements. To learn more about Google Cloud Trust & Compliance, visit our Compliance resource center.

An update on Google Cloud’s commitments to E.U. businesses in light of the new E.U.-U.S. data transfer framework

Mon, 28 Mar 2022 23:00:00 +0000

Last week, the European Commission and U.S. Government agreed on a new E.U.-U.S. data transfer framework. Earlier today, Google shared that it welcomes these efforts by the U.S. government to enhance privacy protections for E.U. data and facilitate trusted transatlantic data flows. For our Google Cloud customers, we intend to make the protections offered by this new framework available once it is implemented.

Last year, we reaffirmed our commitment to E.U. businesses after the European Data Protection Board (EDPB) issued its Recommendations on Supplementary Measures, following Europe’s top court ruling invalidating the E.U.-U.S. Privacy Shield Framework and upholding the E.U. Standard Contractual Clauses (SCCs).

Since then, we've continued to help our customers meet stringent data protection requirements by offering industry-leading technical controls, contractual commitments, and risk assessment resources. We've also continued our advocacy to create more legal certainty around transatlantic data flows.

Today, we would like to provide an update to our customers on this work.

A customer-controlled cloud

Google Cloud¹ continues to be a leading provider of technical and security controls to help meet customers’ data protection requirements, as well as their increasing data sovereignty expectations.

We are committed to building our Cloud on Europe's terms, including by offering customer-managed encryption and data localization for a growing list of key products and collaborating with local partners to provide the highest levels of sovereignty, all while enabling the next wave of growth and transformation for Europe’s businesses and organizations.

Google Cloud Platform

We recently announced the general availability of Assured Workloads for the E.U. This product helps Google Cloud Platform (GCP) customers protect their data by allowing them to:

Store their data in their choice of E.U. Google Cloud region(s)
Ensure that only E.U. persons – located in the E.U. – have access to the data and provide customer support
Deploy cryptographic control for data access, including customer-managed encryption keys

Cloud External Key Manager (EKM) enables customers to encrypt data in a variety of services with keys that are stored and managed in a third-party key management system deployed outside of Google’s infrastructure. Google Cloud continues to be the only cloud provider to enable customers to store and manage encryption keys for cloud-resident data outside the provider's infrastructure with customer’s control over decryption based on specific justifications, including government access requests.

Key Access Justifications greatly advances the control that GCP customers have over their data by giving customers a justification every time their externally hosted keys have to be used to decrypt data. Signed Access Approval (SAA) adds a layer of extra assurance that requires explicit customer consent for any administrative access to customer data or configurations.

Google Cloud’s Confidential Computing portfolio is a breakthrough technology that allows customers to encrypt their most sensitive data in the cloud while in-use. Ubiquitous Data Encryption further extends data protection by providing cryptographic protection for this data at-rest, in-transit, and in-use. The keys used to encrypt customer data outside of GCP using Cloud EKM are securely shared with applications operating within Confidential environments.

Google Workspace

Our Google Workspace (including Workspace for Education) customers can choose to store their covered data in Europe. Additionally, with Client-Side Encryption, we offer customers direct control of encryption keys and the identity service they choose to access those keys. With Client-Side Encryption, customer data is indecipherable to Google, while users can continue to take advantage of Google’s native web-based collaboration, access content on mobile devices, and share encrypted files externally. Client-Side Encryption is currently available in Public Beta for Google Drive, Docs, Sheets, and Slides, and we plan to extend it to Gmail, Calendar and Meet. Additionally, customers can also benefit by choosing third party solutions that offer similar encryption capabilities with select Google Workspace services.

Legal Basis for International Data Transfers

We updated our data processing terms for GCP and Google Workspace and Cloud Identity to reflect various modules of the new E.U. Standard Contractual Clauses (SCCs) approved by the European Commission on June 4, 2021, as well as separate U.K. SCCs.

Google Cloud plans to adopt the new E.U.-U.S. data transfer framework and offer it as a transfer solution to our cloud customers, as further detailed in our data processing terms.

Advocacy and Additional Helpful Resources

We have adopted the Trusted Cloud Principles with industry peers to demonstrate our commitments to protect the rights of our Google Cloud customers. We will continue to support the ongoing work of the Organisation for Economic Co-operation and Development on government access to data and the negotiation of CLOUD Act Agreements — including between the U.S. and E.U. — as vehicles for surveillance reform.

We will continue to publish additional materials on our Cloud Privacy Resource Center, such as our whitepaper on safeguards for international data transfers with Google Cloud.

Millions of organizations with users in Europe rely on our cloud services to run their businesses every day, and we remain steadfastly committed to helping them meet their regulatory requirements by maintaining a diverse set of compliance tools.

^{1. Google Cloud: Google Workspace (including Google Workspace for Education) and Google Cloud Platform (GCP)}

How technology is forging the foundation for more efficient, transparent and secure financial markets

Wed, 23 Mar 2022 13:00:00 +0000

Financial markets were among the first to adopt new technologies, and that has certainly been true of the derivatives markets, which were early adopters of electronic trading. Going forward, new capabilities will transform the way industry participants communicate, analyze, and trade.

I sat down with Google Cloud’s Phil Moyer and former SEC Commissioner, Troy Paredes, for a fireside chat at FIA Boca 2022 to discuss the future of markets and policy, the new technologies that are already paving the way for greater speed and transparency, and how cloud can help promote greater resiliency, performance, and security to enable the long-term vision for the market. The following is a summary of our discussion.

The current state of cloud technology

When it comes to technology adoption, we’re seeing the market and participants adopt cloud technologies, and increasingly, machine learning (ML) on a wider scale. Cloud technology allows for easier, faster, and much more secure experimentation with large datasets and ML.

A recent Google sponsored study by Coalition Greenwich (September, 2021) showed that more than 93% of trading systems, exchanges, and data providers are in some way providing services on the cloud. The same study, revealed that about 72% of the financial industry across the buy side and sell side, intend to consume public cloud-data based market data within the next 12 months.

Data-driven decision-making and risk management have always been, and continue to remain, the cornerstones of the financial markets. Over time, technology innovation has facilitated access to better insights from data, and therefore, better decision-making and the ability to manage risk. That expectation is now mainstream, and will continue to grow in sophistication.

The multi-phased technology trajectory

The movement of exchanges to the cloud will occur in a “crawl-walk-run” fashion, with low-hanging fruits the first to be picked in the near term while bigger, paradigmatic changes will occur over the medium and long term. Some organizations are starting all three stages simultaneously, understanding that each will move at an independent cadence.

The “crawl” phase is one in which foundations are built, starting with organizations moving data to the cloud and experimenting with some degree of analytics. It’s one of the most important phases because it’s where the opportunity to increase transparency and risk management takes shape.

In moving to the cloud, the infrastructure – which in the past relied on a combination of people, processes, and some technology – becomes the code that runs applications. This early phase is key to empowering organizations to shift to a cloud-based, agile-first operating model that makes it easier and more seamless to launch new products in the future, including by freeing up people and resources from IT management to more mission-focused work.

Establishing the cloud operating model simplifies the “walk” and “run” phases where compliance is more automated, latency-sensitive applications are more readily available, and the next generation of exchanges, market participants, and regulators is better prepared to meet future challenges.

The “walk” phase is where much of the innovation happens. Exchanges are making significant progress in leveraging foundational data decisions in the “crawl” phase and innovations in the cloud to improve settlement, clearing, risk management, collateral management, and compliance, and launch new products.

And finally, the “run” phase is where organizations will start to move the latency-sensitive markets to the cloud, as the markets increasingly will demand low-latency and high performance along with transparency and analytics to solve historical obstacles to market access.

Opportunities for both regulators and market participants

Any time significant technological change takes place, regulators explore its implications, particularly with respect to their ability to meet their regulatory objectives.

Increasingly, we are seeing technological change driving more opportunities for regulators and market participants alike. Such changes may also allow better protection of the marketplace, with greater integrity and transparency.

Over time, regulatory regimes – rules, regulations, statutes, interpretations, and guidance – will also adjust to new technologies, both benefiting the marketplace and advancing regulatory goals.

As one example, the cloud is increasing the ability to meet compliance obligations by allowing compliance to be built into transactions. Moreover, predicated on the vision of real-time regulatory reporting, and given the pace of technological change in the marketplace over the last several years, various regulators have been using more advanced analytics. This trend will continue to help them more effectively and efficiently meet their objectives, and monitor and meet the expectations they have for the entire market.

Machine learning’s role in the financial markets

Google Cloud’s head of AI and Industry Solutions, Andrew Moore, said that ML will be doing three key things for us in the next 10 years: giving us meaning, providing concierge services, and serving as a guardian. Extracting information that is critical to investor decision-making can be extremely important. With more data than ever, ML can increase the ability to process it while also becoming more accessible in the cloud and better supporting regulatory objectives.

The technology will likely manifest in trading and anti-money laundering activities as they relate market functions, as well as managing a wide variety of risks – supporting the interests of both investors and regulators in terms of decision-making, surveillance, and protections.

Rather than taking individuals out of the equation, the digitization of markets, assets, and guard rails combined with ML will allow people to focus their expertise in different ways to achieve key objectives.

Building the market foundation for the future

The goals of operational resiliency, security, and privacy will continue to be critical for building the market foundation for both participants and regulators. While technology promises to create advantages in concrete, tangible ways, it will be important to scrutinize potential risks and concerns.

Priority one for technology providers is to build an environment of trustless security, including encryption at motion and encryption at rest, ensuring that markets are operationally resilient while instilling confidence for any exchange that runs on top of that infrastructure. Multicloud architectures and approaches are likely also to be part of the solution for operational resilience.

Throughout time, liquidity has been the outcome of improved access, transparency, and security. Technology providers are responding by sharing both the responsibility for, and fate of, the markets of the future to build an efficient, faster, and more transparent and secure financial industry.

You can learn more about our approach in our newest white paper, Building the financial markets foundation for the future.

Accelerating Government Compliance with Google Cloud’s Professional Service Organization

Mon, 21 Mar 2022 17:00:00 +0000

Did you know that by 2025, enterprise IT spending on public cloud computing will overtake traditional IT spending? In fact, 51% of IT spend in application software, infrastructure software, business process services, and system infrastructure will transition to the public cloud, compared to 41% in 2022_1.. As enterprises continue to rapidly shift to the cloud, government agencies must prioritize and accelerate security and compliance implementation.

In May 2021, the White House issued an Executive Order requiring US Federal agencies to accelerate cloud adoption, embrace security best practices, develop plans to implement Zero Trust architectures, and map implementation frameworks to FedRAMP. The Administration’s focus on secure cloud adoption marks a critical shift to prioritizing cybersecurity at scale. Google Cloud’s Public Sector Professional Services Organization (PSO) has committed to helping customers meet security and compliance requirements in the cloud through specialized consulting engagements.

Accelerating Authority to Operate (ATO)

The Federal Risk and Authorization Management Program (FedRAMP) was established in 2011 as a government-wide program that promotes the adoption of secure cloud services across the federal government. FedRAMP provides a standardized approach to security and risk assessment for cloud technologies and federal agencies. US Federal agencies are required to utilize and implement FedRAMP cloud service offerings as part of the “Cloud First” federal cloud computing strategy.

While Google Cloud provides a FedRAMP-authorized cloud services platform and a robust catalog of FedRAMP-approved products and services (92 services and counting), customers are still tasked with achieving Agency ATO for the products and services they use, and Google Cloud provides many resources to assist customers with this journey. Google Cloud’s FedRAMP package can be accessed by completing the FedRAMP Package Access Request Form and submitting it to info@fedramp.gov. Additionally, customers can use Google’s NIST 800-53 ATO Accelerator as a starting point for documenting control implementation. Finally, Google Cloud’s Public Sector PSO offers the following strategic consulting engagements to help customers streamline the Agency ATO process.

Cloud Discover: FedRAMP is a six-week interactive workshop to support customers that are just getting started with the ATO process on Google Cloud. Customers are educated on FedRAMP fundamentals, Google’s security and compliance posture, and how to approach ATO on Google Cloud. Through deep-dive interviews and design sessions, PSO helps customers craft an actionable ATO plan, assess FedRAMP readiness, and develop a conceptual ATO boundary. This engagement helps organizations establish a clear understanding and roadmap for FedRAMP ATO on Google Cloud.

FedRAMP Security Review is a ten to twelve week engagement that aids customers in FedRAMP operational readiness. PSO consultants perform detailed FedRAMP architecture reviews to identify potential gaps in NIST 800-53 security control implementation and Google Cloud secure architecture best practices. Findings from the security reviews are shared with the customer along with configuration guidance and recommendations. This engagement helps organizations prepare for the third-party or independent security assessment that is required for FedRAMP ATO.

Cloud Deploy: FedRAMP is a multi-month engagement designed to help customers document the details of their FedRAMP System Security Plan (SSP) and corresponding NIST 800-53 security controls, in preparation for Agency ATO on Google Cloud at FedRAMP Low, Moderate, or High. PSO collaborates with customers to develop a detailed technical infrastructure design document and security control matrix capturing evidence of the FedRAMP system architecture, security control implementation, data flows and system components. PSO can also partner with a third-party assessment organization (3PAO) or an independent assessor (IA) to support customer efforts for FedRAMP security assessment. This engagement helps customer system owners prepare for Agency ATO assessment and package submission.

Developing a Zero Trust Strategy

In addition to providing FedRAMP enablement, Public Sector PSO has partnered with the Google Cloud Chief Information Security Officer (CISO) team to assist organizations with developing a zero trust architecture and strategy.

Zero Trust Foundations is a seven-week engagement co-delivered by Google Cloud’s CISO and PSO teams. CISO and PSO educate customers on zero trust fundamentals, Google’s journey to zero trust through BeyondCorp, and defense in depth best practices. The CISO team walks customers through a Zero Trust Assessment (ZTA) to understand the organization’s current security posture and maturity. Insights from the ZTA enable the CISO team to work with the customer to identify an ideal first-mover workload for zero trust adoption. Following the CISO ZTA, PSO facilitates a deep-dive Zero Trust Workshop (ZTW), collaborating with key customer stakeholders to develop a NIST 800-207 aligned, cloud-agnostic zero trust architecture for the identified first-mover workload. The zero trust architecture is part of a comprehensive zero trust strategy deliverable that is based on focus areas called out in the Office of Management and Budget (OMB) Federal Zero Trust Strategy released January 2022.

Scaling Secure Cloud Adoption with PSO

Public Sector PSO enables customer success by sharing our technical expertise, providing cloud strategy, implementation guidance, training and enablement using our proven methodology. As enterprise IT, operations, and organizational models continue to evolve, our goal is to help government agencies accelerate their security and compliance journeys in the cloud. To learn more about the work we are doing with the federal government, visit cloud.google.com/solutions/federal-government.

¹ Gartner Says More Than Half of Enterprise IT Spending in Key Market Segments Will Shift to the Cloud by 2025

Accelerate Google Cloud vendor due diligence by leveraging third party risk management providers

Tue, 08 Mar 2022 17:00:00 +0000

As organizations accelerate adoption of cloud services to deliver innovative solutions and experiences for their customers, risk and compliance teams are adjusting their due diligence programs to better understand and manage the risks associated with outsourcing of business critical workloads. At the core of these efforts is protecting sensitive data and applications in accordance with internal policies and best practices, while maintaining compliance with complex global regulatory requirements, frameworks, and guidelines. Cloud auditability, control monitoring, and continuous cloud risk assessment are prerequisites to building trust and regulator confidence in cloud services that underpin core business processes and apps. As a result, organizations are increasingly dedicating resources to perform comprehensive assessments of relevant cloud policies, processes, and technical implementations, and these resources are always looking for ways to increase breadth and depth of their provider risk assessments while optimizing the costs to the organization.

Here at Google Cloud we are committed to being the industry’s most trusted cloud and our Google Cybersecurity Action Team works closely with customers to help them meet their due diligence, risk management, and regulatory compliance needs. One way we help our customers scale and accelerate their cloud assessments is by collaborating with third party risk management (TPRM) providers. These organizations provide independent due diligence services and platforms to help automate vendor risk management based on the data they collect and provide. We enable the TPRM assessors to examine the controls present in our infrastructure and operations. Based on their observations and assessments, TPRM providers develop independent and unbiased audit reports that can be shared directly with customers.

Some of the benefits of TPRM solutions for Google Cloud customers include:

Comprehensive and regular assessments: TPRM providers perform periodic, multi-tier, multidimensional assessments of Google Cloud’s platform and services on a regular schedule. As part of these assessments, TPRM providers inspect hundreds of security, privacy, business continuity, and operational resiliency controls aligned with industry standards and regulations such as NIST SP 800-53, NIST CSF, ISO 27001, PCI-DSS, HIPAA, CMMC, SOC2, CSA STAR, etc. The information provided in these in-depth assessments help support Google Cloud customers’ own assessment processes that underpin their efforts to meet complex regulatory compliance requirements when managing data and applications in the cloud.

Efficient use of audit resources: TPRM providers facilitate an exchange of risk assessments between Google Cloud and our customers, customizing the reports to the customer's compliance landscape. This helps streamline the vendor risk assessment process and decrease the regular effort required by both the customer as well as Google. By leveraging the broad scope and extensive fieldwork of these TPRM assessments, Google Cloud customers can accelerate their vendor due diligence and overall risk management processes.

Access to independent cloud audit expertise: TPRM solutions provide customers with audit assessments and reports that are fully independent of Google Cloud. Additionally, customers have the opportunity to discuss their cloud controls, risk posture, and audit practice with a third party, experienced in auditing public cloud providers.

Many Google Cloud customers already leverage the TPRM relationships we’ve established. Today, we work with industry leading TPRM providers such as CyberGRX, TruSight, and KY3P to deliver high-quality risk assessments for our customers globally. We are committed to continue to find effective, efficient solutions that can help customers meet customer risk management and regulatory compliance requirements. To learn more about Google Cloud Trust & Compliance, visit our Compliance resource center.

Strengthening our European data sovereignty offerings with Assured Workloads for EU

Compliance Engineering - Continuous Compliance GCP case studies

Wed, 15 Dec 2021 17:00:00 +0000

Our previous article provided tools and techniques to transform your productionalization process and make it ready for Cloud workloads. In this post, we will cover technical examples of GCP controls and how it can help your organization maintain your security and compliance posture in GCP.

In comparison to on-prem infrastructure, GCP is a highly integrated environment and provides out of the box capabilities to evidence a large variety of controls. The following cornerstones build the foundation of an effective control attestation:

Inventory Management - On-prem workloads frequently have discovery tools installed to understand what infrastructure components are actually deployed in the corporate IT environment. In GCP, every component has to be explicitly declared for it to exist. This accurate and real-time inventory is the basis for the below case studies to reach continuous compliance.
Infrastructure as Code - All deployments and configuration in GCP should be implemented in machine readable instructions (such as Terraform) and as part of a CI/CD pipeline staging to higher level environments. The programmatic definition of infrastructure resources allows for efficient checking of the security posture before the deployment takes place (such as a misconfigured Google Cloud Storage bucket that would be exposed to the public Internet).
Compliance as Code - The same is true for the implementation of policies. Programmatic definition, “Compliance as Code” should be used to automate the evidence-gathering and compliance-checking during the lifetime of the workload. Google Cloud’ Risk and Compliance as Code solution can help implement such a process based on best practices.

Now let’s take a few controls out from the Cloud Security Alliance Cloud Controls Matrix (CSA CCM) and show how GCP helps you to fulfill them.

Case Study - Log Key Lifecycle Management events

Let’s have a look at a specific control from CSA CCM regarding log key lifecycle management:

LOG-11 - Logging & Monitoring Transaction/Activity Logging - Log and monitor key lifecycle management events to enable auditing and reporting on usage of cryptographic keys.

Cloud Key Management Service (KMS) in combination with Cloud Audit Logs records customer activities on the Key object, such as creating or destroying a key. Customers can define the retention period of the logs as well as access permissions. A log message would look like the following:

code_block: <ListValue: [StructValue([('code', 'gcloud logging read "logName : projects/my-project/logs/cloudaudit.googleapis.com" --project=my-project\r\n\r\n[...]\r\n permission: cloudkms.cryptoKeys.create\r\n resource: projects/my-project/locations/us-central1/keyRings/sample-keyring\r\n resourceAttributes:\r\n name: projects/my-project/locations/us-central1/keyRings/sample-keyring/cryptoKeys/key-us\r\n service: google.cloud.kms\r\n type: google.cloud.kms.CryptoKey\r\n methodName: CreateCryptoKey\r\n[...]'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f3aa85614f0>)])]>

Figure 1 - Logging of key creation

In order to actively monitor these activities, a counter log-based metric in Operations Suite has to be created for protoPayload.methodName="CreateCryptoKey" . This log-based metric can then be used to create an alarm for each event, or trigger a notification for when a certain threshold is reached. Cloud Monitoring will display an incident notification and have visualizations ready to be inspected.

Case Study - Change restoration

In the traditional IT world, production roll-outs are often staged through the environments guarded by a strict change management process and subject to change approval boards. In the cloud world, equivalent checks should take place but can be accomplished end-to-end much more quickly. As mentioned above, there should be limited human interaction in the production environment. All application and infrastructure deployments should be following the infrastructure as code pattern and leverage automation technologies. Repeatable automated patterns will simplify operations and enable compliance verification at scale. Let’s look at the following example out of the CSA CCM control set:

CCC-09 Change Control & Configuration Management - Change Restoration - Define and implement a process to proactively roll back changes to a previous known good state in case of errors or security concerns.

By describing the state of the infrastructure configuration in Terraform, each change can easily be rolled-out and rolled-back without missing out a step or having a non-reversible change. The control owners have assurance that the changes are automated and fully reversible to a known working state.

As a best practice, we recommend storing the Infrastructure as Code patterns along the application project source code in the version control system, and staging it through the environments by leveraging a CI/CD pipeline.

Case Study - Safeguard Logs Integrity

Extensive logging capabilities of GCP such as Cloud Audit Logs and Access Transparency take a record of activities happening in the environment. Especially in regulated industries, these logs have to be kept for an extended period while ensuring their immutability and integrity. This requirement is reflected in the following CSA CCM:

IAM-12 Identity & Access Management - Safeguard Logs Integrity - Define, implement and evaluate processes, procedures and technical measures to ensure the logging infrastructure is read-only for all with write access, including privileged access roles, and that the ability to disable it is controlled through a procedure that ensures the segregation of duties and break glass procedures.

Leveraging GCP Log Sinks, log entries can be exported into different supported destinations, including Google Cloud Storage. Log entries will be stored as JSON files on a GCS bucket. GCS buckets support data retention policies, which govern how long objects in the bucket must be retained. The “Bucket Lock” feature lets you set a permanent non-reversible configuration of the data retention policy on the corresponding GCS bucket.

Example Terraform for locking a bucket and keeping the files for 2 days:

code_block: <ListValue: [StructValue([('code', 'resource "google_storage_bucket" "my-log-bucket" {\r\n name = "my-log-bucket-8225"\r\n location = "EU"\r\n force_destroy = false\r\n\r\n retention_policy {\r\n retention_period = 172800\r\n is_locked = true\r\n }\r\n}'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f3aa8561190>)])]>

Figure 2 - Bucket Locking mechanism

The Bucket Lock is a non-reversible activity. As outlined in the above example, files stored within that GCS bucket will be retained for the defined period, including roles with privileged access.

code_block: <ListValue: [StructValue([('code', "$ gsutil retention get gs://my-log-bucket-8225\r\n Retention Policy (LOCKED):\r\n Duration: 2 Day(s)\r\n Effective Time: Tue, 30 Nov 2021 13:17:14 GMT\r\n\r\n$ gsutil retention set 1d gs://my-log-bucket-8225\r\nSetting Retention Policy on gs://my-log-bucket-8225/...\r\nAccessDeniedException: 403 Cannot reduce retention duration of a locked Retention Policy for bucket 'my-log-bucket-8225'.\r\n\r\ngsutil rm gs://my-log-bucket-8225/README.txt\r\nRemoving gs://my-log-bucket-8225/README.txt...\r\nAccessDeniedException: 403 Object 'my-log-bucket-8225/README.txt' is subject to bucket's retention policy and cannot be deleted, overwritten or archived until 2021-12-02T05:24:02.18624-08:00"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7f3aa8561bb0>)])]>

Figure 3 - Bucket Locking effects

FSI transformation

Automation has been a key driver for accelerating transformation at Deutsche Bank:

With the last few controls being transformed to ‘cloud ready’ and the initial set of applications passing through the controls, control automation is now at the center of our focus. As we progressively learn, utilize, and gain confidence in the real power of the embedded control framework of Google Cloud, we closely work with the control owners to leverage Google Cloud native capabilities. The success we see in automation is now driving the momentum in the on-prem world as well, therefore the hybrid model for automating controls is the next challenge for us in the upcoming months. Greta Binder, Operational Readiness Product Owner - Vice President at Deutsche Bank

The integrated environment of GCP provides foundational capabilities (such as real-time inventory) that can significantly reduce the burden for control owners as they work to establish and maintain their security and compliance posture. The case studies give examples as to how GCP customers can move into continuous compliance, encompassing real-time attestation and notification (something which could occur in case of a misconfiguration, for example). The more familiar control owners become with the GCP capabilities the more confident they feel to automate their controls.

Building on the control automation examples we covered, potential next steps could be embedding the controls into policies within GKE Policy Controller, and GKE Config Connector seamlessly logging into Cloud Operations Suite, as well as Security Command Center. Read more about this topic as part of our recently released solution Modernizing Compliance: Introducing Risk and Compliance as Code.

Software-Defined community cloud - a new way to “Government Cloud”

Compliance

Expanding GKE posture: Policy Controller violations now in Security Command Center

Integrating Policy Controller with SCC

Get started with Policy Controller and SCC integration

Policy Controller dashboard: Now available for all Anthos and GKE environments

The Policy Controller dashboard

Get started today

Harden your Kubernetes clusters and monitor workload compliance at scale with new PCI DSS policy bundle

The PCI DSS v3.2.1 Policy bundle

Using the PCI DSS v3.2.1 Policy bundle

Viewing the PCI DSS Policy bundle violations on Policy Dashboard

Monitoring the cluster(s) for PCI DSS Policy Bundle violations

Conclusion

Get started today

Apply policy bundles and monitor policy compliance at scale for Kubernetes clusters

Introducing Policy Bundles

Get started today

CISO Survival Guide: Vital questions to help guide transformation success

To start the conversation, begin by asking:

CISOs should ask business leaders:

Business leaders should ask CISOs and the security team:

CISOs and business leaders should ask:

CISOs and risk teams should ask:

Business and technical teams should ask:

Practical advice and the realities of operational transformation

CISO Survival Guide: How financial services organizations can more securely move to the cloud

Detect spoofing exceptions in financial markets with Google Cloud and GTS

Regulatory Surveillance of Trading Activity with Google Cloud

Detecting flashing exceptions

Datasets

Flashing detection

Modernizing this solution using SQL Analytics on Google Cloud

Solution overview

Give it a try

Building real-time market data front-ends with websockets and Google Cloud

CISO Survival Guide: How financial services organizations can more securely move to the cloud

First come the questions, so many questions

Teach your organization to think cloud

The right questions can drive security changes

How CISOs need to adapt their mental models for cloud security

View policy enforcement metrics for ACM Policy Controller

Policy Controller Metrics

Exporting the metrics

Viewing the metrics

Pricing

Alerting on the metrics

Third Party integration

Next steps

Introducing fine-grained access control for Cloud Spanner: A new way to protect your data in Spanner

Announcing policy guardrails for Terraform on Google Cloud CLI preview

The challenge

The solution

New functionality

Primary use cases for policy validation

Getting started

Feedback

Ensuring scale and compliance of your Terraform deployment with Cloud Build

Implementing HKMA’s Secure Tertiary Data Backup (STDB) on Google Cloud

Design Details

Security Controls

Mapping features into STDB goals

Put your archive data on ice with new storage offering

Automatic data risk management for BigQuery using DLP

Managing data risk with data classification

How to get started

Cloud Data Loss Prevention is now automatic!

Verifying the security and privacy controls of Google Cloud: 2021 CCAG customer pooled audit

Earning customer trust through a pandemic: delivering our 2020 CCAG pooled audit

An update on Google Cloud’s commitments to E.U. businesses in light of the new E.U.-U.S. data transfer framework

A customer-controlled cloud

Google Cloud Platform

Google Workspace

Legal Basis for International Data Transfers

Advocacy and Additional Helpful Resources

How technology is forging the foundation for more efficient, transparent and secure financial markets

The current state of cloud technology

The multi-phased technology trajectory

Opportunities for both regulators and market participants

Machine learning’s role in the financial markets

Building the market foundation for the future