Google Cloud's unflagged feature crashed 70+ services for 7 hours

On 12 June 2025, Google Cloud's core API policy-check system, Service Control, crashed into a loop across every region at once. More than 70 Google Cloud services malfunctioned globally, and downstream customers including Cloudflare, OpenAI, Shopify, GitHub and GitLab were disrupted. Gmail, Calendar, Drive and Meet also suffered.

Google's own incident report traced the failure to a decision made two weeks earlier. On 29 May 2025, a new feature for additional quota policy checks was added to Service Control. It rolled out region by region, but the failing code path was never exercised because it required a policy change to trigger — and the change had no error handling and was not protected by a feature flag.

On 12 June, an unrelated policy change inserted into the pooled tables included blank fields. When the new quota-check code ran against them, it hit a null pointer and crashed into a restart loop everywhere at once. Engineers found the cause in ten minutes, but recovery dragged on; the largest region, us-central1, overloaded its own database as tasks restarted and took 2 hours 40 minutes more to recover.

Google Cloud CEO Thomas Kurian apologised on X, and the company said it would re-architect so a single system failure cannot take down operations, audit all systems, and add feature flags to catch this class of change in staging.