Salesforce: one unscoped database script opened all Pardot data and darkened 105 instances

On 17 May 2019, Salesforce ran a database script tied to its Pardot marketing product that "resulted in granting users broader data access than intended": current and former Pardot users could read and edit everything in their orgs, ignoring permission settings. To contain the exposure, Salesforce blocked access to every instance containing an affected customer - 105 instances in all - taking down tenants that had nothing to do with Pardot.

The full disruption lasted 15 hours 8 minutes, ending at 0104 PDT on 18 May - and a new disruption began at the same minute. To make instances safe to reopen, Salesforce wiped permissions for affected users and restored only administrator accounts, telling customers to re-assign user permissions by hand. Over the weekend it built a script to restore permissions from backups, but by Sunday it reported an 89% success rate, so roughly one organisation in ten still had user rights wiped - and in some runs the script re-granted the broad access it was meant to remove.

Repairs ran into the following week: by Monday 20 May thousands of tickets were open, sandbox instances were still down, and SVP of engineering Anmol Bhasin apologised on customer calls, saying the first fix had "not been successful" and that all available engineering resources were on the problem. The fix became a second, longer outage than the original break.

105 instances offline; about 15 hours of full outage, then days of read-only cleanup; the weekend restore script failed for roughly 1 in 10 orgs

The change ran far beyond its intended scope: a permission fix for a subset of Pardot orgs hit the whole platform, and no guard caught the widening before users saw data they should never have seen.

The only safe containment was to take whole instances offline, so one product-team error became a shared outage for unrelated customers on the same infrastructure.

The rollback was never designed: the restore script was written over the weekend under pressure, failed on about 11% of orgs, and sometimes re-granted the access it was meant to remove.

Recovery pushed work onto customers - admins rebuilt user permissions by hand - stretching a one-day incident into a week of cleanup and support load.

Unscoped database scripts are company-level risk. Scope every change, stage the rollout, and build the rollback first - or the cure locks out more customers than the disease.

Access was gradually restored from 18 May; most organisations were working again by Monday 20 May, with sandboxes and stragglers taking several more days as Salesforce ran its restore script instance by instance. Salesforce said no action was needed beyond permission checks for the majority of customers, but the incident became a standard case study in change-scoping and blast-radius control for multi-tenant SaaS - and a reason platform teams treat database scripts on production with the same caution as code deploys.