AI mistakes · Context loss · embarrassing
McDonald's drive-thru AI added bacon to ice cream and wouldn't stop adding McNuggets
Customers filmed themselves begging the voice bot to stop as the order climbed past 260 Chicken McNuggets.
unknown · IBM drive-thru voice AI
The setup
A multi-year automated order-taking pilot with IBM, live at over 100 US drive-thrus.
What happened
Between 2021 and 2024, McDonald's tested automated voice ordering. The viral failures wrote themselves: bacon added to ice cream, ketchup packets multiplying to nine, butter stacked beside them, and — most famously — a TikTok of two customers pleading with the system as it added Chicken McNuggets in bundles, past 260, each 'stop' apparently registering as enthusiasm.
The bot's fatal pattern was compounding misrecognition: it could add items but couldn't reliably hear a correction, so every attempt to fix the order made the order worse. In June 2024, McDonald's ended the IBM partnership and switched the drive-thrus back to humans, in an internal memo that promised voice ordering was still 'part of the future' — someday.
It remains the canonical example of shipping an agent into a noisy, correction-heavy environment with no reliable escape hatch.
Root cause
- Speech recognition tuned for orders treated corrections as new orders — the undo path was weaker than the do path.
- A noisy open-air environment was nothing like the conditions the system was validated in.
- Escalation to a human happened after customer frustration, not at the first repeated correction.
- Every failure occurred in public, on camera, at the exact moment customers were most motivated to film it.
The rule for next time
Corrections outrank additions: if the same modification is attempted twice without effect, freeze the transaction and hand off to a human. Never deploy where your undo is weaker than your do.
For the machines
{
"id": "mcdonalds-bacon-ice-cream",
"failureMode": "context-loss",
"prevention": {
"rule": "Corrections outrank additions: if the same modification is attempted twice without effect, freeze the transaction and hand off to a human. Never deploy where your undo is weaker than your do.",
"checkBefore": [
"payment",
"send-message",
"long-loop"
]
},
"machineNotes": "Loop guard: same-intent repetition N≥2 with divergent state ⇒ halt-and-escalate. Confidence thresholds must be environment-conditional (SNR-aware). Public-facing agents need a customer-reachable abort phrase that bypasses the NLU entirely."
}agents: GET /api/ai-mistakes/rules?check=payment