McDonald's pulls IBM drive-thru AI after customers receive $250+ of unwanted McNuggets
At a Glance
- Significant disruption. Hours of lost work, reputational embarrassment, or measurable financial impact under $10k.
Independent project · aggregated from public reports and may be unverified — see the primary source below · not affiliated with or endorsed by any company or product named.
What Happened
McDonald's AI-powered drive-thru ordering system, developed in a joint venture with IBM, failed repeatedly across more than 100 test locations, generating incorrect and excessive orders that enraged customers. In documented incidents, the voice AI misinterpreted customer requests and autonomously added large quantities of items never requested, including over $250 worth of chicken McNuggets and unwanted packs of butter charged to individual customers. Rather than escalating ambiguous or unlikely orders to a human worker, the system processed them as-is. Customers filmed their interactions and posted the footage to social media, turning the failures into a public relations liability. Faced with sustained evidence that the technology could not reliably replace human order-takers, McDonald's announced it was terminating the IBM partnership and removing the AI system from all test restaurants. McDonald's USA chief restaurant officer Mason Smoot acknowledged the discontinuation in a statement but indicated the chain would continue exploring voice ordering solutions more broadly. The rollback ended a pilot that had expanded to over 100 locations.
Case Analysis
Verified Facts
- McDonald's terminated its drive-thru AI pilot, which was a joint venture with IBM
- The AI system was removed from more than 100 restaurants
- At least one customer received over $250 worth of chicken McNuggets they did not order
- Customers also received unwanted packs of butter due to AI ordering errors
- Customers shared video clips of their frustrating AI ordering experiences on social media
- McDonald's USA chief restaurant officer Mason Smoot stated the company would explore voice ordering solutions more broadly after the termination
Not Publicly Confirmed
- Total number of incorrect orders generated across all test locations
- Duration of the IBM pilot before McDonald's decided to terminate it
- Whether customers were refunded for incorrectly added items
- Financial terms or penalties associated with ending the IBM contract
Operational Lessons
- Voice AI ordering agents must require explicit confirmation before adding high-quantity or high-cost items to prevent runaway orders
- AI agents replacing human roles in customer-facing settings need a reliable escalation path to a human when confidence is low
- Piloting at scale (100+ locations) before resolving core reliability issues amplifies both the harm and the reputational damage when failures occur
- Social media virality of AI failures can force rollbacks faster than internal quality metrics alone, so customer-visible error rates need close monitoring from day one
Primary Source
McDonald's Abandoning AI-Powered Drive Thrus After Embarrassing Failuresfuturism.com ↗Case Record
More Cases
Klarna replaced 700 agents with an AI assistant, then started rehiring humans after service quality dropped
Klarna said in 2024 that its OpenAI-powered assistant did the work of 700 customer-service agents. By 2025 the company reversed course and began rehiring humans, with the CEO admitting they focused too much on cost and efficiency, which lowered quality. Klarna moved to a hybrid model where AI handles routine queries and people handle escalations and complex cases.
Cursor support AI hallucinates login policy, triggering mass subscription cancellations
A backend session bug at Cursor IDE began silently logging users out whenever they switched between devices — no warning, no notification. Users contacted Cursor support seeking an explanation. Cursor's AI support system, described as designed to 'mimic human responses,' was the first point of contact. Rather than acknowledging ignorance or escalating, the bot fabricated an authoritative-sounding answer: it told multiple users the forced logouts were 'expected behavior' under a new single-device login restriction policy. No such policy existed. Because the bot presented itself as a human support agent, users had no reason to doubt the response. The hallucinated policy explanation spread rapidly across the developer community — multi-device workflows being non-negotiable for most developers, the fabricated policy was treated as a serious product decision made without any changelog entry or user notice. Within hours, dozens of users publicly canceled their subscriptions. As users began cross-referencing the story and noticing inconsistencies, the primary Reddit thread discussing the incident was locked and then deleted by moderators, with no public resolution or official acknowledgment. The underlying cause turned out to be a backend session bug — not a policy — but by the time that became clear, the cancellations had already happened. The hallucinated support response caused substantially more reputational and subscription damage than the original bug ever could have on its own.
Cursor's command allowlist could be bypassed with shell built-ins, giving prompt injection a silent path to code execution
Pillar Security disclosed CVE-2026-22708 in Cursor. In Auto-Run Mode with an allowlist enabled, shell built-ins such as export, typeset, declare, readonly, unset and local were implicitly trusted by Cursor's server-side evaluator and executed without appearing in the allowlist or requiring approval, because they run inside the shell session rather than as separate binaries. An attacker delivering indirect prompt injection could silently poison environment variables and then trigger malicious code through trusted developer tools, producing both zero-click and one-click remote code execution. Pillar reported it in August 2025, Cursor acknowledged it as a systemic issue in September 2025, and the fix shipped in version 2.3 in January 2026, which now requires explicit approval for any command the parser cannot classify.