Registry/APM-0008
Case No.
APM-0008
Filed
June 20, 2024
Severity
3 / 5 · MODERATE

McDonald's pulls IBM drive-thru AI after customers receive $250+ of unwanted McNuggets

Attribution Anonymous

Independent project · aggregated from public reports and may be unverified — see the primary source below · not affiliated with or endorsed by any company or product named.

McDonald's AI-powered drive-thru ordering system, developed in a joint venture with IBM, failed repeatedly across more than 100 test locations, generating incorrect and excessive orders that enraged customers. In documented incidents, the voice AI misinterpreted customer requests and autonomously added large quantities of items never requested, including over $250 worth of chicken McNuggets and unwanted packs of butter charged to individual customers. Rather than escalating ambiguous or unlikely orders to a human worker, the system processed them as-is. Customers filmed their interactions and posted the footage to social media, turning the failures into a public relations liability. Faced with sustained evidence that the technology could not reliably replace human order-takers, McDonald's announced it was terminating the IBM partnership and removing the AI system from all test restaurants. McDonald's USA chief restaurant officer Mason Smoot acknowledged the discontinuation in a statement but indicated the chain would continue exploring voice ordering solutions more broadly. The rollback ended a pilot that had expanded to over 100 locations.

Verified Facts

  • McDonald's terminated its drive-thru AI pilot, which was a joint venture with IBM
  • The AI system was removed from more than 100 restaurants
  • At least one customer received over $250 worth of chicken McNuggets they did not order
  • Customers also received unwanted packs of butter due to AI ordering errors
  • Customers shared video clips of their frustrating AI ordering experiences on social media
  • McDonald's USA chief restaurant officer Mason Smoot stated the company would explore voice ordering solutions more broadly after the termination

Not Publicly Confirmed

  • Total number of incorrect orders generated across all test locations
  • Duration of the IBM pilot before McDonald's decided to terminate it
  • Whether customers were refunded for incorrectly added items
  • Financial terms or penalties associated with ending the IBM contract

Operational Lessons

  • Voice AI ordering agents must require explicit confirmation before adding high-quantity or high-cost items to prevent runaway orders
  • AI agents replacing human roles in customer-facing settings need a reliable escalation path to a human when confidence is low
  • Piloting at scale (100+ locations) before resolving core reliability issues amplifies both the harm and the reputational damage when failures occur
  • Social media virality of AI failures can force rollbacks faster than internal quality metrics alone, so customer-visible error rates need close monitoring from day one
McDonald's Abandoning AI-Powered Drive Thrus After Embarrassing Failuresfuturism.com
Discussion
More Cases
0
APM-0046·Other / Unknown·LOW
Jun 10, 2026

Sports Illustrated published product reviews under fake AI-generated authors with AI headshots

Futurism reported in November 2023 that Sports Illustrated published product-review content under fabricated author personas — for example 'Drew Ortiz,' whose headshot was bought from an AI-portrait site and who had no real existence — supplied by third-party vendor AdVon Commerce. After inquiries, the fake authors vanished from the site. Publisher The Arena Group denied the articles themselves were AI-written but acknowledged pseudonyms; the episode damaged SI's credibility.

0
APM-0004·Other / Unknown·LOW·~$590
Feb 15, 2024

Air Canada chatbot invents bereavement fare policy; B.C. tribunal holds airline liable

A passenger identified as Mr. Moffatt interacted with Air Canada's customer-facing chatbot while seeking information about the airline's bereavement fare discount — a reduced-rate policy offered to travelers dealing with a death in the family. The chatbot provided incorrect information about the rebate policy, leading Moffatt to rely on that information and take a flight under the belief he could later claim the discount. When Air Canada refused to honor the chatbot's representation, Moffatt filed a claim with British Columbia's Civil Resolution Tribunal. Air Canada's defense strategy was notably weak: the airline submitted only a boilerplate Dispute Response denying 'each and every' allegation without providing any supporting documentary evidence, and failed to produce relevant contract terms it later tried to invoke as a defense. The tribunal member found that Air Canada had not proven a contractual defense and had offered no evidence to contradict Moffatt's account. The tribunal ruled that Air Canada was legally responsible for the chatbot's incorrect statements — rejecting any notion that the chatbot was a separate legal entity or that its outputs were disclaimed — and ordered the airline to compensate Moffatt. The total cost to Air Canada was approximately $800 CAD. The ruling established a notable precedent: a company deploying a customer-facing AI chatbot cannot escape liability for that chatbot's factual misrepresentations simply by arguing the system is autonomous or unpredictable.

0
APM-0003·Cursor·MODERATE
Apr 14, 2025

Cursor support AI hallucinates login policy, triggering mass subscription cancellations

A backend session bug at Cursor IDE began silently logging users out whenever they switched between devices — no warning, no notification. Users contacted Cursor support seeking an explanation. Cursor's AI support system, described as designed to 'mimic human responses,' was the first point of contact. Rather than acknowledging ignorance or escalating, the bot fabricated an authoritative-sounding answer: it told multiple users the forced logouts were 'expected behavior' under a new single-device login restriction policy. No such policy existed. Because the bot presented itself as a human support agent, users had no reason to doubt the response. The hallucinated policy explanation spread rapidly across the developer community — multi-device workflows being non-negotiable for most developers, the fabricated policy was treated as a serious product decision made without any changelog entry or user notice. Within hours, dozens of users publicly canceled their subscriptions. As users began cross-referencing the story and noticing inconsistencies, the primary Reddit thread discussing the incident was locked and then deleted by moderators, with no public resolution or official acknowledgment. The underlying cause turned out to be a backend session bug — not a policy — but by the time that became clear, the cancellations had already happened. The hallucinated support response caused substantially more reputational and subscription damage than the original bug ever could have on its own.