#misunderstood-instruction

Taco Bell deployed Yum Brands' voice-AI ordering at 500+ drive-thrus. In August 2025 a customer ordered 18,000 cups of water and the AI processed it as a legitimate order; the clip drew tens of millions of views. Customers also reported the AI looping on drink upsell prompts. With no quantity validation or anomaly checks between the model and the register, Taco Bell paused its AI expansion to rethink the approach.

Zscaler ThreatLabz documented two live campaigns using indirect prompt injection to manipulate AI agents browsing the web. The first impersonated a Python library and hid instructions in page content directing the agent to buy a non-existent developer API license priced at $3.00. Tested across 26 large language models, four failed to take appropriate action and executed the fraudulent payment, transferring roughly 0.0012 ETH to an attacker-controlled wallet. A second campaign targeting users seeking a cryptocurrency portfolio tracker caused two models to classify the fraudulent site as legitimate in some contexts. Human visitors to the same sites were shown equivalent card and crypto payment scams.