#data-exfiltration

In April 2023, within about 20 days of allowing ChatGPT, Samsung's semiconductor division had three incidents of employees pasting confidential data into ChatGPT — proprietary source code to check for bugs, code for defect-detection equipment, and a recording of an internal meeting transcribed for summarization. Because prompts can be retained by the provider, this risked exposing trade secrets. Samsung banned generative AI tools company-wide and warned that violations could lead to termination.

PromptArmor disclosed in August 2024 that Slack AI could be manipulated through indirect prompt injection: an attacker posting in any public channel could plant instructions that, when a victim later queried Slack AI, caused it to render a markdown link exfiltrating private-channel content (such as secrets or API keys) to the attacker's server via the URL — without the attacker ever accessing the private data directly. A later update that pulled files and DMs into answers widened the attack surface. Slack deployed a patch.