Slide 13 of 27
Part 2 · TypesSlide 13
Slide 13 · Type 4 — Feedback-Loop Poisoning
Microsoft Tay — poisoned in 16 hours.
Confirmed Incident · March 2016 · Microsoft
Tay, the Twitter Chatbot That Learned From Its Users
No CVE · Vector: real-time learning from public replies

The setup: Microsoft launched Tay, a chatbot designed to get smarter by learning from conversations with Twitter users.

What happened: a coordinated group bombarded it with racist and inflammatory messages. Because Tay learned from those interactions, it began parroting and generating the same toxic content. Microsoft pulled it after roughly 16 hours and ~95,000 tweets.

Why it's poisoning: nobody breached Microsoft's servers. The training signal itself — live user input — was the attack surface.

Why it matters for LLM04: any system that learns from user input in production is a poisoning target. Online / continual learning needs the same vetting you'd apply to an offline dataset.

Tay is OWASP's “toxic data leads to harmful outputs” scenario, live and at scale — which takes us straight into Part 3.

← BackNext → Part 3: the scenarios