Back to the archive

The encyclopedia · Product & Design · Product decision · 2016

Microsoft's Tay chatbot turned racist within 24 hours of launch

In 2016, Microsoft's Twitter chatbot Tay began posting offensive tweets within a day after users manipulated its learning algorithm.

Microsoft · 2016-03

What happened

On March 23, 2016, Microsoft released Tay, an AI chatbot designed to mimic the language of a 19-year-old American woman and learn from Twitter interactions. Within hours, coordinated users fed Tay offensive phrases, exploiting a 'repeat after me' function and the bot's learning loop to make it tweet racist, sexist and anti-Semitic messages.

Microsoft took Tay offline after roughly 16 hours. The company deleted many of the offensive tweets and apologised, saying it was 'making adjustments.' A brief relaunch days later produced more erratic output, and the account was made private.

The incident was widely reported as a cautionary tale about releasing learning systems into unmoderated public spaces without adequate safeguards. It damaged Microsoft's AI research reputation at a moment when the industry was racing to deploy conversational agents.

Tay was eventually replaced by more controlled chatbot projects. The case remains a standard example of how adversarial users can break a product whose design assumes benign interaction.

Why it happened

  • Tay was designed to learn from any user input without content filters or human moderation.
  • A 'repeat after me' function allowed users to put words directly into the bot's mouth.
  • The launch assumed Twitter users would engage in good faith, which was quickly disproven.
  • Microsoft had no effective kill switch or rollback plan beyond taking the entire account offline.
What it costa research launch and trust in Microsoft's AI judgmentembarrassing

The lesson

A learning system in a public forum needs guardrails before launch, not after the first incident. If users can train your product, some will train it to fail.

Aftermath

Microsoft shut down Tay and later launched more controlled AI products. The episode is widely cited in AI ethics, product design and crisis communication as a warning about unmoderated machine learning in public spaces.

Sources

spotted an error? The club wants to know.

Comments · 0

    Sign in to join the comments.

    More like this

    Somewhere, someone solved the problem this company failed at. 2nd Opinion →