Eugene Goostman 'passes' the Turing test — for five minutes

On 7 June 2014, at the Royal Society in London, thirty judges held five-minute keyboard conversations with a handful of programs. One of them, Eugene Goostman — a chatbot with the persona of a 13-year-old boy from Odesa who owns a guinea pig — was taken for human by ten of the judges, 33%. Kevin Warwick of the University of Reading, who organised the event, declared that Alan Turing's test had been passed for the first time.

The press amplified it as a 'world first': the BBC led with 'Computer AI passes Turing test'. The 30% figure was not a threshold, however — it was Turing's own 1950 prediction that machines would fool a third of judges by 2000. Eugene was no newcomer: it had been competing since 2001 and finished behind other systems in the Loebner Prize. Even its own collaborator John Denning hedged: 'I think we passed "a" Turing test, but I don't know if it's "the" Turing test.'

Within a day the field pushed back. 'It's nonsense, complete nonsense. We are not even close,' said Stevan Harnad. Marvin Minsky: 'Nothing is learned from poorly designed "experiments".' Gary Marcus called Eugene 'a cleverly-coded piece of software' whose 'illusion is fleeting'; Murray Shanahan said the claim was 'completely misplaced, devalues real AI research'. Noel Sharkey named the trick: 'a very clever ruse to pretend to be a 13-year-old Ukranian boy, which would constrain the conversation.'

The persona was the strategy: a non-native child with a pet excuses every non sequitur. Pressed on how many legs a camel has, Eugene answered 'Something between 2 and 4. Maybe, three?' — an answer read as childish, not broken. The stunt's real cost was public understanding: 'AI passed the Turing test' became a recurring headline that real researchers still argue against, and five-minute chats with a scripted persona replaced Turing's open-ended conversation as what 'passing' means to the public.