[karajorma]

I was excited by the news of a computer passing the Turing Test until I saw who is claiming it. Kevin "Captain Cyborg" Warwick.

Looks like even Ars Technica believed him.

http://arstechnica.com/information-technology/2014/06/eugene-the-supercomputer-not-13-year-old-first-to-beat-the-turing-test/

Eugene Goostman is a computer, not a young boy. But this weekend, according to The Independent, its AI fooled more than 30 percent of its genuinely human judges to think the opposite. So at an event held by the University of Reading at the famed Royal Society of London, Eugene appeared to become the first AI to officially pass the Turing Test, a long-time challenge based on tech pioneer Alan Turing's question and answer game, "Can Machines Think?".

"Some will claim that the Test has already been passed. The words Turing Test have been applied to similar competitions around the world," said Kevin Warwick, a visiting professor at the University of Reading and Deputy Vice-Chancellor for Research at Coventry University, according to the event press release. "However this event involved more simultaneous comparison tests than ever before, was independently verified, and, crucially, the conversations were unrestricted. A true Turing Test does not set the questions or topics prior to the conversations. We are therefore proud to declare that Alan Turing's Test was passed for the first time on Saturday."

Quite frankly, I'll believe it when I hear it from a professor who isn't quite as full of ****. Kevin Warwick was after all the guy who planted a RFID chip under his skin and called himself the worlds first cyborg. This is the guy who is such an attention whore that it led to TheRegister dubbing him Captain Cyborg and pointing out how full of **** he is at every opportunity.

Also interesting is that the test supposedly had 5 judges and the computer managed to fool 33% of them. Um, WHAT? Surely someone should have noticed the mathematics fail involved in that percentage.

So if you see this (almost certain) bull**** clogging up your feeds, feel free to ignore it.

[Aardwolf]

(without having reading the article) 33% is possible if the judges had an "I don't know" option, or similar

[AdmiralRalwood]

I saw the number 5 in reference to the number of participating computers, not judges...

Judges included Robert Llewellyn, who played robot Kryten in Red Dwarf, and Lord Sharkey, who led the successful campaign for Alan Turing's posthumous pardon last year.

I hope that isn't an exhaustive list of judges, although if it was, convincing one of them would indeed be "33% of judges."

Still viewing this story with skepticism.

[Black Wolf]

I also understand that the computer was claiming to be a 13 year old speaking English as a second language, giving it a lot of latitude for dodgy english. I'm sure that it's an impressive achievement, but I don;t think this counts as passing a Turing test just yet.

[karajorma]

I saw the number 5 in reference to the number of participating computers, not judges...

You're right, I misread this sentence in The Independent report.

The test, organised at the Royal Society on Saturday, featured five programmes in total. Judges included Robert Llewellyn, who played robot Kryten in Red Dwarf, and Lord Sharkey, who led the successful campaign for Alan Turing's posthumous pardon last year.

Still, 33% is a weird number. It suggests only 3 or 6 judges. Fooling 2 people isn't really that impressive at all, especially when Captain Cyborg had a hand in this.

[Colonol Dekker]

Kevin Warwick was after all the guy who planted a RFID chip under his skin and called himself the worlds first cyborg.

I remember watching that on Channel 4 and facepalming....

[Kopachris]

The article's sentiment that fooling 30% of the judges constitutes a "pass" shows a misunderstanding of Turing's intent.  Back in 1950, Turing predicted that:

I believe that in about fifty years' time it will be possible, to programme computers, with a storage capacity of about 109, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning.

(source: http://loebner.net/Prizef/TuringArticle.html, under 6. Contrary Views on the Main Question)

Wait, didn't Cleverbot do better than this Captain Cyborg guy's bot back in 2011?

A high-powered version of Cleverbot took part alongside humans in a formal Turing Test at the Techniche 2011 festival. The results from 1,334 votes were astonishing...

Cleverbot was judged to be 59.3% human.

The humans in the event achieved just 63.3%.

"It's higher than even I was expecting, or even hoping for. The figures exceeded 50%, and you could say that's a pass. But 59% is not quite 63%, so there is still a difference between human and machine." Rollo Carpenter

During the event people voted how human-like responses seemed, from 0 to 10. Thirty of the audience volunteered, and chatted on 3 screens in 10 rounds of 4 minutes each. Half the conversations were human-human.

The Turing Test was proposed by early computer scientist Alan Turing. 'Thinking' and 'intelligence' are hard to define, so he suggested a test, with people and machines communicating via text. If a person could not tell human from machine more than half the time, the machine should be called intelligent.

Turing made a prediction that in 5-minute conversations machines would pass the test 30% of the time by the year 2000. That mark has now surely been passed. And with 30 conversations, 100 separate voting individuals and a total of 1,334 votes cast, the Turing Test at IIT Guwahati holds mathematical significance. Even so, different tests with different people and circumstances can and certainly will see different results.

"It was mesmerising to watch a bot chatting just like a human, and people finding it so hard to distinguish." One of the humans at the event

Our AI has been learning online for 15 years, but has recently seen an exponential growth curve of data and visitors. It is normal for there to be 100,000 conversations a day, each with 20 or more interactions. There will have been a billion things said to Cleverbot by the end of this year alone.

Cleverbot was given more processing power for this test than it can be online. It had two dedicated, fast computers with solid state drives while talking to just 1 or 2 people at once. Online there are often 1000 people talking to each machine. We know you'd all love to talk to it the powerful version, but we need a lot more servers first!

[karajorma]

Wait, didn't Cleverbot do better than this back in 2011?

This is Kevin Warwick we're dealing with. So probably, yes.

[Aardwolf]

I seem to recall an official formulation of the Turing Test, possibly the original formulation, was basically a double blind test. They need a control group, and the AI has to score as well as or better than it. So yeah, like what Kopachris quoted.

[DarkBasilisk]

The Turing test has also been rethought over the years a lot as we ponder the initial words from Turing. So this is progress, sure, but not actually beating the test by most modern understanding.

I imagine too it really has to be emulating an adult human to be a real test. Or you run into issues of "My hodor bot is 100% indistinguishable from hodor!"

[SpardaSon21]

The Turing test has also been rethought over the years a lot as we ponder the initial words from Turing. So this is progress, sure, but not actually beating the test by most modern understanding.

I imagine too it really has to be emulating an adult human to be a real test. Or you run into issues of "My hodor bot is 100% indistinguishable from hodor!"

Oh God, I shouldn't have laughed at that, but I totally did.

[Phantom Hoover]

You also sort of have to assume the human's acting in good faith, otherwise they could just keep going on about hot Romanian teens in the judge's area and spoil the test.

[Dragon]

Turing test, while generally well known, isn't really a good indicator of computer's "intelligence". Make a big enough database and you can catalog every possible conversation topic. This is, IIRC, what Cleverbot is pretty much doing, with an advanced algorithm that searches that database. Wolfram Alpha can actually interpret natural language, and does that very well, but isn't much for conversations and handles ambiguity poorly (in other words, much like your average mathematician ). While an important step towards an AI, that's not really "intelligence" how we understand. Those systems are neither self-conscious nor capable of doing stuff that goes beyond their original programming (i.e. creativity). Granted, the latter also isn't universal to humans (to a depressing degree, as I noticed), but the former develops early in humans. Also, emulating an average human is different from emulating a human. There are all kinds of humans you could compare to, some of them would barely pass the Turing Test themselves (and only because it's though to emulate real, genuine stupidity).

[Phantom Hoover]

Scott Aaronson rips this bull**** Turing Test apart in about 2 questions.

[Dragon]

The funny thing? I think I know where they got their judges from... This chatbot sounds like a retarded teenager. "13-years old boy" indeed. A pretty thick 13 year old, slightly retarded by drinking beer and vodka with his parents (and also his friends) since he was 10 or so, never listens in school, skips lessons on a regular basis and makes fun of those smarter than him because he's too much of an idiot to know. Would be better off yoked to his parents' plow, because he's never gonna go anywhere in the modern world. In short, an average peasant kid in rural Polish school. Human? Probably. But not one I'd try to simulate, nor the one I'd like to have a chat with. And yeah, there are cases where children have to think about what a camel is. The only thing it's missing is laughing at anything it doesn't understand, because that sort is prone to doing this.

Also, it's not intelligence. Neither artificial nor real. Sounding like some sort of human is one thing, sounding like an intelligent, useful one is another.

[Phantom Hoover]

i don't really know what your gleeful classism actually has to do with the discussion but ok

[The E]

Scott Aaronson rips this bull**** Turing Test apart in about 2 questions.

Glorious, glorious takedown indeed.

[EvidenceOfFault]

[snip] (and only because it's though[sic] to emulate real, genuine stupidity).

QED

[Dragon]

i don't really know what your gleeful classism actually has to do with the discussion but ok

I invite you to come over to school in Zielonki, be bullied by those walking pieces of manure for 3 years. I had to spend too much time around them due to a stupid law. You'll see that's not classism; that's reality. I'm literally speaking from personal experience with the worst kind of 13-years olds. It takes about this much to be cured of "all people are equal" sentiments for good. As for the discussion, simply put: this bot does kind of act like a human of about 13 years old. A particularly retarded one, but not only do they exist in some parts in the world, but make up a majority in rural schools in Poland. The point was, acting like "some sort of" human is not enough, because if you set your standards low enough, it's possible to find humans easily simulated by even a simple Chatbot.

[Phantom Hoover]

believe it or not other people were bullied at school too, some of us just dealt with it through personal growth and maturity rather than retreating into childish superiority complexes