Hard Light Productions Forums
Off-Topic Discussion => General Discussion => Topic started by: karajorma on June 09, 2014, 12:15:27 am
-
I was excited by the news of a computer passing the Turing Test until I saw who is claiming it. Kevin "Captain Cyborg" Warwick.
Looks like even Ars Technica believed him.
http://arstechnica.com/information-technology/2014/06/eugene-the-supercomputer-not-13-year-old-first-to-beat-the-turing-test/
Eugene Goostman is a computer, not a young boy. But this weekend, according to The Independent, its AI fooled more than 30 percent of its genuinely human judges to think the opposite. So at an event held by the University of Reading at the famed Royal Society of London, Eugene appeared to become the first AI to officially pass the Turing Test, a long-time challenge based on tech pioneer Alan Turing's question and answer game, "Can Machines Think?".
"Some will claim that the Test has already been passed. The words Turing Test have been applied to similar competitions around the world," said Kevin Warwick, a visiting professor at the University of Reading and Deputy Vice-Chancellor for Research at Coventry University, according to the event press release. "However this event involved more simultaneous comparison tests than ever before, was independently verified, and, crucially, the conversations were unrestricted. A true Turing Test does not set the questions or topics prior to the conversations. We are therefore proud to declare that Alan Turing's Test was passed for the first time on Saturday."
Quite frankly, I'll believe it when I hear it from a professor who isn't quite as full of ****. Kevin Warwick was after all the guy who planted a RFID chip under his skin and called himself the worlds first cyborg. This is the guy who is such an attention whore that it led to TheRegister dubbing him Captain Cyborg and pointing out (http://www.theregister.co.uk/2012/07/26/captain_cyborg_gong/) how full of **** (http://www.theregister.co.uk/2010/05/26/captain_cyborg_cyberfud/) he is at every (http://www.theregister.co.uk/2011/06/17/captain_cyborg_on_radio_4/) opportunity (http://www.theregister.co.uk/2004/04/29/captain_cyborg_is_back/).
Also interesting is that the test supposedly had 5 judges and the computer managed to fool 33% of them. Um, WHAT? Surely someone should have noticed the mathematics fail involved in that percentage.
So if you see this (almost certain) bull**** clogging up your feeds, feel free to ignore it.
-
(without having reading the article) 33% is possible if the judges had an "I don't know" option, or similar
-
I saw the number 5 in reference to the number of participating computers, not judges...
Judges included Robert Llewellyn, who played robot Kryten in Red Dwarf, and Lord Sharkey, who led the successful campaign for Alan Turing's posthumous pardon last year.
I hope that isn't an exhaustive list of judges, although if it was, convincing one of them would indeed be "33% of judges."
Still viewing this story with skepticism.
-
I also understand that the computer was claiming to be a 13 year old speaking English as a second language, giving it a lot of latitude for dodgy english. I'm sure that it's an impressive achievement, but I don;t think this counts as passing a Turing test just yet.
-
I saw the number 5 in reference to the number of participating computers, not judges...
You're right, I misread this sentence in The Independent report.
The test, organised at the Royal Society on Saturday, featured five programmes in total. Judges included Robert Llewellyn, who played robot Kryten in Red Dwarf, and Lord Sharkey, who led the successful campaign for Alan Turing's posthumous pardon last year.
Still, 33% is a weird number. It suggests only 3 or 6 judges. Fooling 2 people isn't really that impressive at all, especially when Captain Cyborg had a hand in this.
-
Kevin Warwick was after all the guy who planted a RFID chip under his skin and called himself the worlds first cyborg.
I remember watching that on Channel 4 and facepalming....
-
The article's sentiment that fooling 30% of the judges constitutes a "pass" shows a misunderstanding of Turing's intent. Back in 1950, Turing predicted that:
I believe that in about fifty years' time it will be possible, to programme computers, with a storage capacity of about 109, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning.
(source: http://loebner.net/Prizef/TuringArticle.html, under 6. Contrary Views on the Main Question)
Wait, didn't Cleverbot do better than this Captain Cyborg guy's bot back in 2011 (http://www.cleverbot.com/human)?
A high-powered version of Cleverbot took part alongside humans in a formal Turing Test at the Techniche 2011 festival. The results from 1,334 votes were astonishing...
Cleverbot was judged to be 59.3% human.
The humans in the event achieved just 63.3%.
"It's higher than even I was expecting, or even hoping for. The figures exceeded 50%, and you could say that's a pass. But 59% is not quite 63%, so there is still a difference between human and machine." Rollo Carpenter
During the event people voted how human-like responses seemed, from 0 to 10. Thirty of the audience volunteered, and chatted on 3 screens in 10 rounds of 4 minutes each. Half the conversations were human-human.
The Turing Test was proposed by early computer scientist Alan Turing. 'Thinking' and 'intelligence' are hard to define, so he suggested a test, with people and machines communicating via text. If a person could not tell human from machine more than half the time, the machine should be called intelligent.
Turing made a prediction that in 5-minute conversations machines would pass the test 30% of the time by the year 2000. That mark has now surely been passed. And with 30 conversations, 100 separate voting individuals and a total of 1,334 votes cast, the Turing Test at IIT Guwahati holds mathematical significance. Even so, different tests with different people and circumstances can and certainly will see different results.
"It was mesmerising to watch a bot chatting just like a human, and people finding it so hard to distinguish." One of the humans at the event
Our AI has been learning online for 15 years, but has recently seen an exponential growth curve of data and visitors. It is normal for there to be 100,000 conversations a day, each with 20 or more interactions. There will have been a billion things said to Cleverbot by the end of this year alone.
Cleverbot was given more processing power for this test than it can be online. It had two dedicated, fast computers with solid state drives while talking to just 1 or 2 people at once. Online there are often 1000 people talking to each machine. We know you'd all love to talk to it the powerful version, but we need a lot more servers first!
-
Wait, didn't Cleverbot do better than this back in 2011 (http://www.cleverbot.com/human)?
This is Kevin Warwick we're dealing with. So probably, yes.
-
I seem to recall an official formulation of the Turing Test, possibly the original formulation, was basically a double blind test. They need a control group, and the AI has to score as well as or better than it. So yeah, like what Kopachris quoted.
-
The Turing test has also been rethought over the years a lot as we ponder the initial words from Turing. So this is progress, sure, but not actually beating the test by most modern understanding.
I imagine too it really has to be emulating an adult human to be a real test. Or you run into issues of "My hodor bot is 100% indistinguishable from hodor!"
-
The Turing test has also been rethought over the years a lot as we ponder the initial words from Turing. So this is progress, sure, but not actually beating the test by most modern understanding.
I imagine too it really has to be emulating an adult human to be a real test. Or you run into issues of "My hodor bot is 100% indistinguishable from hodor!"
Oh God, I shouldn't have laughed at that, but I totally did.
-
You also sort of have to assume the human's acting in good faith, otherwise they could just keep going on about hot Romanian teens in the judge's area and spoil the test.
-
Turing test, while generally well known, isn't really a good indicator of computer's "intelligence". Make a big enough database and you can catalog every possible conversation topic. This is, IIRC, what Cleverbot is pretty much doing, with an advanced algorithm that searches that database. Wolfram Alpha can actually interpret natural language, and does that very well, but isn't much for conversations and handles ambiguity poorly (in other words, much like your average mathematician :)). While an important step towards an AI, that's not really "intelligence" how we understand. Those systems are neither self-conscious nor capable of doing stuff that goes beyond their original programming (i.e. creativity). Granted, the latter also isn't universal to humans (to a depressing degree, as I noticed), but the former develops early in humans. Also, emulating an average human is different from emulating a human. There are all kinds of humans you could compare to, some of them would barely pass the Turing Test themselves (and only because it's though to emulate real, genuine stupidity). :)
-
Scott Aaronson rips this bull**** Turing Test apart in about 2 questions. (http://www.scottaaronson.com/blog/?p=1858)
-
The funny thing? I think I know where they got their judges from... :) This chatbot sounds like a retarded teenager. "13-years old boy" indeed. :) A pretty thick 13 year old, slightly retarded by drinking beer and vodka with his parents (and also his friends) since he was 10 or so, never listens in school, skips lessons on a regular basis and makes fun of those smarter than him because he's too much of an idiot to know. Would be better off yoked to his parents' plow, because he's never gonna go anywhere in the modern world. In short, an average peasant kid in rural Polish school. Human? Probably. But not one I'd try to simulate, nor the one I'd like to have a chat with. And yeah, there are cases where children have to think about what a camel is. :) The only thing it's missing is laughing at anything it doesn't understand, because that sort is prone to doing this.
Also, it's not intelligence. Neither artificial nor real. :) Sounding like some sort of human is one thing, sounding like an intelligent, useful one is another.
-
i don't really know what your gleeful classism actually has to do with the discussion but ok
-
Scott Aaronson rips this bull**** Turing Test apart in about 2 questions. (http://www.scottaaronson.com/blog/?p=1858)
Glorious, glorious takedown indeed.
-
[snip] (and only because it's though[sic] to emulate real, genuine stupidity). :)
QED
-
i don't really know what your gleeful classism actually has to do with the discussion but ok
I invite you to come over to school in Zielonki, be bullied by those walking pieces of manure for 3 years. I had to spend too much time around them due to a stupid law. You'll see that's not classism; that's reality. I'm literally speaking from personal experience with the worst kind of 13-years olds. It takes about this much to be cured of "all people are equal" sentiments for good. As for the discussion, simply put: this bot does kind of act like a human of about 13 years old. A particularly retarded one, but not only do they exist in some parts in the world, but make up a majority in rural schools in Poland. The point was, acting like "some sort of" human is not enough, because if you set your standards low enough, it's possible to find humans easily simulated by even a simple Chatbot.
-
believe it or not other people were bullied at school too, some of us just dealt with it through personal growth and maturity rather than retreating into childish superiority complexes
-
You don't get it, do you? Were you really so lucky to grow up only among equally or more intelligent people? Probably easier in whatever civilized place you live. Well, I had to grow up among a band of idiots. The only way not to become like people you live with is to distance yourself from them. As such, I had a choice of being an idiot obsessed with sex and alcohol, or resolving to put myself above all this. One can not function in a large group without taking up it's characteristics. Cutting them off wasn't so hard, really, they were so low that even you would've felt superior should you meet one of them (unlikely, unless you take a trip to Poland). It's not a superiority complex, it's just fact. I have to say, until I actually saw those people, I was, just like you, naively thinking that all people are equal and all that. Actual contact with people set me straight quickly, and you know what's funny?. This approach worked, and I'm more today than any one of those idiots could ever hope to be. It wasn't by chance (nor money - schools in Poland are free) that I went to one of two best high schools in my city, and I was one of the 3 people from that school in there, and the only one from my class.
Besides, that doesn't matter. You're ignoring a point I was making, and derailing the thread. If you don't have anything to say on topic of the AI, don't say nothing. I won't say anything more, either. I invite you to try conversing with rural 13-year olds from some backwards country one day. You'll either see my point, or find yourself right at home with them... Either way works for me. :)
Back on topic, is that chatbot available online? The article quoted provided a link, but it doesn't work for me.
-
believe it or not other people were bullied at school too, some of us just dealt with it through personal growth and maturity rather than retreating into childish superiority complexes
I would have said this but less harshly. Sounds like you had a tough childhood, but not an unusual one.
-
Can we get back on the subject please?
-
Sure. But there isn't much more to be said about the chatbot. :) It's not particularly smart, could probably pass for a somewhat moronic human for a while, and doesn't do anything revolutionary.
I formulated an argument that was ignored in favor of personal discussion by PH, so let's try again. Turning test isn't the best way of determining a computer's intelligence. In fact, I'd say that there can be no definite intelligence test one could use to compare computers to humans, because human intelligence varies greatly. Even the most advanced chatbots do not really understand what they're saying, nor can they create anything new, or form an independent opinion on a subject. IMO, a true, strong AI would need to be able to do that to be compared to humans. Anything else is just a souped up database search engine with a natural language interpreter. Those two would be important parts of any AI (our memory and reading ability, respectively), but there's more to true intelligence than that.
I would have said this but less harshly. Sounds like you had a tough childhood, but not an unusual one.
You're right, of course. I was unusual, not what happened to me. :) I react differently from most, and I'm prone to making observations and pondering the world I live in. Indeed, people I discussed my worldview with agreed with me. They didn't came to those conclusions themselves because they didn't think about it, nor observed morons so long and so closely (I had a choice between morons and wheat. Wheat is less annoying, but also less entertaining). I wouldn't be so confident in what I'm saying if I didn't discuss it before. My experiences are relevant only in that I've seen a lot of stupidity, and thus can probably identify and classify it.
-
I would definitely appreciate that whole subthread coming to an end. This is a topic on computers, not your past experiences. Karajorma has already made that clear. Next time it comes back up should not happen. If it does, there will be problems.
-
Fine, I just hate leaving things just... dangling like that. Besides, we've already agree that the bot doesn't really do what was claimed. There's hardly anything but random banter left to say. And derailing something is easiest at the end of the tracks, so to speak. :)
-
If you have nothing to say, stop speaking. Final warning.
Scott Aaronson rips this bull**** Turing Test apart in about 2 questions. (http://www.scottaaronson.com/blog/?p=1858)
Glorious, glorious takedown indeed.
I was glad to see the addendum that Veselov isn't responsible for this nonsense and has no illusions that his program has passed the Turing Test. When even the guy who wrote the program thinks it's being overhyped that's a pretty strong sign that people have gone a little bit off the deep end about it.
-
Getting back to the main topic, the reason the Turing test is formulated the way it is stems from Turing's own theoretical work on abstract computing machines. Two Turing machines are considered equivalent if they always produce the same output given the same input; and since thanks to the halting problem you can't meaningfully compute any non-trivial information about a TM that's the only way to really think about it. Applying this concept to the question of whether a machine can be as intelligent as a human brain and taking some artistic licence you arrive at the idea that interrogating the computer to see if it can be discerned from a human is an important first hurdle along the way.
-
While I never bought the hype from the very moment I saw the headline, I must say that Aaronson is either being a jackass or a fool. Yes, of course his conversation with the internet bot he found is abhorrently godawful, but this has been the case with all internet bots, whose more complex versions powered up by potent machines dedicated to a single conversation in a more direct talk (in bot contests, etc.) are waaaay more believable and interesting than that conversation implies. The test was not made with the "kind" of mind Aaronson talked to, in a nutshell. Or, if the main engine was more or less the same, the amount of resources the net bot had at its disposal is obviously considerably smaller in all sorts than the original that went through the tests.
I would very much like to read the original scripts of the talks that alledgedly fooled the judges.
-
(http://cdn.themis-media.com/media/global/images/library/deriv/746/746718.jpg) (http://www.escapistmagazine.com/articles/view/comicsandcosplay/comics/critical-miss/11611-Turing-The-Premises)
-
I would very much like to read the original scripts of the talks that alledgedly fooled the judges.
As would I.
Who wants to bet they are never made available, let alone published in a peer reviewed journal then? People seem to be forgetting this is Captain Cyborg that we're dealing with.
-
http://www.theguardian.com/technology/2014/jun/09/eugene-person-human-computer-robot-chat-turing-test
It's kind of painful to read some of the exchanges.
-
You think? Could you tell me what are you? I mean your profession.
Oh what a fruitful conversation! :-)
-
Yeah, i've seen a lot of the examples. It's pretty pathetic, I'm certain I've seen better chatbots in the past? Either way, it really shouldn't be able to fool anybody. It repeats itself often and acts very strange...
-
i've personally spent more time than is healthy trolling iGod... so yeah...
if nothing else, it was a sorta "hur durrr" way to pass a slow afternoon...
-
I've personally interacted with with a bot that I probably wouldn't have known was a bot if I wasn't told. Granted, I wasn't trying to figure out if it was or not. It was a virtual teacher designed to help with basic programming. It also didn't speak very much because I in no way needed help with the trivial programming task that was the test case, but the few times it did it was quite impressive. It even asked me if I learned to code in C because it thought my style was similar.