Hard Light Productions Forums
Off-Topic Discussion => General Discussion => Topic started by: The E on June 06, 2014, 07:13:00 am
-
The US Secret Service, the federal law enforcement agency tasked with protecting the safety of current and former national leaders and their families, visiting heads of state, and others, posted a work order on Monday seeking the development of social media analytics software capable of detecting sarcasm online.
Seriously!
In addition to the “ability to detect sarcasm and false positives," the work order seeks the development of software with such alternative capabilities as “influencer identification,” “access to historical Twitter data,” the “ability to search online content in multiple languages,” “audience segmentation,” and “data visualization representations, [like] heat maps,” etc.
The agency hopes that such software would allow it to “synthesize large sets of social media data“ and “identify statistical pattern analysis.” Ed Donovan, a spokesman for the service, said the "objective is to automate our social media monitoring process. Twitter is what we analyze. This is real live stream analysis… We are looking for the ability to quantify our social media reach. We aren’t looking solely to detect sarcasm,” reported The Washington Post.
Source (http://arstechnica.com/tech-policy/2014/06/us-secret-service-wants-software-to-detect-sarcasm-on-social-media/)
I for one welcome our new, sarcasm-detecting overlords.
-
To be fair, I would love to have a sarcasm detector online :P
I would be impressed if (when?) they get that thing working.
-
Detecting sarcasm is a hard task. Doing so successfully in a medium devoid of the signifiers we use to denote sarcasm in speech requires the analyzer to have a sufficiently accurate model of the speaker's state of mind and his opinions on a given subject; whether it's possible to develop such a model just from social media posts is IMHO doubtful.
Therefore, the only possible solution is to develop an AI that can understand subtext, social cues, and context.
Also, I can't stop laughing at the "detect [...] false positives" part.
-
To be fair, I would love to have a sarcasm detector online :P
I would be impressed if (when?) they get that thing working.
I would imagine it's to remove obvious, overt sarcasm from the list of positives rather than subtle sarcasm.
Good luck creating a model for detecting subtle sarcasm when plenty of us don't have a fully functional online sarcasm detector. :)
And if they succeed, all terrorists and criminals will speak in a new sarcastic code online. :lol:
-
"He used... sarcasm. He knew all the tricks, dramatic irony, metaphor, bathos, puns, parody, litotes and... satire. He was vicious."
-
Sarcasm is barely detectable by humans (thus its usage in the first place doh), and they pretend they can actually detect it with an algorithm? Who dafuq is writing these checks for these projects? Seriously, just fire the dumb ****.
-
Sarcasm is barely detectable by humans (thus its usage in the first place doh), and they pretend they can actually detect it with an algorithm? Who dafuq is writing these checks for these projects? Seriously, just fire the dumb ****.
The same people who underwrote stealth technology, are underwriting laser and mass driver technology, along with power assistance suits and a host of other stuff impossible only a handful of decades previously. all it takes is the money and an explicit goal, the expertise will follow.
edit:
what happens when china develops an AI to post on the behalf of government functionaries, will the US system be able to detect this?
-
lolwot it's impossible for an algorithm to be able to detect sarcasm, at least before it learns how to understand all human communication, culture, nuance, etc. It's basically AGI in a nutshell. We are decades before this happens. Hell, Google probably still thinks I'm an asian black woman about to get pregnant for the second time, conservative and queer, conspiratorial and with a taste for bananas. Really, this "checkpoint" is the last one in general artificial intelligence, it should be the last objective, not the "next to go", so to speak.
When you have a computer able to detect sarcasm, you'll have something to which you can humanly talk to.
-
lolwot it's impossible for an algorithm to be able to detect sarcasm, at least before it learns how to understand all human communication, culture, nuance, etc. It's basically AGI in a nutshell. We are decades before this happens. Hell, Google probably still thinks I'm an asian black woman about to get pregnant for the second time, conservative and queer, conspiratorial and with a taste for bananas. Really, this "checkpoint" is the last one in general artificial intelligence, it should be the last objective, not the "next to go", so to speak.
When you have a computer able to detect sarcasm, you'll have something to which you can humanly talk to.
well its a good thing i said decades in my post then
-
derp
-
Sarcasm detecting AI? Now there's a really useful machine!
EDIT: In all seriousness, I wonder whether it would be possible to cause the proposed AI to get stuck in a loop just by writing a conundrum, or insert other funny stuff into text that gets interpreted as instructions? Or to do something more entertaining, like taking over the NSA computer doing the reading?
-
To be fair, this is a somewhat important initiative in the US - you have no idea how many people get arrested and charged on the basis of satirical and/or sarcastic social media posts. True threat distinction is important if they want to continue using automated threat detection.
-
I suspect it will turn out to be nothing more than an algorithm that detects keywords like Terrorism, bomb, etc followed by a smiley face.
The guy who wrote up the proposal was actually joking but lacking a machine to detect sarcasm, DARPA thought it was real and decided to fund it. :p
-
To be fair, this is a somewhat important initiative in the US - you have no idea how many people get arrested and charged on the basis of satirical and/or sarcastic social media posts. True threat distinction is important if they want to continue using automated threat detection.
Well, I've no idea of the build-in surveillance capabilities in the social media, nor about the number of people put to jail because of social media stuff. But I suppose you could jail 90 % of the population just by looking at their social media posts. I know I have advocated not caring too much about speed limits in rural areas or highways for example. However, my understanding of the law is that there is a requirement to have proof of wrong-doing before arrest can happen. And no, shouting nasty stuff to police officer in public usually doesn't get you arrested or charged (though it definitely is not a good idea), so why should it in Facebook? Any sort of pre-emptive action by police tends to become difficult to justify - well, at least in the court of this country.
EDIT: On the other hand, I've never written how much over the speed limit I've been driving to social media, just in case of this sort of surveillance turns real some day :) Then again, I wouldn't write about driving over speed with exact numbers to the local newspaper with my name either. They might get a case to prosecute after all. But my understanding is, the police must still have some kind of independent measurement data for it, put it other way, the person's own words do not actually mean anything in case of traffic speed violations. Makes you wonder whether there is a double standard brewing up here?
-
Ahh, the classics:
-
I wrote this sentence to help the secret service test their algorithms:
The authorities' inability to detect sarcasm is no laughing matter.
Good luck fellow patriots!
-
To be fair, this is a somewhat important initiative in the US - you have no idea how many people get arrested and charged on the basis of satirical and/or sarcastic social media posts. True threat distinction is important if they want to continue using automated threat detection.
This is what humans are good for.
But... have you paid attention to what happens in any of those cases? The prosecution doesn't care about intent. They don't want "true threat distinction", they want to lock up anybody who makes the TSA look bad (and similar).
Well no, maybe it's a case of "you made us look bad by virtue of being unable to detect your sarcasm"... Idunno.
-
They don't want "true threat distinction", they want to lock up anybody who makes the TSA look bad (and similar).
By that logic, they should just lock up everyone in TSA... :) The only people responsible for making it look bad are TSA employees themselves, and the reason they're thought about as nothing but a bunch of thieves and bullies is that they are nothing but thieves and bullies.
It's more for the NSA, I think, not TSA. The former have the real reason to use such an AI. Given the recent controversies, it must have become clear that they have to be darn careful about who they brand as "terrorist", especially if they want to expand. Indeed, the problem with any "acceptable" monitoring is that over 99% of what you see is not what you're looking for. Contrary to popular belief, the NSA isn't looking for any sort of "thoughtcrime" or slightest signs of dissent, but actual threats. Since jokes about terrorism vastly outnumber actual, serious mentions of it, it's more of a question of detecting whether someone is actually serious.
Still, it'd be especially problematic, considering even humans have big problems with spotting sarcasm, especially on the internet (but often in spoken conversation, too). It might be better to have a program that can "sort of tell" if someone is joking, then pass anything unsure to humans. I can see a "sarcasm and false positive detection" algorithm taking off a lot of weight from the NSA analysts' shoulders even if it's rather rudimentary. Remember, the data sets involved are immense, and effective monitoring of internet requires very good filtering systems, both human and automatic. Especially that you're risking both arresting innocents and allowing a terrorist attack to happen if they screw up.
-
Yeah w/e, brain compression is lossy, and I've gotten to the point of lumping all the corrupt bureaus together.
As I said (or hinted?), maybe they could avoid the false positives a little better, but in the meantime we need something that says "any testimony made on the Internet is not admissible as evidence of intent". Maybe it's okay as a place to start an investigation, i.e. "hey this guy made a bomb threat, let's go see if he actually has the bombs he says he has", but... bleh.
-
It's more for the NSA, I think, not TSA. The former have the real reason to use such an AI. Given the recent controversies, it must have become clear that they have to be darn careful about who they brand as "terrorist", especially if they want to expand. Indeed, the problem with any "acceptable" monitoring is that over 99% of what you see is not what you're looking for. Contrary to popular belief, the NSA isn't looking for any sort of "thoughtcrime" or slightest signs of dissent, but actual threats. Since jokes about terrorism vastly outnumber actual, serious mentions of it, it's more of a question of detecting whether someone is actually serious.
If the NSA was out to make an AI they wouldn't be posting openly-available work orders about it. I'll quote the Stratfor Glossary's entry on the NSA:
National Security Agency. Also called The Fort. Owns
Sigint and Elint. Completely out of control. It is so
compartmentalized they refer to other offices as B1 or D8
and genuinely don’t know what anyone else does.
-
If the NSA was out to make an AI they wouldn't be posting openly-available work orders about it. I'll quote the Stratfor Glossary's entry on the NSA:
Probably so, but that doesn't mean they couldn't use something like this. Note, Secret Service officially commissioned the system, but that doesn't mean other agencies won't get it. I imagine that NSA (if they know what's good for them) would be very interested in this given recent developments, but I can see uses for CIA and even FBI as well. TSA, on the other hand, has little reason for something like this - their job is, apparently, wealth redistribution (they redistribute valuable items from wealthy passengers to poor TSA agents) and emission control (they reduce number of polluting aircraft by reducing demand for air travel). :) And given intelligence they've shown, somehow I doubt in their ability to even use internet (though the smartest ones might account for some of the 4chan trolls).
-
i don't even know why the TSA is relevant beyond your desire to shoehorn in as many attempted jokes about them as possible
-
This quote:
, they want to lock up anybody who makes the TSA look bad (and similar).
And I like making fun of them. :)
Still, the main point still stands. Secret Service is not an intelligence agency, IIRC, their main duty is to bodyguard the President. I don't think they run their own data-mining operation just to detect potential plots against the President. It's highly likely that even if they do end up using this system for something, an organization like the NSA or the CIA would have much bigger need for an AI like this.
-
But if they wanted such an AI, and it was possible to make one, they'd have one. And we wouldn't know a thing about it. So much like the TSA I don't understand why you keep bringing this up.
-
But if they wanted such an AI, and it was possible to make one, they'd have one.
Yes. But it, currently, is not possible to make one. Indeed, this effort, if undertaken seriously, will involve some of the greatest minds on this planet and some of the most amazing developments in the field of AI. I just don't see this being anything but a money sink otherwise. Mind you, they're looking to making something which is better than humans in recognizing emotions and context behind an internet text. The last time you could conceal such an effort from public eyes was the Cold War. Therefore, it makes sense for a relatively inconspicuous (and a fairly well-regarded one) government agency to commission such a project, and have others (that actually need it) get it later, in a discreet manner.
-
Idk, maybe we are overestimating the technical difficulty of this. How often, when you see something you think is serious (but it turns out was sarcasm) do you actually go to the trouble of looking up that person's posting history?
-
You don't need to "look up" anything if you're a forum member for long enough to recognize sarcasm. And if you're new, you either wait for the others to reply, or act like a noob and reply having no idea if the post is sarcasm or not, and then are promptly corrected, maybe with an image of Picard facepalming. :) Needless to say, not a viable option for a filtering program. Also, this assumes the person in question is any good at internet sarcasm. If they're not, and the claim isn't too extreme, you need to start assuming. My approach to extreme, but ambiguous claims, stupidity and so on, is to always laugh them off, on a basis that if it's a joke, we're meant to laugh, and if it's not, then laughing at the claim is one of the better insults you can use. Especially that doing it the other way (taking everything seriously), the covering behind the flimsy "Poe's law" if you're off mark results in coming off as dour and probably insulting intelligent, but more humorous people as a side effect. Unfortunately, the government will be looking precisely for such extreme claims, so "laugh it off" default behavior isn't a good idea, either.
Now, here comes the hard part: you can tell if someone is sarcastic/joking or not by checking how other people reply. Easy for a human, mind(CPU?)-boggingly difficult for a computer of any kind. You need to catch context of the discussion, the person's feeling on the subject, essentially comprehend the entire conversation. Needless to say, if a computer does that, it'll have the right (and probably capability) to call itself an AI.