Hard Light Productions Forums

Site Management => Site Support / Feedback => Topic started by: The E on December 06, 2010, 09:16:17 am

Title: The bot invasion
Post by: The E on December 06, 2010, 09:16:17 am
Seems there's been an upsurge in bot activity lately. Can anything be done about that?
Title: Re: The bot invasion
Post by: Fury on December 06, 2010, 09:31:24 am
Not much. Except possibly upgrading to SMF 2.0 RC4 and enabling CAPTCHA for first three posts. Assuming of course that there isn't someone who's willing to bypass CAPTCHAs manually or that CAPTCHA hasn't been broken.
Title: Re: The bot invasion
Post by: pecenipicek on December 06, 2010, 11:07:05 am
or getting one or two mods for the damned thing...
Title: Re: The bot invasion
Post by: Galemp on December 06, 2010, 11:47:16 am
It's far from being out of control. I got two email notifications the other day, and by the time I had checked it out another moderator had resolved the issue. I think we're just fine.
Title: Re: The bot invasion
Post by: TopAce on December 06, 2010, 11:54:20 am
Bots?

Where?
Title: Re: The bot invasion
Post by: headdie on December 06, 2010, 11:56:21 am
I flagged one yesterday but that's all I have seen for a good month or more
Title: Re: The bot invasion
Post by: General Battuta on December 06, 2010, 11:56:28 am
Bots?

Where?

Lots. We delete their posts very rapidly.

There have been quite a few in the past couple days.
Title: Re: The bot invasion
Post by: Lucika on December 06, 2010, 12:21:23 pm
http://www.bigbluecup.com/games.php?action=detail&id=1059
:P
Title: Re: The bot invasion
Post by: Droid803 on December 06, 2010, 01:35:53 pm
What?
Title: Re: The bot invasion
Post by: Goober5000 on December 06, 2010, 02:27:48 pm
The problem is that board moderators only see the bots that appear on their board.  Global moderators and admins see every bot.  It's not uncommon for me to check my email one morning and find twelve reported posts from four or five different boards.  This is probably what The E was referring to, since he's a global mod.

I'll talk to rev_posix about upgrading to SMF 2.0 beta.
Title: Re: The bot invasion
Post by: General Battuta on December 06, 2010, 02:34:51 pm
Just got another one a few minutes ago.
Title: Re: The bot invasion
Post by: NGTM-1R on December 06, 2010, 05:40:54 pm
I'd say you could add more mods, or switch the mods around so that some of the boards have a more active moderating presence, but now that I consider it I don't think you have any surplus moderating material.
Title: Re: The bot invasion
Post by: The E on December 06, 2010, 05:44:42 pm
It's not like it's that bad a problem. The issue is more that several bots managed to get through the captcha in a very short timeframe, which is never a good sign.
Title: Re: The bot invasion
Post by: pecenipicek on December 06, 2010, 05:53:23 pm
or getting one or two mods for the damned thing...
just to avoid any confusion and angry glares, i meant mods, as in modifications, not moderators. the smf customisation site is full of em, mostly good/awesome ones.
Title: Re: The bot invasion
Post by: General Battuta on December 06, 2010, 05:54:52 pm
I'm 99% sure that the existing global mods have caught every single bot within thirty seconds of posting. The issue is that it's annoying.
Title: Re: The bot invasion
Post by: karajorma on December 06, 2010, 06:17:43 pm
Tell me about it. I must have deleted 6 or 7 bots in the last week alone.
Title: Re: The bot invasion
Post by: Goober5000 on December 06, 2010, 08:02:56 pm
The problem is that not only do you have to delete the posts (which is easy, any moderator with access can do it) -- you also have to ban and delete the bot.  And that's something that 1) takes several clicks to accomplish; and 2) can only be done by an admin.
Title: Re: The bot invasion
Post by: karajorma on December 06, 2010, 09:15:54 pm
Yep. That's the annoying bit.
Title: Re: The bot invasion
Post by: pecenipicek on December 06, 2010, 09:39:19 pm
The problem is that not only do you have to delete the posts (which is easy, any moderator with access can do it) -- you also have to ban and delete the bot.  And that's something that 1) takes several clicks to accomplish; and 2) can only be done by an admin.
are you telling me that global mods cant ban?
Title: Re: The bot invasion
Post by: General Battuta on December 06, 2010, 09:50:37 pm
Nope.
Title: Re: The bot invasion
Post by: Jeff Vader on December 07, 2010, 01:06:29 pm
The problem is that not only do you have to delete the posts (which is easy, any moderator with access can do it) -- you also have to ban and delete the bot.  And that's something that 1) takes several clicks to accomplish; and 2) can only be done by an admin.
are you telling me that global mods cant ban?
We're global moderators, meaning we can moderate on all the public boards. You don't see regular moderators banning people from their boards either.
Title: Re: The bot invasion
Post by: pecenipicek on December 07, 2010, 11:47:01 pm
admins choice if you ask me.



my opinion on the matter of banning stuff however is that global mods should also have the ability to do so.



(from personal experience tho, boards i admin are mostly single or dual admin boards, unlike HLP)
Title: Re: The bot invasion
Post by: Niue on December 08, 2010, 02:16:03 am
(http://13things.net/wp-content/uploads/2009/03/math.png)
 :D :D :D
Title: Re: The bot invasion
Post by: Scotty on December 08, 2010, 02:35:18 am
Would that be -cos(-TT/2), or did I make a mistake somewhere?
Title: Re: The bot invasion
Post by: FUBAR-BDHR on December 08, 2010, 02:40:11 am
It's always 0 or infinity.
Title: Re: The bot invasion
Post by: Herra Tohtori on December 08, 2010, 04:09:21 am
(http://13things.net/wp-content/uploads/2009/03/math.png)
 :D :D :D



Well it's a partial derivate by x from a function that doesn't have a variable x; evaluated when x is 2pi - but there is no x in the function itself anywhere, which means the answer is always 1 0, the slope coefficient of a horizontal line graph...
Title: Re: The bot invasion
Post by: FreeSpaceFreak on December 08, 2010, 06:30:35 am
Well it's a partial derivate by x from a function that doesn't have a variable x; evaluated when x is 2pi - but there is no x in the function itself anywhere, which means the answer is always 1, the slope coefficient of a horizontal line graph...
The slope of a horizontal line... wouldn't that be 0? :P
Title: Re: The bot invasion
Post by: Macfie on December 08, 2010, 09:01:40 am
Since slope is the run divded by the rise and the rise is zero the slope would be infinite.
Title: Re: The bot invasion
Post by: General Battuta on December 08, 2010, 09:06:13 am
Since slope is the run divded by the rise and the rise is zero the slope would be infinite.

I'm pretty sure the slope of a horizontal line actually is zero. Vertical lines have infinite slope. I think your run/rise is backwards; slope is usually delta Y over delta X.  :nervous:

Title: Re: The bot invasion
Post by: Macfie on December 08, 2010, 10:35:20 am
You are correct, I had the formula backward.  So it would be zero.
Title: Re: The bot invasion
Post by: Lucika on December 08, 2010, 12:25:38 pm
Yay for nerdy offtopics!
Title: Re: The bot invasion
Post by: Herra Tohtori on December 08, 2010, 02:39:11 pm
Yeah, my bad. It's still a trick question, though, unless I'm reading something else wrong...
Title: Re: The bot invasion
Post by: headdie on December 08, 2010, 02:48:05 pm
any further administrator comments on whether SMF 2 is the way to go?
Title: Re: The bot invasion
Post by: rev_posix on December 11, 2010, 02:44:06 am
any further administrator comments on whether SMF 2 is the way to go?
Has anyone looked at the existing modules for anti-spam?  It doesn't appear that there are any installed at the moment, but most of them do seem to need at least 1.1.12 (current installed version is 1.1.11).

Personally, I'm hesitant to install anything on any system that's an RC for a production system, ifyaknowwhatImean...
Title: Re: The bot invasion
Post by: Galemp on December 11, 2010, 12:22:53 pm
(http://13things.net/wp-content/uploads/2009/03/math.png)

Ironically bots would be more likely to get that right than humans. I prefer KittenAuth, where you have to pick the pictures of kittens out of an array of other cute fluffy animals.
Title: Re: The bot invasion
Post by: Goober5000 on December 11, 2010, 12:34:43 pm
KittenAuth is awesome, but last time I visited their site I couldn't download their code.  All their site links were broken.

As for SMF 2 RC, http://forum.nasaspaceflight.com/ is another forum I go to pretty frequently, and they use SMF 2 RC3.
Title: Re: The bot invasion
Post by: Shade on December 11, 2010, 12:45:08 pm
While we're on the topic, I just want to say Good Job to the supermods. On a couple of occasions now we've had bots active when no admins were around to bring down the banhammer, but the global mods have been fast enough on the trigger that you barely noticed the posts before they were deleted.
Title: Re: The bot invasion
Post by: General Battuta on December 11, 2010, 01:02:28 pm
Thank you.
Title: Re: The bot invasion
Post by: General Battuta on December 14, 2010, 11:22:53 pm
Two more today. Got them within ten seconds of their posts, I think, but it's getting to be annoying. Doubly so for the admins, I imagine.
Title: Re: The bot invasion
Post by: Dilmah G on December 14, 2010, 11:25:05 pm
Gargh, this is getting absolutely ridiculous.
Title: Re: The bot invasion
Post by: Scooby_Doo on December 15, 2010, 12:13:06 am
Time to power-up the photon beam cannons.  :)
Title: Re: The bot invasion
Post by: Galemp on December 15, 2010, 01:03:41 am
Hm. Just had another few reported posts in the past couple days, after a loooooong dry spell.

Maybe this is something we should start taking seriously.
Title: Re: The bot invasion
Post by: Fury on December 15, 2010, 01:06:10 am
:sigh:

I need to do something about this, don't I? :(
Title: Re: The bot invasion
Post by: Goober5000 on December 15, 2010, 02:10:28 am
:shaking: It's far worse (http://tvtropes.org/pmwiki/pmwiki.php/Main/ItGotWorse) than we thought. :shaking:

Take a look at the last several pages of new members, sorted by member ID.  We are absolutely infested with spam accounts.  They don't even have to post... they can ply their devious trade simply by putting links in their signatures!

I have temporarily disabled new member registration while we discuss what to do.  I went and ban-deleted the most recent two dozen accounts with links in their signatures, but that only accounted for two days.  I have no idea how long they've been getting under the radar.

I suggest a multi-pronged approach to this new outbreak:
1) Run a database query that blanks all signatures for members with a postcount less than X (where X is to be determined, but probably not more than 5).
2) Run a database query that deletes all unactivated members, or members with 0 posts who haven't visited in X days (where X is, let's say, 30).
3) Manually review the username list for the past 30 days (from step 2) and delete all unactivated members, or members with 0 posts, who have ridiculous user info (e.g. a name like "whid0ds9w" and an email like "[email protected]").
Title: Re: The bot invasion
Post by: Fury on December 15, 2010, 02:18:28 am
or members with 0 posts who haven't visited in X days (where X is, let's say, 30).
Not doable thanks to our General Discussion board, which does not count posts.
Title: Re: The bot invasion
Post by: Goober5000 on December 15, 2010, 02:20:46 am
The database is capable of distinguishing people with a postcount of 0 from people who have posted 0 times.
Title: Re: The bot invasion
Post by: Fury on December 15, 2010, 02:27:30 am
That's good then.
Title: Re: The bot invasion
Post by: karajorma on December 15, 2010, 02:44:59 am
I have temporarily disabled new member registration while we discuss what to do.  I went and ban-deleted the most recent two dozen accounts with links in their signatures, but that only accounted for two days.  I have no idea how long they've been getting under the radar.

Unless we're doing something immediately I'd rather you didn't do that. Better to let bots register and delete them than to prevent legitimate users from joining. Close registration only if we're going to have something in place by tomorrow or the day after.

I did look into preventing the X^0 group from having signatures but as far as I can see, you can't do that with SMF (You can with VB but there were other issues there).
Title: Re: The bot invasion
Post by: Goober5000 on December 15, 2010, 02:51:31 am
Until we solve the problem I'm not keen on keeping registration open while it has this big gaping security hole.  It may take us until the weekend to run the SMF upgrade, and in that time we may end up with another 200 bots to delete. :(

I'll reactivate registration if Fury or another admin concurs, but I think the risks outweigh the benefits at this time.  There was something like a 50-1 ratio of bots to legitimate registrations.
Title: Re: The bot invasion
Post by: karajorma on December 15, 2010, 03:05:35 am
I'd just rather not be turning away legitimate users. It's not going to be that hard to clean up the database surely?


Or is SMF super crappy in this respect too? :rolleyes:
Title: Re: The bot invasion
Post by: Goober5000 on December 15, 2010, 03:08:54 am
I'd just rather not be turning away legitimate users. It's not going to be that hard to clean up the database surely?
See for yourself.  Sort the members by join date, then read all the usernames from the second page onward.


Quote
Or is SMF super crappy in this respect too? :rolleyes:
No, it's pretty easy to delete a member.  The problem is, it requires several clicks, and you can't batch-delete a bunch of users.  In the same way, banning also requires several clicks and can't be batched.  Multiply these tasks by several scores of users and it gets very frustrating very quickly.


EDIT: As for cleaning up the database itself, that would actually be quite a bit easier, yeah.  It would take a bit of thought, but it would not be tedious at all to execute the SQL queries I described in my earlier post.
Title: Re: The bot invasion
Post by: karajorma on December 15, 2010, 03:19:37 am
That's what I meant by super crappy. VB allows batch operations of this type IIRC. The problems I had with GW are due to the weird way it was set up.

I think we might as well delete anyone who hasn't activated their account (except those from the last 7 days) cause we have a pretty stupid number of those.
Title: Re: The bot invasion
Post by: Goober5000 on December 15, 2010, 03:23:25 am
All right, I'll have to take care of those tomorrow.  It's 4 am here...
Title: Re: The bot invasion
Post by: karajorma on December 15, 2010, 03:26:21 am
I can do that much myself at least.
Title: Re: The bot invasion
Post by: Galemp on December 15, 2010, 07:53:29 am
On behalf of the users of HLP, thank you all. :)
Title: Re: The bot invasion
Post by: TopAce on December 15, 2010, 11:04:38 am
Looks like adding/removing badges will not cause serious headache for a while. :P
Title: Re: The bot invasion
Post by: Goober5000 on December 15, 2010, 05:42:09 pm
Looks like adding/removing badges will not cause serious headache for a while. :P
Yes, that's one positive aspect. :)

Unfortunately, it will be replaced by users complaining vociferously that the HLP theme and all their favorite forum mods haven't been updated yet. :p
Title: Re: The bot invasion
Post by: Aurora Paradox on December 15, 2010, 05:53:15 pm
Take all the time you need admins.  In my opinion, having HLP bot free takes precedence over restoring the forum theme.
Title: Re: The bot invasion
Post by: mjn.mixael on December 15, 2010, 07:40:59 pm
I want my theme and badges back!  :nervous:







 :ick:
Title: Re: The bot invasion
Post by: Droid803 on December 15, 2010, 09:02:44 pm
Has the invasion been stopped?
Or at least quelled?
Curious to hear the results.
Title: Re: The bot invasion
Post by: Goober5000 on December 15, 2010, 10:59:58 pm
The invasion has, by definition, been stopped because new registrations are currently subject to admin approval.

Once we get all the security features on-line, we'll re-enable email registration and see if they did any good.
Title: Re: The bot invasion
Post by: Delta_V on December 16, 2010, 12:31:04 am
I guess this says something about the mods and admins around here.  I don't think I ever saw a single bot post, and I've been on a fair amount the last few days.  I had no idea it was even a problem.  But yeah, I'll put up with the generic backgrounds for a while if it means no bots.
Title: Re: The bot invasion
Post by: FUBAR-BDHR on December 16, 2010, 12:42:56 am
The real question is how long before they break all the new security.  It's a never ending war. 
Title: Re: The bot invasion
Post by: Black Wolf on December 16, 2010, 01:55:53 am
I guess this says something about the mods and admins around here.  I don't think I ever saw a single bot post, and I've been on a fair amount the last few days.  I had no idea it was even a problem.  But yeah, I'll put up with the generic backgrounds for a while if it means no bots.

Heh, it's actually kinda fun. Since we're all spread over different timezones, there's often only one or two global mods or admins online, so it's a matter of checking "New Posts" every couple of minutes and dealing with them faster than they can make new topics. That said, I'd be happy to see them disappear completely.
Title: Re: The bot invasion
Post by: headdie on December 16, 2010, 02:50:11 am
The real question is how long before they break all the new security.  It's a never ending war. 


There is that, but isn't the break/reduced load from manually deleting each post and spam account worth it even if it is for a couple of months
Title: Re: The bot invasion
Post by: Starman01 on December 16, 2010, 04:33:48 am
I'm always amazed to see how people are able to program bots that can outrun or read these captcha's. To bad they don't spend their talents on something more usuable. This botwar (nor here, but everywhere) is getting out of hand. In some places I registered the captchas were already so much complicated, that I hardly couldn't read them myself and needed several tries.

If you are looking for a good one, I saw one that I found actually very genial

Example : It shows 3 Persons.

One Ski-Driver, one Bow-Hunter, one Fisher. In each silluette are 3 letters, and the question is :

What is written within the Ski-Driver ?

Sounds too me like something that is hard for a bot to break (unless the bots are outrunning the graphical part)
Title: Re: The bot invasion
Post by: Fury on December 16, 2010, 05:12:22 am
If you log out and try to register, you'll see something similar here as well. :)
Title: Re: The bot invasion
Post by: Starman01 on December 16, 2010, 05:19:34 am
 :lol: Good one.

And even better, thank heaven I'm already registered here. Some of the captcha's are hardly readable to me (I have a red/green colour weakness with my eyes), and the ship images is too dark for me too (maybe because I still have a CRT, it happens often with pictures you people post here). In that case, I have to reload the captchas a few times until I get one readable (or the audio understandable), and have to post the image in Photoshop to brighten it up

But that's just me, and I'm certainly an exeption :) Good work sofar, should keep the bots out (and hopefully no humans)  :yes:
Title: Re: The bot invasion
Post by: Nemesis6 on December 16, 2010, 07:13:32 pm
If it continues like this, you could always block Chinese and Russian IPs. One site, can't remember which one it was, but it was one of those image-board type sites, had to block the entire country of Ukraine to stop spammers.

Obviously this isn't optimal by any stretch of the imagination, but it could get the job done if it gets to the point where it becomes a big problem. Maybe just block all future registrations emanating from these countries. By the way, in case you haven't noticed, I'm assuming the bots have Chinese or Russian IPs! :)
Title: Re: The bot invasion
Post by: karajorma on December 16, 2010, 08:31:16 pm
Or we could do something sensible instead of blocking off entire countries.
Title: Re: The bot invasion
Post by: FUBAR-BDHR on December 16, 2010, 10:30:34 pm
Oh no Kara's in China that must mean he's a bot.

Like they wouldn't just use proxy servers to get around the blocks.
Title: Re: The bot invasion
Post by: Starman01 on December 17, 2010, 03:48:46 am
Hire two or three people from the board, that handle new registration and make a small pm contact first, so you can check if a human is on the other side :)
Title: Re: The bot invasion
Post by: jr2 on December 18, 2010, 09:45:02 am
That doesn't fix humans registering bots... right?
Title: Re: The bot invasion
Post by: Lucika on December 18, 2010, 02:56:49 pm
That doesn't fix humans registering bots... right?
Not sure, I'm guessing their amount is minimal compared to the automated bots though.
Title: Re: The bot invasion
Post by: Starman01 on December 18, 2010, 03:06:57 pm
a.. human ... bot... ?

That makes me headache, my brain doesn't like paradox stuff :) I didn't knew this existed, I thought this is all about automated software :)
Title: Re: The bot invasion
Post by: Goober5000 on December 18, 2010, 03:18:12 pm
Well, we did get one spam registrant: "Jeniferleona", registered yesterday.

That's a lot better than dozens per day though.  And we still have to clean out all the old botty members.
Title: Re: The bot invasion
Post by: jr2 on December 20, 2010, 03:07:26 pm
a.. human ... bot... ?

Basically, human wants money fa$t.  So human can either get programs to control bots doing both registering and spamming - this gets a lot done fast as long as the bots aren't detected by CAPTCHA or w/e.  Or do the registering work himself; a bit slower, but it assures that his bots won't be detected as bots until they have had a chance to post.
Title: Re: The bot invasion
Post by: Starman01 on January 13, 2011, 02:49:32 pm
Sorry for digging this up, but after the discussion with the unreadable captchas, can it be that there are still bots registering ? Just noticed the username : djnattyd 

I cannot think that someone will use that sort of name if he wants to seriously participate......

Just wondering though  :nervous:
Title: Re: The bot invasion
Post by: FUBAR-BDHR on January 13, 2011, 03:24:07 pm
All the security in the world won't keep someone from manually registering a bot. 
Title: Re: The bot invasion
Post by: Mongoose on January 13, 2011, 03:39:43 pm
That name sounds like it could be reasonably human to me.
Title: Re: The bot invasion
Post by: Nohiki on January 13, 2011, 03:44:59 pm
I dunno if SMF has such a module, but the bots in general try to fill in every input, including multiple image-reading algorythms for captcha. The trick was to tell the user not to fill in the captcha, while the bot would and thus the bot won't register. It worked for me quite well as it also filtered boons who didn't bother to read what they were supposed to do :D
Title: Re: The bot invasion
Post by: Starman01 on January 13, 2011, 03:45:43 pm
Ah yes, I forgot the "human" bot thingie........ strange people are wandering this planet  :wtf: Well, i guess everybody needs a hobby :)
Title: Re: The bot invasion
Post by: Polpolion on January 13, 2011, 03:55:55 pm
I have a confession guys... I am a bot.
Title: Re: The bot invasion
Post by: Starman01 on January 13, 2011, 04:01:13 pm
We knew it   :eek2:  Burn him, I say :)

  :P
Title: Re: The bot invasion
Post by: Goober5000 on January 14, 2011, 04:33:56 am
I dunno if SMF has such a module, but the bots in general try to fill in every input, including multiple image-reading algorythms for captcha. The trick was to tell the user not to fill in the captcha, while the bot would and thus the bot won't register. It worked for me quite well as it also filtered boons who didn't bother to read what they were supposed to do :D
This is a good idea. :yes: