Continuing dispatches on the war against spam

[denise]

A few days ago we let you know about spam prevention measures that we were taking to help stem some of the flood of garbage. One of those temporary measures included geoblocking all IPs from several of the countries that are our largest source of spam. This did (as we knew it inevitably would) have some collateral damage for real users, and we're very sorry!

We're continuing to experiment: this time we've slightly expanded the range of countries we're geoblocking to include the ones that we held off on geoblocking because it would affect too much legitimate use, but we've limited the geoblocking only to the account creation page. This should mean that if you were having trouble accessing the site because of geoblocks, you should be able to access 99% of the site without a problem, and the only page you won't be able to access is the account creation page. With luck, this should cut back heavily on our spam account creation without disrupting legitimate use of the site. The current list of countries that are geoblocked from account creation are Bangladesh, Cambodia, Egypt, India, Indonesia, Morocco, Pakistan, Singapore, Turkey, and Vietnam. (If you're an existing user from one of those countries and you'd like to make an additional account, email support@dreamwidth.org with the username you'd like to register and we can register it for you. If the number of requests gets to be enough that it's taking up too much of our time, we may have to pause this until we can build automated exceptions, but we'll start there.)

We will continue to monitor the results of these experiments and adjust as necessary: when we do one of these experiments, we always make sure to define in advance what "too much interference with legitimate use" will look like, and we try very hard to stick to it. I apologize to everyone who's been collateral damage in our efforts to filter out more of the goddamn spammers.

Comments

[dennisgorelik]

> IP addresses from multiple netblocks, from multiple providers, that are completely clean in every reputational database

I am suggesting to use your own reputational database of IP addresses, and keep your own database private.
So it will be hard for spammers to find out if their IP address is already blacklisted.

Bad IP should not prevent Dreamwidth account creation, but instead should allow spammer to create the account, so Dreamwidth can collect other spam indicators, such as:
- Email address and email domain.
- Connections to other Dreamwidth accounts.
- Content keywords.
- Other involved IP addresses.
- ...

> move on to the next group of clean ones

Do you mean that it is easy for a scammer to get access to clean IP addresses?
The spammer's dilemma is that if IP address is easy to access - then this IP address is quickly getting blacklisted.


> if I pulled the IPs of our last 100 spam accounts, every single one of them will have a cleaner reputation than the IP address you are currently using

Does Dreamwidth maintain the internal database of IP addresses Spam/Ham scores (based on Dreamwidth users activity)?

> Because they stop using them when they start accumulating negative reputation.

If Dreamwidth does not immediately delete spam accounts, then it may be quite tricky for spammers to detect that their IP address accumulated negative reputation in internal Dreamwidth database.

> there were about 130 ISPs licensed to operate in Bangladesh. We saw spam from over 100 of them.

So penalize IP addresses from 100 Bangladesh ISPs and do not delete accounts created from the remaining 30 Bangladesh ISPs.
This will put users' pressure on the bad ISPs to deal with spammers in their own IP networks.

> if your only understanding of toll fraud comes from a Google search and reading a surface-level article

I run a job board and deal with spam and scam every day.
Spam is a relatively minor issue for us vs scam (which is operated manually and not on a bot scale).

For spam indicators we use:
1) IP addresses (and networks).
2) Email addresses.
3) Content keywords.
4) Browser User Agents.
5) User's feedback.

> Site behavior is also not an accurate spam detection system.
> It detects less than 5% of spam account creation, and some days less than 1%.

What do undetected spam accounts do?

If they do something harmful - why you cannot detect such harmful behavior?

[dissectionist]

Denise, by this point I feel like this guy is just sealioning you. I’m sorry you’re having to deal with it.

[denise]

Let me try this again in extremely small words:

When we suspend a spam account, we already block the IP address. The spam we see is from fresh IPs that have never been used to create accounts before. The parts of your "one simple trick" suggestions that aren't useless, we have been doing for years, and they have stopped working.

Please stop. You are not making helpful suggestions. We have consulted some of the world's foremost experts on social media spam management. You are not one of the world's foremost experts on social media spam management. If you were, you would not be making useless suggestions that stopped working in 2015 and demanding that I explain the absolute basics of the field to you.

[dennisgorelik]

If you think that the spam detection system I describe is "one simple trick" - you misunderstand what I describe.

If you are not interested in the discussion about spam detection strategies - it is ok.

I thought that you posted about spam problems in order to get more ideas that might help you with improving your spam detection algorithms.

[denise]

I have explained to you repeatedly why your suggestions are not helpful and asked you to stop. Because you haven't listened to that request, I have banned you from dw_maintenance. In the future, to avoid being banned in other official communities, if site staff ask you to stop commenting in a discussion in an official community because your contributions are not helpful, stop commenting.

[marinarusalka]

Denise, you have the patience of a saint. I would’ve banned that dude three comments in, and not nearly as politely as you did.

Thank you for your hard work.

[kore]

Oh my God seriously. That was some deep level explaining he didn't deserve but it was intriguing to hear about, like an irritant ending up as a pearl!

[denise]

I am usually very happy to explain the nuances of things! My willingness to do so is often directly proportional to how many times I've explained it to the same person, heh.

[lovingboth]

A rare upside of spamsplaining :)

[andrewducker]

Seconding all of this.

[havocthecat]

I'm sorry you had to deal with that, but I learned a lot from reading it, if it helps to know someone else got something out of...that.

[denise]

It is actually sickly fascinating, in the way that people genuinely do not believe how bad the problem is and how absolutely localized the worst of the operations are. I just did the first full spam run with 24 hours of having geoblocking on and compared it to last Saturday (since there are absolutely weekly patterns): last week 69% accounts created were spam, this week it was 19%. And 2/3rds of that 19% were actually IPs from one of those 7 countries, just not flagged as such in our intermediary's geolocation database: it would have been 6% if their db were more accurate.