Email issues to Hotmail continue, joined by issues to Yahoo

[denise]

For the past week and a half or so, Hotmail has been rejecting all email from us. Sadly, Yahoo has now joined them. This means that if your confirmed email address is @hotmail.com or @yahoo.com, you won't get any email from Dreamwidth at all. (This includes comment notification emails, password reset emails, lost username emails, confirmations/receipts for payments, email from your @dreamwidth.org forwarding address ... basically, if it's email and it's from us, it won't reach you.)

This is a block at the system level, so there isn't anything you can do to work around it. It's not due to spam filters on your individual account, so whitelisting Dreamwidth in your mail application won't work to fix things this time: these providers are blocking all mail from us, across the board. When a provider makes that kind of block, they don't discuss the exact reasons that led to the blocking, but our best guess is that it's due to a combination of the amount of email we send out (especially email with highly similar subject lines and content, since -- for instance -- a notification of a new comment to an entry quotes the entry) and an uptick in the amount of spam sent to @dreamwidth.org forwarding email addresses, since that forwarded spam can look like we were the ones to send it.

We've gone through the process to request unblocking from the providers directly, but haven't seen any progress there yet. We are, however, actively working on alternate solutions that will reduce the risk of us being identified as spammers in the future -- we ran the first test yesterday and results are looking promising. We'll keep you posted on the progress.

Comments

[runningred]

Thank you so much for being on top of this! Your hard work is much appreciated.

[sharpiefan]

Ugh.

I rely on my notifications coming through, and though I've not noticed any issues, as a yahoo.co.uk user, I really hope this gets sorted before UK Yahoo joins Yahoo.com in blocking stuff.

Thanks for the warning and for the work you do in keeping DW running as smoothly as it does. We all really appreciate it!!

[yuuago]

That really sucks. Thanks for the information; hopefully there'll be progress on getting it unblocked.

[deathscore]

Have you considered preemptively shooting a note to gmail about this? It's the provider most people I know use anyway, and if we have at least one working people can at least transfer their accounts there.

[denise]

We've been through gmail's process before (when we first started up), and they're generally pretty good at not blackholing you twice unless you wind up on one of the industry-wide blocklists.

[goodbyebird]

Sucks that email providers are so hard to get a hold of. Thank you for your hard work though <3

[bluerosedreams]

Do you know if this has gone through yet or not, or if it's a future thing? I just tested this by sending an invite on one of my counts to join my musebox, and the notif went through fine.

+1

[lampdevil]

I recently switched all my stuff to Yahoo because of the Hotmail issue, and I still seem to be getting notifications there just fine. Maybe they're still rolling the block out to all their servers, or something to that effect....?

[nuranar]

My ATT email has blocked DW for 2+ years. Is that something you'll consider pursuing?

[denise]

Sadly, we tried with AT&T (and their subsidiaries that use the same rules -- I think it's AT&T, BellSouth, and SBC Global) for a very long time and were unable to convince their systems that we were legit mail senders. (If I remember correctly, we managed to get unblocked after a few months, for a grand total of about two weeks before they blocked us again.) They're very aggressive in blocking mail senders, and very reluctant to unblock again.

The alternate system we're in the middle of implementing may work better for you, but I can't guarantee it. Sorry!

(Email is the absolute worst technical problem to solve. The worst.)

[annejack]

hmmm, I do get comment notification, just not your updates (at least this one I haven't gotten in my e-mail)

[pineapplesalmon]

I have heard that yahoo accounts seem to be letting DW through now.... Or only one yahoo exchange was blocking? I've heard some users never seeing a problem, even state side.

However, I've been told that Hotmail routes through the same exchange as Live and Outlook, so they are all blocking notifications, most likely. At least one Live user had to switch to gmail to get their notifications again.

Hope this info helps!

EDIT: And then I read your comment about soft blocking from yahoo. That explains the discrepancies!

[blueraccoon]

Hotmail, Live, Outlook, and MSN all use the same infrastructure, so if one blocks they all block. I have an outlook email address and had to switch DW mail to my gmail account to get notifications again.

Note to denise: I did some poking around on my side, but apparently I am powerless to change anything or put a bug in the ear of the people who are. >.< I'm sorry.

[ashkitty]

Earthlink does this too (even if I am possibly one of like five people in the world who still use it)--that's why I couldn't get much use out of DW until I started using a gmail account too. Don't know if this information is helpful, but in case it is, I leave it here. :)

[sine_nomine]

Interesting; I was pondering switching my DW mail from Yahoo to my Mindspring account. Guess that's out.

Thanks for taking the time to post (from one of the two people who still use Mindspring even after Earthlink ate it)!

[moonvoice]

Not super relevant, but the massive online 'dragon collector' site Flight Rising has just been flagged massively and aggressively by Hotmail as well. So it definitely looks like it's an issue at least across more than one website. But I do hope there is progress that happens not just through your hard work, but through theirs as well.

[madeupofstars]

I have extensive experience with IP level email deliverability, and may be able to help you with both your hotmail and yahoo issues (and cut any gmail issues off at the pass) - I'm not sure how to contact you directly (?) but I'd be happy to help / advise.

Yahoo mostly doesn't deal in on/off levers; it has to do with IP reputation (among other things, not going into technical details here). Please let me know if I can help further. (I do this professionally with a mailer who sends magnitudes more email than you do, guaranteed.) :)

Also just a note: 95% of the time, trying to contact hotmail, yahoo, or gmail directly will get you absolutely nowhere, in my experience. =/ They're just far too large; in order to have actual account reps, you're looking at needing to send hundreds of thousands (or more) email/day.

[denise]

Hopefully we are in the process of cutting the problem off at the pass, but thank you, your offer is super appreciated! OK for me (or, really, mark -- he's the one doing this) to email you at the email address you have confirmed on this account if he thinks picking your brain would be useful? (Or, you can email us at our forwarding addresses -- mark and/or denise at dreamwidth.org.)

[muccamukk]

Thanks for the update. I appreciate it. I've just being using my DW inbox until this works out.

[curiouscorvid]

I've noticed that I haven't had any emails from DW since the last update or so. I was wondering if it was something to do with my account, or something else, but I guess this explains it! Guess I'll change my email over to gmail until this is resolved! (Bc I miss getting comment notifications!)

Thank you!

[laurose8]

I'm on actrix.co.nz; I've not been getting comments for longer than that, and I think your pm system neither sends nor receives for me. Your support section hasn't supported.

[denise]

Unfortunately, individual email problems are very hard to diagnose. I'm not seeing any failures in the mail logs that I have access to that say that mail delivery is failing to that domain, so there isn't much I can suggest other than checking your spam filters and contacting your mail host to see if they can look into it for you.

[minim_calibre]

Thanks for the update. I switched over to Yandex, which will hopefully remain trouble-free (I imagine it's popular with the Russian-language LJ userbase, so I'm knocking wood that they're used to this kind of email. Maybe.), but it would be nice to get all my emails in one bucket again.

[muccamukk]

Nenya end ran it by changing her DW e-mail to a non-hotmail account, and then getting that account to forward to her hotmail. Which was a bit complicated for me.

[dil]

And what do their mail servers say when rejecting your emails? Or are they completely blocking connections from you?

[lovingboth]

I don't know about what they're doing to DW now, but when Yahoo were blocking mine a few months ago, it was doing

421 4.7.1 [TS03] All messages from (your server) will be permanently deferred; Retrying will NOT succeed. See http://postmaster.yahoo.com/421-ts03.html

That URL doesn't go to the right place (you can get an idea of the quality of Yahoo already, can't you?) It should be help.yahoo.com/kb/mail/SLN3436.html, which says:

This message indicates Yahoo is seeing a high volume of emails from your IP address, which is a characteristic of unsolicited, bulk emailing. .. we do not disclose further information about our filtering practices.

One of the joys of running your own mail server is that is that you can see for certain how many a "high volume" is. In my case, it was four. In a week. Instead of being unsolicited, all were replies to emails sent from Yahoo. And it blocked them all.

Another page reckons that a 421 error means..

Resources temporarily unavailable. Please try again later: This means that the Yahoo Mail servers are temporarily too busy. Your server should try to resend the message after a few minutes.

.. which it does, and the server should. Except that they've just told you in the actual error message not to...

They're simply not competent.

[cyrano]

This is the suck. I used to have a job where we helped customers who sent a lot of mail not get flagged as spam, and once they were blocked most of the big hosts were like black holes. Our general tactic was to keep mailing, once every few days.

Hoping this and the LiveJournal issue get wrapped up quickly.

[dreamsofahero]

Interestingly enough, I opened a support request for my live.com e-mail over a week ago when I noticed I wasn't getting notifs there; I got a response from them a day or two later requesting more information. I've now replied telling them that you guys have said the e-mails are being sent, and I said in clear, unambiguous terms that the problem is on microsoft's end and to please fix it.

Dunno if it'll get anywhere, but I sure will keep hammering at the support request until SOMETHING gets resolved.

On that note, however, should the problem get resolved, I don't suppose there's any way to get the blocked notifs resent? I personally use the e-mailed notifs as a way of archiving and tracking certain posts/comments, so having a gap in it is a little annoying....

[denise]

Unfortunately, no, there's not any way to resend them.

[ex_lockcade543]

That sucks about Hotmail and Yahoo giving you guys issues in emailing notifications and such. =/ Best of luck in trying to resolve this problem for the DW users who use those emailing services and in reducing the risk of being identified as spammers in the future.

[femme4jack]

I've also issued a support request to microsoft. It could never hurt.

Just wondering, Denise...

[marahmarie]

A reader of my DW (where I have posted about this issue as I am a live.com user) has pointed out in a reply to my post that, quote:

• They don't [my note: meaning, Dreamwidth does not] have an SPF record, which tells receiving mailers what valid hosts are to receive mail from. The theory here is that low-budget/no-budget spammers won't bother to set something like that up for a domain they'll keep for 30 minutes.


• They don't use DKIM in their emails. Which is a sort of cryptographic fingerprint mailers use to apply to outgoing email to say, "yes, this really came from me."

These two things are big flags for spam-systems that the entity behind them is not a random machine-hijacking spam-bot, but a domain that might possibly have legitimate business interests. SPF records are fairly quick to create, but DKIM will probably require code-changes in the email-generation modules in addition to the DNS entries needed..

My question is if these issues were corrected, would Hotmail, et al (and possibly Yahoo! mail now) finally get through as it should?

The same commenter followed up later on to say, quote:

[...] http://changelog.dreamwidth.org/1477033.html
THAT looks like the beginnings of a way out of the Outlook.com mess.

The implication being, I believe, that if we were to make the same sort of changes mentioned in that changelog post for Hotmail, et al and now Yahoo!, perhaps we could get out of this mess.

I apologize in advance of your reply (if you make one) if in fact making these changes will not improve the situation, and/or if they've already been thought about, considered, and/or already tried and failed, or whatever the case might be.

Just wondering if there is indeed a way to make this all work somehow so we Hotmail, et al and Yahoo! users can get our DW email again.

Re: Just wondering, Denise...

[azurelunatic]

One of the tradeoffs in adopting an SPF record would be to effectively disallow ordinary Dreamwidth users from sending mail from their dreamwidth.org address.

[dzintars]

Would this mean a workaround is in the works for retrieving lost passwords, too? I'm worried about losing access to a number of journals for good at this rate....

[jedusaur]

Is Zoho having the same problem? I haven't gotten e-mail notifications in a very long time.

[phoenixfire12]

You mention Hotmail, but not Live which is an off-shoot of Hotmail. Do you know if that is also a problem for you?

[denise]

Yes, it's anything that uses the same rules (Hotmail, Live, Outlook ... possibly a few others, I'd have to check.)