dw_maintenance | (no subject)

People getting mail at Gmail are reporting email delays of about an hour. As far as I can figure out, it's because some of our recent changes to our network have wiped out our existing sending reputation with them and made us start building reputation again, and they're refusing all mail on first delivery and making us re-send it. (It's a common spam reduction technique, because spammers don't bother retrying if the first attempt fails.)

There isn't much we can do about it but wait it out until Gmail decides that we're legit senders again, but we'll poke at it and see if there's anything we can do to make the process go faster. (I doubt there will be, though; Gmail is persnickety.) In the meantime, to get comment notification email faster, you can switch your confirmed email to a different provider, or just refresh your on-DW inbox.

EDIT: And people are now letting me know that mail's delayed to other providers, too, which is probably follow-on effects from having to send everything to Gmail at least twice. There isn't a lot we can do about it; I'm sorry about the hassle, folks.

Flat | Top-Level Comments Only

My deepest sympathies on email deliverability problems. That kind of thing falls into my Dayjob responsibilities, so I know those feels.

I noticed that dreamwidth.org doesn't have a DMARC entry yet. This is Yet Another Thing to say to mailers, "no, really, I'm me". It works along with SPF and DKIM, which are already set up.

Last month, Amazon SES added support to customize the MAIL FROM: address in the SMTP conversation. This is a big thing for SES customers, as it makes DMARC much easier to set up. Create an SPF record in the 'dreamwidth.org' domain, set it up right with -all at the end, and deliverability would increase a bit. Better yet for troubleshooting, gmail will put a helpful additional header in messages saying how well incoming email passes DMARC, SPF, and DKIM

dmarc=pass (p=NONE dis=NONE) header.from=dreamwidth.org

Pretty handy.

One caveat I know from bitter experience. Setting a policy to enforcing will break mail-forwards. People who have gmail addresses but forward email off of gmail will generate bounces like, "Mail administrators at $domain have administratively prevented this email from delivering," which will break things. Great for platforms where nonrepudiation is more important than deliverabiity, not so great for platforms where deliverability is paramount.

One of the things that has turned out to create difficulties is that Dreamwidth offers an @dreamwidth.org forwarding address as well as sending notifications from an @dreamwidth.org address. Therefore a lot of the stuff which could be done to verify Dreamwidth's notifications would screw up that.

`\_(ツ)_/`

Sadly, we can't do DMARC, because we offer the forwarding email address for paid accounts and a lot of people send mail "from" their dreamwidth.org address from a bunch of places.

Putting you in the awkward position of being both an automated mail-generator, as well as remailer. That's a tough position, and there is no way to keep everyone happy. We can't use enforcing-mode DMARC at DayJob because we need deliverability far more than anti-phishing; all it really does for us is tell other mailers that subdomains should be bit-bucketed.

Waiting for IP reputation to regenerate sucks a lot.

The incredibly painful thing about all of this is that there's windows of normal delivery and you think 'hey, maybe it's gotten back to normal', then you have an hour to six/seven hours up of delay, and it sucks.

Yeah, exactly. We've discussed a few ways of fixing it while keeping the DW mailing address -- a necessity, given how much certain people, myself included, rely on it! -- but all of them require more than a bit of work, so we haven't settled yet on what the best solution is.

Fucking email!