dw_maintenance | (no subject)

People getting mail at Gmail are reporting email delays of about an hour. As far as I can figure out, it's because some of our recent changes to our network have wiped out our existing sending reputation with them and made us start building reputation again, and they're refusing all mail on first delivery and making us re-send it. (It's a common spam reduction technique, because spammers don't bother retrying if the first attempt fails.)

There isn't much we can do about it but wait it out until Gmail decides that we're legit senders again, but we'll poke at it and see if there's anything we can do to make the process go faster. (I doubt there will be, though; Gmail is persnickety.) In the meantime, to get comment notification email faster, you can switch your confirmed email to a different provider, or just refresh your on-DW inbox.

EDIT: And people are now letting me know that mail's delayed to other providers, too, which is probably follow-on effects from having to send everything to Gmail at least twice. There isn't a lot we can do about it; I'm sorry about the hassle, folks.

Flat | Top-Level Comments Only

Just to fill you in a little on things that are happening, as per this post with a google forum representative, a few people have been trying to work things out with them to figure out the delay. This morning, after having sent in header information on one of my delayed notifs, I received this:

Just ran the header through our logs, and since it was sent by an @amazonses.com domain, we only have records for when it was received by our servers and delivered. To that effect, it was received and delivered to xxxxxxx@gmail.com without delay (took ~2 seconds to be delivered after it hit our servers).

If there was a delay, it occurred on the outbound side, with @amazonses.com, and unfortunately our logs can't track that. To look into the additional claims, we'd need more headers, but so far there doesn't appear to be any issues on our end.

There's a nifty tool you can access here https://toolbox.googleapps.com/apps/messageheader/ that will show you where delays occur (if any)

when I ran the same header using that tool, this is what I see - basically once the message was received by Google, it was delivered to recipient within 2 seconds. (shows a screenshot of the program used w/my header).

It appears if there is a problem, it's not on Gmail's servers, or they haven't gained enough info yet. I'm not entirely sure what amazonses.com is, but could there be another way to look at this? The problem is getting worse, not better, and there have been consistent delays the last 3 days with an hour being the lowest time of delay.

Thanks for the info! That's really weird, because we're seeing things going out on our end reliably...

This is well past my level of knowledge, but I'll pass along the info.

I know this is a giant pain in the ass, both for us because wtf, notifs, and for you because I can only imagine the amount of complaining going on behind the scenes, but I really hope that it can eventually be pieced together so a solution can be found. I'm guessing you guys are just as frustrated as we are, more so since it's like 'Well, damn. What the hell do we do to fix this?' I hope that can help at least a little for whoever fiddles with things and when and whatever happens, i know I'd love a progress report (when and if it can be managed), just so at least we know where we stand.

Thanks for being as on top of this as can be given the weird effed up circumstances.

A follow up notice I got from them was:

Google had said, "It's still not showing the outbound activity though, presumably where the delay occurred, which would need to come from Amazonses' server logs. "

that means the message coming from Dreamwidth (going thru the Amazonses' server) is where the delay is occurring. Once it's delivered to Google it goes nearly instantaneously to the recipient. Obviously, Google isn't privvy to the information before it reaches Google.

However, I had a couple more questions and have asked my Google contact. It has to do with Amazonses in particular. When I hear back, I'll reply in this thread.

So it looks like you might have to have a chat with amazonses.

AmazonSES is Amazon's Simple Email Service (or something like that), which Dreamwidth and a lot of other sites use for sending large volumes of email. It's generally pretty reliable, though, and if it was affecting everyone using Amazon's cloud services and SES, we'd probably hear about it more widely.

It's confusing. Could the Google forum people be looking just at the last attempt, and not seeing the earlier attempts that got greylisted and delayed, because only the last one is coming through?

Honestly, I'm not entirely sure as I'm not at all technically inclined. I just did as asked, which was to c/p the 'show original' in the delayed notif and send it to them. I also included the original site link to the notif in question so they could see that there was indeed a delay of 2+ hours from when I got the DW notif on site/when it was posted to when the email came through on my end. There may be more info to cross, I don't know, but I'll just have to wait to hear back from the person helping on that site. I don't actually know if maybe this is greylisting for DW on AmazonSES's end, but this definitely all started on/around the most recent code push and it has gotten steadily worse since then, with it starting with small windows of notif delays to now where we have notif delays consistently for the last 2-3 days.

Other than what I'm doing, which is relaying info, I don't know what else to do. If anyone has any suggestions, I think we'd be happy to hear them.

For the last...thirty six hours or so? Closer to forty eight maybe? I've been seeing hours and hours of delays for notifs? Since it's on a back and forth post with one other person, I know it's not just me.

Anyone else?

Me too. Gmail as well.

Same here, along with my whole plurk list, apparently, with email providers that aren't just gmail.

Same here as well, and likewise with a lot of other folks on my plurk timeline. I'm hearing of delays of 24+ hours and can confirm delays of at least 21, going by my dw inbox.

On gmail, here; not sure about any of the other people I'm hearing reports from.

same here. not getting any notification.

The last email (I used Gmail) notification I received was Friday 8/5 at 7:30 AM CDT and I checked my character's DW inbox and saw a comment from 17 hours ago I have still yet to see an email for.

Yes, I'm on gmail as well.

same, was hoping for at least an update of being aware of the problem

I'm having the same issue with yahoo.

Anyone with GMail - which is pretty much everyone - isn't getting notifs.

I had a feeling it was the same Gmail issue, but some are reporting yahoo mail issues, too. *GROWL*

I'm having the same issue with my university email.

Page 2 of 2