Resolved problem with emailed notifications

[karzilla]

Sorry for the delayed response on the issue with emailed comment and support notifications not being delivered. We weren't receiving our emailed comment and support notifications saying that there was a problem...

We restarted the relevant software services around 10AM EDT, and the backlog of delayed notifications should have all been sent by now. Thanks for your patience, and extra thanks to everyone who tried to let us know about the problem!

Comments

[brave_heart_verity]

[alexseanchai]

Yay working again!

[dickgrayson]

Thank you! :D

[dagbok]

Thank you!

[mildred_of_midgard]

Haha, that reminds me of this: https://twitter.com/devops_borat/status/291243959370518528

DevOps Borat

90% of devops are fail simple interview question of what is most critical piece of infrastructure. Is outgoing mail on Nagios server!

Glad it's fixed!

[wohali]

Thanks for fixing things! Have you considered adding something like PagerDuty or Twilio (or even an IRC bot) to provide an alternative outgoing notification path?

[denise]

We have plenty of alerts, the length of the mail queue was just not one of them :)

[kareila]

I think alierak just starting monitoring the rate of outgoing emails and set an alert for when it drops to zero...

[cxcvi]

"We didn't get the email saying that email is down because email was down."

Glad to have it back, though.

[lurkerwolf]

Glad it's fixed now. It's understandable it took time to fix it when not even you guys got notifications.

[kore]

We weren't receiving our emailed comment and support notifications saying that there was a problem...

Oh, DEAR.

[rydra_wong]

We weren't receiving our emailed comment and support notifications saying that there was a problem...

BUT WE WERE WAVING. WHY DID YOU NOT SEE US WAVING.

(Thanks for getting it fixed!)

[azurelunatic]

It slowly dawned on me that I hadn't seen any official notice taken, but the clue took a while to arrive in the post-surgical haze.

Perfect storms and all that.

[hokuton_punch]

Thank you so much! My inbox was feeling like a sad, empty ghost town. I'm glad it was a quick fix.

[jazzyjj]

No wonder my work email inbox just saw an influx of these requests, *grin*. Glad things are fixed though for those who were affected. I'm off to bed now. Shutting this thing down in...

[sarasa_cat]

Thanks for fixing this!

[gwendraith]

Thank you for fixing it promptly as soon as you knew there was a problem :)

[filialucis]

Thanks for sorting it out!

[mirrorofsmoke]

We weren't receiving our emailed comment and support notifications saying that there was a problem
Oh man, that's really kind of meta isn't it? Maybe that's the wrong word, but thank you for the fix. I was about to go leave a support request for it when I found that hey a bunch of people beat me to it. :D