Long-standing notification issue fixed

[mark]

While working on the notifications system, I discovered the cause of an issue that has been around for a while that caused a number of users to be unable to receive some notifications. In the worst cases, you would never get an email or inbox notification of new comments, no matter how many times you toggled it on.

This is now fixed.

The story is kind of gory but the short version is that a couple of years ago when we created a new database cluster and moved a bunch of users to it, some subscriptions got 'left behind' on the old cluster. Our subscriptions and notifications system was seeing these old notifications, but later in the process when attempting to generate a notification, was throwing them out because they didn't match the user's 'correct' cluster.

I've pruned the notification rules that were on the incorrect cluster for all users (about 1.1 million notification rules, actually) and this should resolve the issue where notifications didn't work in a number of cases.

Comments

[pronker]

*whoosh, info goes over head* but thanks for your work. :)

[havocthecat]

Yikes! But also, wow, thanks for fixing that.

[ilyena_sylph]

+applauds you some more, with an adorable monster!+

[kore]

Thank you for the information and the transparency!

[althea_valara]

Congrats on fixing a long-standing bug!

Are notifications being delayed right now due to this? Because I noticed i haven't received a notification for this comment in either my DW inbox or email.

[alexseanchai]

\o/

[veritas_poet]

I've very rarely gotten notifications. I gave up expecting them. But no one reads my journal anyway these days, so I guess it doesn't matter.

[sovay]

Are notifications being delayed right now due to this? Because I noticed i haven't received a notification for this comment in either my DW inbox or email.

I have also just received a comment without any form of notification.

[edit] I have now received a second comment without either e-mail or inbox notification. I am used to seeing both.

Thanks!

[echan]

Huzzah!

And, separately, I greatly appreciate the details you provide when things are borked and/or fixed.

[rydra_wong]

Thirding ...

[trobadora]

Same here.

I have two comments on a locked post, but only received a notification for one of them. The other is missing both via email and in the DW inbox.

[asymbiotic]

Right now I'm not receiving notifications on at least this account (I've received two of an estimated 15 in the last two hours, both from the same account oddly enough) and several other people in rp-land are reporting no notifications either.

[casimirian]

Wooooo! Awesome job!

[casimirian]

If you haven't yet, try finding journals to read and people to share your journal with this community. :)
https://addme.dreamwidth.org

Root cause analysis?

[dennisgorelik]

> was throwing them out because they didn't match the user's 'correct' cluster

Did you investigate why (and when) your system was programmed that way?

The correct behavior in the situation when code processes inconsistent data -- is to notify developers about unexpected situation that developers have to revisit and recode.

If code silently ignores problems - it leads to years of misdiagnosed misbehavior.

[krait]

Congrats on tracking down the notification issue! I know it's been bugging (heh) people for a while now.




I am currently having an issue where trying to post a poll returns the message: Error updating journal: Poll error: All poll-question (or lj-pq) tags must be nested inside an enclosing poll (or lj-poll) tag.

I have opened a support ticket, but I'm really hoping there's a faster way to figure out what's going on, because the pole in question is kinda time-relevant (it's for holiday cards). Is there somewhere I can check to see if others are having the problem and find a solution?

[veritas_poet]

Thanks.

[momijizukamori]

Hmm, I was just able to create a poll without any issue - if you add the code that's giving you the error to the support ticket, we might be able to do more debugging.

(in a pinch, I know a lot of people use google docs forms for holiday card info-gathering)

[pauamma]

Could that (or something like it) also be the cause of the double notifications I (and IIRC some others) am getting (in both email and my DW inbox) for some things, most notably newsposts?

[kalloway]

I get double newsposts but that's it, weirdly.

[graildiary]

what if we still have accounts that are not receiving notifs, or accounts getting double notifs? i have filed support tickets about these issues in the past and received no response :/

[darjeeling]

This problem has been plaguing some of my RP journals for over two years, I could kiss you right now for having fixed it. Thank you!!

[mark]

Haha, well thanks :)

Have you gotten a notification from one of them? Just to prove it's working?

[darjeeling]

Yes! One of my affected journals was grimreaping and I just dropped a test comment on one of the posts there, and it came through immediately to my email as normal! Before, it wouldn't go to either the DW inbox or my email. I'd given up on using the journal again because there was no way to track my game tags.

[mark]

Boom, glad it works now! Yay!