Denise (![[staff profile]](https://www.dreamwidth.org/img/silk/identity/user_staff.png)
denise) wrote in ![[site community profile]](https://www.dreamwidth.org/img/comm_staff.png)
dw_maintenance2011-12-30 07:53 pm
denise) wrote in dw_maintenance2011-12-30 07:53 pm
Site slowdowns and internal server errors
Hello, my lovely Dreamwidthians! I bring to you information about several site issues this afternoon/evening/other-time-of-day-as-appropriate-for-your-timezone, and the steps we're taking to fix them:
* Some people have noticed intermittent site slowdowns today. This is partially because of increased traffic, and partially, as we feared, because the import queue, and the speed at which we're processing import jobs, is putting a lot of load on the database servers. Short-term fix:mark is juggling things around to reduce database load and trying to direct some queries to the backup database machine. Long-term fix: we have a ticket in with our hosting provider to upgrade the database machines to make them bigger, better, faster, and more.
* You may get some "internal server error" messages when submitting data to the site, or messages saying that the site sent no data or the connection was reset. This is almost certainly related to the new load balancing solution we put into place last night, and Mark is doing some configuration tweaks and fixes that will hopefully stop it from happening. We're not 100% sure on the exact cause -- there are a few things it could be -- but we're going to keep whacking at it until the errors go away.
As part of these fixes, Mark may need to take the site down for a brief maintenance window -- he's hoping he'll be able to make the fixes without having to put the site into maintenance mode, but he might not be able to. The site may be slow for a while in the next few hours, and there may be brief periods of downtime. We're working to fix the problems as fast as we can!
(Well, okay, Mark is working to fix the problems. I'm cheering him on and passing him towels and Gatorade.)
* Some people have noticed intermittent site slowdowns today. This is partially because of increased traffic, and partially, as we feared, because the import queue, and the speed at which we're processing import jobs, is putting a lot of load on the database servers. Short-term fix:
mark is juggling things around to reduce database load and trying to direct some queries to the backup database machine. Long-term fix: we have a ticket in with our hosting provider to upgrade the database machines to make them bigger, better, faster, and more.* You may get some "internal server error" messages when submitting data to the site, or messages saying that the site sent no data or the connection was reset. This is almost certainly related to the new load balancing solution we put into place last night, and Mark is doing some configuration tweaks and fixes that will hopefully stop it from happening. We're not 100% sure on the exact cause -- there are a few things it could be -- but we're going to keep whacking at it until the errors go away.
As part of these fixes, Mark may need to take the site down for a brief maintenance window -- he's hoping he'll be able to make the fixes without having to put the site into maintenance mode, but he might not be able to. The site may be slow for a while in the next few hours, and there may be brief periods of downtime. We're working to fix the problems as fast as we can!
(Well, okay, Mark is working to fix the problems. I'm cheering him on and passing him towels and Gatorade.)
no subject
Sorry, I get so annoyed when companies have problems and don't take the time to let me know what's going on. I really want people who use Dreamwidth to have the feeling that this is their site, too, and so they should expect to know what's going on with it.
Particularly if they're helping support us financially. That's just a no brainer.
Anyway, a rant for another day...
no subject
but having run even a one should have read: but having run even a small one
and as a new user i thank you!!!
i am a recent LJ refugee and i am SO impressed by the openness here and communication about problems and willingness to listen to users/customers, it's just WOW!! I got myself a premium account!! i could no longer pay for a site that had zero customer service....i will happily pay more for a site that provides more customer service and is still growing at a rate they can manage....
seriously, THANK YOU!!!
no subject
no subject
<3
My husband works as a sysadmin so I hear the "behind the scenes" rants, and so I usually understand WHY sites can't always tell us what's going on. I understand sometimes it's that it's policy, and sometimes it's literally that there just isn't the time.
I can't say how much I appreciate that you guys take the time to keep us in the loop, even for things that are relatively minor. I have been on other sites, quite a lot of them, that don't address issues until they can have a PR team or something address it (or at least I think that's what's going on?), and it gets really frustrating knowing things are broke and not knowing if it's being addressed (because even though if it's that broke, it probably is being addressed, we don't know for sure, and the acknowledgement is great).
You guys go way above and beyond what any other site I've seen has done, and I for one really appreciate it. Thank you. :)