|Denise (denise) wrote in dw_maintenance,|
@ 2012-02-09 11:08 pm UTC
So, if we're transfering record amounts of data, and if (as mentioned on our offsite status twitter) we added two new webservers today to help push out traffic, why has the site been so sluggish today?
As I mentioned in the 2 Feb dw-news post, the answer is very complex. There are a lot of underlying causes that can look to you guys (the people who are just trying to load your reading page and comment on various posts) like the site is just plain sucking, and I know it must be tempting to wonder: hey, this keeps happening, why can't they just fix it?
Today (the answer changes from day to day, and sometimes from hour to hour), the biggest slowdown comes from entries with large numbers of comments, because the code that generates those pages has a bunch of inefficiencies that don't start showing up until you see a whole lot of people loading those entries and interacting on them -- the code that generates the page takes time to run, and every additional microsecond it takes, it ties up a process on the webserver it's running on. The webservers can only run so many processes at once, so when there's no free webserver processes available, you ask the site for the page and it has to yell "Hang on a second, I'll get to you as soon as I've got a process free!" at you. Even if you aren't viewing or commenting on an entry with lots of comments, somebody else is, and so the servers wind up being busy. (This is like how if you walk into the deli and just want a root beer, you still have to wait in line with all the people who are buying sandwiches.)
We've known this was going to be a problem for a while, but fixing the underlying cause is going to take a lot of work, because there are a bunch of tiny inefficiences that add up to big problems when they're taken all together.
mark has been making a bunch of code fixes that will speed things up in the short term, and allen is working on the more sweeping code changes that will speed things up in the long term. The code changes Mark just made should help a lot, but high traffic periods (evenings, US time) may continue to be sluggish for a few more days.
So, in short: We're busy! We're all really sorry about the slowdowns at peak traffic times, and we're working really hard to increase capacity and speed up site performance. Thank you all for your patience (you really are the best users a site ownership team could ask for!) and for continuing to use Dreamwidth. This is a very exciting problem to have. :)