Mark Smith (![[staff profile]](https://www.dreamwidth.org/img/silk/identity/user_staff.png)
mark) wrote in ![[site community profile]](https://www.dreamwidth.org/img/comm_staff.png)
dw_maintenance2012-04-28 05:22 pm
mark) wrote in dw_maintenance2012-04-28 05:22 pm
Site slowness expected shortly
Hi all,
Summary: some unexpected but required maintenance is going to cause the site to slow down by some amount for the next hour or two.
Some details: We have run into an issue with one of our larger communities that requires us to make a change to the way we store comments in memcache. This means that our caches are all going to invalidate here in a few minutes when I roll the code out, so the databases are going to get very busy until we repopulate the caches.
Those of you who have been on LJ for a few years might remember when ONTD ran into an issue when they hit 16,777,215 comments and LJ had to do the same thing. I helped LJ with the code to fix the problem, but for some reason, never committed it on Dreamwidth. Now we have hit the same issue and I feel silly for having not done that, but here we are, and hence this maintenance.
Please let me know if you see any major problems, and again, sorry for the inconvenience. This really isn't a great time to be doing site maintenance.
Update: Congratulations to![[community profile]](https://www.dreamwidth.org/img/silk/identity/community.png)
sixwordstories for being the first community or journal to hit the 16.7 million comment limit in the database. :)
At any rate, currently the community is being migrated to a database where I fixed up the column widths so they won't have this limit anymore. It will take some time to move that much data, though. In the meantime the community will be in read-only mode.
I'm sorry that we hit this -- I really thought we had made this fix on Dreamwidth after LJ ran into it. It turns out we didn't, though, hence this situation. Alas.
Summary: some unexpected but required maintenance is going to cause the site to slow down by some amount for the next hour or two.
Some details: We have run into an issue with one of our larger communities that requires us to make a change to the way we store comments in memcache. This means that our caches are all going to invalidate here in a few minutes when I roll the code out, so the databases are going to get very busy until we repopulate the caches.
Those of you who have been on LJ for a few years might remember when ONTD ran into an issue when they hit 16,777,215 comments and LJ had to do the same thing. I helped LJ with the code to fix the problem, but for some reason, never committed it on Dreamwidth. Now we have hit the same issue and I feel silly for having not done that, but here we are, and hence this maintenance.
Please let me know if you see any major problems, and again, sorry for the inconvenience. This really isn't a great time to be doing site maintenance.
Update: Congratulations to
sixwordstories for being the first community or journal to hit the 16.7 million comment limit in the database. :)At any rate, currently the community is being migrated to a database where I fixed up the column widths so they won't have this limit anymore. It will take some time to move that much data, though. In the meantime the community will be in read-only mode.
I'm sorry that we hit this -- I really thought we had made this fix on Dreamwidth after LJ ran into it. It turns out we didn't, though, hence this situation. Alas.
![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
Page 1 of 3