database maintenance

[mark]

Hi all,

Over last night and this morning I've done some database maintenance on our global slave. This should be transparent and cause no problems, but please let me know if you see anything amiss.

For the technically curious, I had to do a dump-and-reload on our MySQL instance so that I could free up space in the InnoDB data file. It had grown to ~240GB back when the machine was also home to the user data, but now that it is just used for global data, it is only about 8GB.

After things are settled for a few days, I will have to perform the same operation on the global master. That will be slightly more exciting, but should cause no downtime either.

Comments

[greyduck]

I like that "slightly more exciting." *grin*

[sgsabbage]

I had to do the exact same thing on one of our servers this week, it's rather irritating that the data file can't be shrunk without dropping the databases. Turning on the file-per-table option seems to make it a little more maintainable, though.

[sharpiefan]

Don't know if it's due to this, but I'm having notification delays...

[mark]

Yeah.

My original plan was to dump and take down our 5.0 slave, make backups, then upgrade to Percona 5.5 and enable innodb_file_per_table. I was thwarted when it became apparent that our Ubuntu 9.04 machines aren't compatible with the Percona 5.5 packages -- too old.

I didn't want to leave the database down long enough to do a full system reinstall, and frankly don't have the time today, so I brought it back up on the old 5.0 and didn't set the options. (I seem to recall something weird with that option in 5.0, or maybe it just doesn't work? I don't know.)

Anyway, hopefully sometime Soon(TM) I can find time to upgrade to 10.04 ... of course, 12.04 is coming out very soon. I don't know that I want to trust the first release on our databases, though. That's probably unwise.

[mark]

I just checked, we don't have any backlog of notifications or emails right now. It's probably a transient email issue and will self-resolve.

[sharpiefan]

OK. Hope so! Thanks for looking at it!

[dmsj]

It would appear that things are not crossposting to LJ properly. I'll keep poking around at my entry, but I can't *see* anything wrong. When I go in to change it, the box is unchecked. I check it, click Save, and it says it's crossposting.... but it doesn't, and my Inbox tells me, "Crosspost of Blogged to dawnstar@LiveJournal action failed.
Crosspost to dawnstar@LiveJournal failed.

Failed to crosspost entry to dawnstar@LiveJournal: Failed to connect to http://www.livejournal.com/interface/xmlrpc."

Is that on our end or theirs?

[discodiva76]

Live Journal is down atm....

[mark]

Yeah, LJ is currently broken. Sorry. :(

[swordznsorcery]

Having a global slave sounds fabulously decadent; and a global master rather brings to mind Ming the Merciless.

This is creating some very interesting mental pictures of DreamWidth HQ. ;)

[wytchcroft]

why be sorry - did you break it? nah, but go on - just for once, let's have an EVIL LAUGH! i don't know, you guys always making with the nice... LOL.

[0jack]

LJ is just jealous of our server troubles and has gone offline so it can be better than us at something—namely BEING BROKEN. Don't worry, LJ. You were always better at that, even though I love you.

[apis_mellifera]

So glad I am not the only one thinking that.

[ariandar]

Seconded!

[ccommack]

Probably a dumb question, but: is there a particular reason why 11.04 and 11.10 are worse than 10.04? Not been around long enough to have the rock-solid stability DW wants and needs?

[mark]

Never a dumb question!

We're running 9.04 right now which is about three years old. It's no longer supported, so it puts us in a bind and makes it hard to upgrade and receive security updates, etc. Canonical (the folks behind Ubuntu) have structured things such that the LTS releases (8.04, 10.04, 12.04, etc) are supported for five years -- but the non-LTS releases are only supported for two years.

Ideally, we should be on an LTS version or we should commit to upgrading releases more often than we do.

[tameiki]

This is creating some very interesting mental pictures of DreamWidth HQ

*doubles over laughing* OMG. *gasps for breath*

Yeah, I had a thought along similar BDSM lines when I read that! =D

[mark]

We are a friendly, happy, open, awesome company with an awesome web site and hella rad users. But there are still certain things I won't talk about with my staff account. ;)

[tameiki]

We are a friendly, happy, open, awesome company with an awesome web site...

That, you are, Mark! And I don't blame you for not going there. *snickers*

[dragondancer5150]

No idea if this is related or not, but I'm trying to get to and copy the link/web address for one thread on a post. I get email notifications, and I used the "Go to the top of the thread this comment is part of" link in the email. That takes me to the thread, but in the navigation field of my browser, I'm only getting "about:blank" instead of the web address. Going to the original post, and then clicking on "link" of the top comment of the thread also successfully gets me just the thread, but now it doesn't change the web address from the "base" to the one with the..."extension"? - not sure that's the right word - for just the thread.

If it helps, the post I'm talking about is - http://dear-mun.dreamwidth.org/1527859.html - and I'm trying to get the address for /just/ the thread that starts with "derpjack_ftw".

No idea if this has to do with the maintenance or not, but since you said "if you see anything amiss", and I've never had this problem before.

Thanks!

[denise]

That definitely sounds like something with your browser! Here's your thread: http://dear-mun.dreamwidth.org/1527859.html?thread=43278643#cmt43278643

[dragondancer5150]

Huh. I'll have to see if I can figure it out then.

Thanks!!

[jae]

I have no idea what that first sentence means, but I find it hilarious. Thank you for the amusement! :)

-J

[kuzimama]

good job, keep it simple :))))))))))))))))))

[aoyagi-rikka.livejournal.com]

I don't know if this is related, but I had some trouble using my OpenID login. LJ is my main OID identity, but I tried using my Google URL also and it still gave me an error along the lines of "website is not authenticating." Obviously, it eventually let me it, but if other people are having the same problem they may not be able to report it as no login = anon = can't reply here.

Re: Newbie here...

[mark]

Sorry, I don't know what that domain is or who runs it.