dw_maintenance | (no subject)

(no subject)

Update on yesterday's post about importing: we are continuing to carefully babysit the importer to ensure the maximum number of people's import attempts succeed, and we'll continue to add more resources as necessary in order to keep it from affecting usage for the rest of the site as much as possible. Thank you all very much for your patience!

If you've been waiting for a slower moment to retry an import attempt that failed: now is a good time to retry, since the overall queue is pretty short! If you started an import job earlier today or yesterday and you haven't gotten a failure in your inbox yet: there are a few comment import jobs that we're trying to baby along because they involve a very very high number of comments or because the comment threads in the account tend to be very deeply nested or involve very complex response trees. Because we preserve threading data on import, in order to keep the imported entry looking as much like the original entry as possible, it takes a lot of working memory to do a comment import for large accounts or accounts with very complex comment threads. We've been tweaking the settings on how much memory an individual import worker is allowed to take up: usually all that memory management is handled automatically, but we're trying to find the "sweet spot" for allowing complex comment imports to succeed without running out of memory, taking up all the memory and leaving none for any of the other import workers running, or losing the safeguards for not letting a single process get stuck and just eat up more and more memory over time.

alierak has been putting in heroic levels of work on managing the importer queue these past few days: thanks, Robby!

And, of course, some jobs are still failing because LJ is still intermittently blocking us, allegedly because "too many password failures makes a block happen automatically because it looks like someone trying to break into accounts". So there's still a chance your import will fail because it was assigned to a worker operating on an IP that's currently blocked. We're doing everything we can to work around that issue, but there's only so much we can mitigate it. If your import fails, wait half an hour and try again. (So far, the unluckiest person I've seen who kept hitting blocked workers took seven tries for it to finally work, but it did finally work!)

For people asking about Scrapbook photos: unfortunately, LJ doesn't provide a feed that would let us import those.

lannamichaels has a method to at least download your photos, and

blue_ant reminded me that Semagic, the Windows-based LJ client, will also download your photos (and notes that you may need to keep trying a few times).

Finally, because I've seen a bunch of people making references to deleting their LiveJournal account: before you delete your LJ account, it's a good idea to claim your LiveJournal OpenID with your Dreamwidth account. Doing this will cause all of your imported comments (including in communities), and any entries you made to a community that's imported, to update to having been made by your DW account instead of your LiveJournal OpenID account. Doing this before you delete your LiveJournal account will let you keep managing any old comments in your journal, any comments in a community that's imported, and any entries you made in a community that's imported, exactly as though you'd posted them with your DW account, and avoid the need to authenticate against your LJ account, which you can't do after it's been deleted. If you've already deleted your LJ account, we strongly recommend temporarily undeleting it and following that process before you delete it again!

EDIT: I forgot a lot of people don't know what OpenID is, sorry! OpenID is one of the protocols that lets you use one site's login credentials to log onto another site without having to create a whole separate account. If you've ever wanted to buy something from a website once, didn't want to create a whole account for it, and instead made the purchase with your Facebook/Google/AppleID login: that's the same concept. (Not exactly the same protocol, but the same concept.)

When the importer imports comments (or entries in communities), it attributes all the comments to the commenter's LiveJournal OpenID. That way, the person who left the comment can log into Dreamwidth using their LJ account and still have the same level of control they had over the comment (or community entry) as they had on the LiveJournal version of the entry.

Claiming your LJ OpenID account with your DW account means that when other people (or communities) import their journals, you'll be able to control those comments with your DW account instead of having to log in using your LiveJournal OpenID -- which you can't do anymore once you delete your LiveJournal account. It makes sure you don't accidentally lose control over comments and entries that you left in other accounts if those accounts have already been/are later imported to DW.

Threaded | Top-Level Comments Only

Thank you so much! This is a little confusing to me, especially about claiming my LJ OpenID. I have no idea what that is.

This morning I attempted to import my LJ. I have a zillion :) LJ entries. Apparently, it imported all my entries, but after 4 attempts, it could not import the comments. Do I have to do the whole process again, or do I just try to import the comments again?

Here was the last error message:

Your comments have failed to import.
Failure running job: Lost connection to MySQL server during query at /dw/cgi-bin/DW/Worker/ContentImporter/Local/Comments.pm line 173.

I dont think I understand what OpenID is.

If you claim your livejournal comments can being logged in to livejournal somehow edit your dreamwidth?

If you delete your livejournal without claiming your openid and someone else claims your old livejournal name, can they then claim the openid comments with that livejournal name?

If you claim your livejournal comments openid and later someone else gets the livejournal name can that... do anything on dreamwidth?

Thank you all so much. I imported my LJ years ago but all this help you are giving now is so generous and wonderful.

I already imported my LJ (years ago) and I'm actually not planning to delete it (I don't use it anymore, but Google search works better there than here for reasons I've never been able to determine). I started the process to claim my openID and hit this warning:

>>>Claiming this account will merge all of the comments and entries that are currently owned by the OpenID account into your account. This action is immediate and irreversible -- we will not be able to unmerge them.<<<

What does this actually mean?? I have no idea what this is saying it will do.

OpenID is a protocol that lets you use one site's credentials to log onto another site without having to create an account -- if you've ever used your Facebook/Google/AppleID account to log into another site instead of creating an account, that's the same general concept (although those sites don't use the exact same protocol). When we import stuff, we attribute the comments to the commenter's LiveJournal OpenID so they can log into DW using their LJ OpenID, without having to create an account on DW, and have the same level of control over their comment they have on the LJ version of the entry. Claiming your LJ OpenID with your DW account means that, for instance, if your best LJ friend imports/has imported their LJ account to DW, you can control all of the comments you left for them back in 2005 on Dreamwidth, using your DW account, without having to log into DW with your LiveJournal OpenID. The reason to do this before you delete your LJ account is because once you delete your LJ account, we can't authenticate that you're the person who owned it, so there's no way for you to log into DW with your LJ OpenID anymore. Does that make more sense?

That error message means you hit one of the windows where LJ was blocking us: wait half an hour, try again. It may take a few attempts to succeed.

I was able to get everything imported. It took a few days, whatever you did last night was the sweet spot for the comments. It said they failed (like the bunch of other times), but they are all there, found out this morning when I came back to try again.

Thank you for working on this.

I had a quick, clean import today (though it's possible that my old, original import already handled everything and there wasn't actually anything left to bring over).

Some suggestions that might help other folks:
- I logged into LiveJournal first, and they made me change my password because it was "too old", so that could be a failure point for some folks.
- Another could be use of the Google password manager, which might be defaulting folks' DW password in the import-kickoff pages; I just made sure to paste the new LJ one at each checkpoint.

Thank you.

Do I need to import all my entries AND the comments again? The entries already came over to DW. It's just the comments that didn't.

I edited the post to include the explanation I provided above! OpenID is one of the protocols that lets you log into a site using your account from another site instead of having to create a new account.

Claiming your LiveJournal OpenID account with your DW account will not mean that being logged into LJ will do anything to your DW. I believe LJ still prevents accounts that someone renamed to from using OpenID, but I'm not 100% sure: if they have changed that, there is a small chance someone who renamed to your old LJ username could claim your LJ OpenID on DW, which is another reason to do it now before you delete your account. If you claim your LJ OpenID account with your DW account, and someone later renames their LJ to your former LJ username, there's nothing they can do to your DW account with it. All that claiming your LJ OpenID with your DW account will do is update imported comments and imported community entries so that they're owned by your DW account so you won't have to authenticate against LiveJournal anymore, which you can't do once you've deleted your account!

I don't think I ever set up my LJ to be used as an OpenID account - would they have made it one automatically?

I imported my LJ to DW _years_ ago now, and rarely ever go back to LJ. I don't want anything about my DW to change at this point. If we don't intend to delete our LJs, is there really any point in doing this claiming thing?

Say you left a comment on your best friend's journal in 2005. Bestie has since imported their journal to DW. If you don't claim your LJ OpenID with your DW account, you would need to log into DW using your LJ account in order to manage that comment, which you can't do anymore once you delete your account. Claiming your LJ OpenID with your DW account means that comment in Bestie's journal will look like it was made by

naomikritzer instead of naomikritzer.livejournal.com, you'll get emailed notifications if someone replies to it, and you'll be able to edit it or delete it without having to log into your LJ account and then use that LJ account to log into DW (which you can't do anymore once your LJ account is deleted).

The warning is just making sure you really really want us to update every imported comment/entry that looks like it comes from naomikritzer.livejournal.com to look like it comes from

naomikritzer instead! As the message says, there's no way we can undo it, so we just triple-check.

If you want a better example: check out my initial announcement that DW was going to be a thing. That entry was originally posted to LJ, and imported to DW. The first comment thread underneath the post, those comments belong to

zorkian and

synecdochic: both Mark and I claimed our LJ OpenID accounts, so even though we originally made those comments on LJ, they look like they were made by our DW accounts (and we can now manage them with our DW accounts). The second comment thread was started by

svmadelyn, who hasn't claimed their LJ OpenID with their DW account, so it still displays as

svmadelyn.livejournal.com and they would have to first log into their LJ account, then use that LJ account to log into DW, in order to manage the comment.

Selecting the checkbox to import comments automatically selects the checkbox to import entries: this is because otherwise, if it had been a while since your last entries import, there could potentially be comments that should go on entries that hadn't been imported yet. The data source from LJ that we use for importing does let us just say "yo, are there any new entries since [date of last import]?" and if LJ says "nope, there aren't" we skip straight to the comments stage: it doesn't try to reimport everything from scratch from the beginning.

Thank you for explaining

Oh, good call, I forgot that LJ was doing the "change your password" thing. (Hilariously, except it isn't really hilarious at all, I just have to laugh because otherwise I'll scream: it's not because your password is 'too old', it's because your password was compromised in the massive LiveJournal security breach they have never admitted happened, much less disclosed to their users. If you used that same password anywhere else on the internet, for any other account, anywhere, go change it there, too: the dumpfile is now widely circulating and it's trivially easy for someone to get a copy of it.)

Hi Denise! Thanks for this update, especially on OpenID!

I have a question. Is it possible to import a community that I'm a moderator and maintainer of on LJ, but not the owner? I tried, but it asked me for a password. I'm not sure if it meant my snacky LJ password, or the password for the community, which I don't know and am not sure if it ever had one.

That might be better than my original suspicion -- that they force-expired passwords when they changed the TOS to be "Russian-language version is the only binding version". (Which I considered only because I hadn't signed into their site since they wanted me to accept that new TOS. Today I did, only to get access for long enough to export/delete.)

"Better" does not mean "remotely good", of course ...

Oh dang, I did not know OpenID let you hang onto posts that way! (Semi)tragically, I imported years ago, so it's way too late for me to go back and claim my LJ OpenID, but I love it anyway.

You don't need to 'set up' anything for your LiveJournal account to function using OpenID. It's exactly like if you want to buy something from a site that you know you don't intend to return to so you don't want to create an account for that site and just log in using your Google account instead! When we import comments (and community entries) from LiveJournal, we show them as having come from LiveJournal, so people can still manage them with their LiveJournal accounts (by logging into DW with their LJ account).

The reason to claim your LJ OpenID with your DW account would be to make sure you can still manage those imported comments and community entries without having to log into LJ. For example, if you made a post to a community on LJ in 2005, and that community imports all its entries to DW, the DW copy of the post would look like it was made by katuah.livejournal.com (or whatever your LJ username was). If you wanted to edit or delete it, you'd have to log into LJ, and then use your LJ to log into DW. If you claim your LJ OpenID with your DW account, that entry would look like it was posted by

katuah, and you'd be able to edit or delete it with your DW account.

I know this is a bit confusing, so an example: here is the post I made to my LJ account first announcing Dreamwidth. That post was made on LJ, and I imported it to DW. The first comment thread on the post shows as having been made by

zorkian and

synecodochic: both Mark and I made those comments on LJ, using our LJ accounts. We both claimed our LJ OpenID accounts with our DW accounts, so now we can manage those comments while logged into our DW accounts if we want to. The second comment thread was made by

svmadelyn. They haven't claimed their LJ OpenID with their DW account, so it shows as having been made by

svmadelyn.livejournal.com. If they wanted to manage that comment, they'd have to first log into their LJ account and then use their LJ account to log into DW in order to manage it.

If you don't intend to delete your LJ, the only reason to do it would be convenience! But it's a good idea anyway, especially if you participated heavily in communities on LJ, just to make sure you'll always be able to control the imported DW copy of comments/community entries that you left ages ago.

I am not 100% confident in this answer, because I haven't kept up on any changes LJ made to how comms work, but I'm 99% sure that any maintainer (but not any moderator) of a community can import it. The password it's asking for is the password to the LJ account that you told it is a moderator of the community.

Yeah, sadly, if you deleted your LJ account long enough ago that it's been purged, there's no way to authenticate against it in order to claim your OpenID. Which is why I'm glad I remembered to warn people this time!

Yeah, the "new ToS" thing they did was just making you log into the account and confirm that you agree to the new ToS; they didn't make you change your passwords for that. The forced password changes are because they couldn't keep pretending someone got their entire user database anymore after we disclosed it to our users because too many people had the same password for their LJ and DW account and we were going to have to force-reset passwords because bad guys were exploiting them for spamming purposes.

Thank you, Denise! It's all finished now. ♥

Oh, thank you! I am going to try that PW. I think that's where I was getting stuck with the importer.

Totally worked! Thank you again!

Good to hear!

Threaded | Top-Level Comments Only

Page 1 of 5