Denise (![[staff profile]](https://www.dreamwidth.org/img/silk/identity/user_staff.png)
denise) wrote in ![[site community profile]](https://www.dreamwidth.org/img/comm_staff.png)
dw_maintenance2022-03-11 05:52 pm
denise) wrote in dw_maintenance2022-03-11 05:52 pm
(no subject)
Update on yesterday's post about importing: we are continuing to carefully babysit the importer to ensure the maximum number of people's import attempts succeed, and we'll continue to add more resources as necessary in order to keep it from affecting usage for the rest of the site as much as possible. Thank you all very much for your patience!
If you've been waiting for a slower moment to retry an import attempt that failed: now is a good time to retry, since the overall queue is pretty short! If you started an import job earlier today or yesterday and you haven't gotten a failure in your inbox yet: there are a few comment import jobs that we're trying to baby along because they involve a very very high number of comments or because the comment threads in the account tend to be very deeply nested or involve very complex response trees. Because we preserve threading data on import, in order to keep the imported entry looking as much like the original entry as possible, it takes a lot of working memory to do a comment import for large accounts or accounts with very complex comment threads. We've been tweaking the settings on how much memory an individual import worker is allowed to take up: usually all that memory management is handled automatically, but we're trying to find the "sweet spot" for allowing complex comment imports to succeed without running out of memory, taking up all the memory and leaving none for any of the other import workers running, or losing the safeguards for not letting a single process get stuck and just eat up more and more memory over time.![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
alierak has been putting in heroic levels of work on managing the importer queue these past few days: thanks, Robby!
And, of course, some jobs are still failing because LJ is still intermittently blocking us, allegedly because "too many password failures makes a block happen automatically because it looks like someone trying to break into accounts". So there's still a chance your import will fail because it was assigned to a worker operating on an IP that's currently blocked. We're doing everything we can to work around that issue, but there's only so much we can mitigate it. If your import fails, wait half an hour and try again. (So far, the unluckiest person I've seen who kept hitting blocked workers took seven tries for it to finally work, but it did finally work!)
For people asking about Scrapbook photos: unfortunately, LJ doesn't provide a feed that would let us import those.lannamichaels has a method to at least download your photos, and blue_ant reminded me that Semagic, the Windows-based LJ client, will also download your photos (and notes that you may need to keep trying a few times).
Finally, because I've seen a bunch of people making references to deleting their LiveJournal account: before you delete your LJ account, it's a good idea to claim your LiveJournal OpenID with your Dreamwidth account. Doing this will cause all of your imported comments (including in communities), and any entries you made to a community that's imported, to update to having been made by your DW account instead of your LiveJournal OpenID account. Doing this before you delete your LiveJournal account will let you keep managing any old comments in your journal, any comments in a community that's imported, and any entries you made in a community that's imported, exactly as though you'd posted them with your DW account, and avoid the need to authenticate against your LJ account, which you can't do after it's been deleted. If you've already deleted your LJ account, we strongly recommend temporarily undeleting it and following that process before you delete it again!
EDIT: I forgot a lot of people don't know what OpenID is, sorry! OpenID is one of the protocols that lets you use one site's login credentials to log onto another site without having to create a whole separate account. If you've ever wanted to buy something from a website once, didn't want to create a whole account for it, and instead made the purchase with your Facebook/Google/AppleID login: that's the same concept. (Not exactly the same protocol, but the same concept.)
When the importer imports comments (or entries in communities), it attributes all the comments to the commenter's LiveJournal OpenID. That way, the person who left the comment can log into Dreamwidth using their LJ account and still have the same level of control they had over the comment (or community entry) as they had on the LiveJournal version of the entry.
Claiming your LJ OpenID account with your DW account means that when other people (or communities) import their journals, you'll be able to control those comments with your DW account instead of having to log in using your LiveJournal OpenID -- which you can't do anymore once you delete your LiveJournal account. It makes sure you don't accidentally lose control over comments and entries that you left in other accounts if those accounts have already been/are later imported to DW.
If you've been waiting for a slower moment to retry an import attempt that failed: now is a good time to retry, since the overall queue is pretty short! If you started an import job earlier today or yesterday and you haven't gotten a failure in your inbox yet: there are a few comment import jobs that we're trying to baby along because they involve a very very high number of comments or because the comment threads in the account tend to be very deeply nested or involve very complex response trees. Because we preserve threading data on import, in order to keep the imported entry looking as much like the original entry as possible, it takes a lot of working memory to do a comment import for large accounts or accounts with very complex comment threads. We've been tweaking the settings on how much memory an individual import worker is allowed to take up: usually all that memory management is handled automatically, but we're trying to find the "sweet spot" for allowing complex comment imports to succeed without running out of memory, taking up all the memory and leaving none for any of the other import workers running, or losing the safeguards for not letting a single process get stuck and just eat up more and more memory over time.
alierak has been putting in heroic levels of work on managing the importer queue these past few days: thanks, Robby!And, of course, some jobs are still failing because LJ is still intermittently blocking us, allegedly because "too many password failures makes a block happen automatically because it looks like someone trying to break into accounts". So there's still a chance your import will fail because it was assigned to a worker operating on an IP that's currently blocked. We're doing everything we can to work around that issue, but there's only so much we can mitigate it. If your import fails, wait half an hour and try again. (So far, the unluckiest person I've seen who kept hitting blocked workers took seven tries for it to finally work, but it did finally work!)
For people asking about Scrapbook photos: unfortunately, LJ doesn't provide a feed that would let us import those.
lannamichaels has a method to at least download your photos, and blue_ant reminded me that Semagic, the Windows-based LJ client, will also download your photos (and notes that you may need to keep trying a few times). Finally, because I've seen a bunch of people making references to deleting their LiveJournal account: before you delete your LJ account, it's a good idea to claim your LiveJournal OpenID with your Dreamwidth account. Doing this will cause all of your imported comments (including in communities), and any entries you made to a community that's imported, to update to having been made by your DW account instead of your LiveJournal OpenID account. Doing this before you delete your LiveJournal account will let you keep managing any old comments in your journal, any comments in a community that's imported, and any entries you made in a community that's imported, exactly as though you'd posted them with your DW account, and avoid the need to authenticate against your LJ account, which you can't do after it's been deleted. If you've already deleted your LJ account, we strongly recommend temporarily undeleting it and following that process before you delete it again!
EDIT: I forgot a lot of people don't know what OpenID is, sorry! OpenID is one of the protocols that lets you use one site's login credentials to log onto another site without having to create a whole separate account. If you've ever wanted to buy something from a website once, didn't want to create a whole account for it, and instead made the purchase with your Facebook/Google/AppleID login: that's the same concept. (Not exactly the same protocol, but the same concept.)
When the importer imports comments (or entries in communities), it attributes all the comments to the commenter's LiveJournal OpenID. That way, the person who left the comment can log into Dreamwidth using their LJ account and still have the same level of control they had over the comment (or community entry) as they had on the LiveJournal version of the entry.
Claiming your LJ OpenID account with your DW account means that when other people (or communities) import their journals, you'll be able to control those comments with your DW account instead of having to log in using your LiveJournal OpenID -- which you can't do anymore once you delete your LiveJournal account. It makes sure you don't accidentally lose control over comments and entries that you left in other accounts if those accounts have already been/are later imported to DW.
no subject
If the page shows success, you are good! it may take a bit of time for all the comments and entries your OpenID 'account' posted to be shifted over to your DW account (looking up all of them is really hard on the database, so we do it veeeeeeeerrrrrrry slowly in the background) but there's nothing more you need to do yourself; the site will do the rest of it for you.