Imported content removal tool

[denise]

For people who got bitten by LJ's bug that was causing duplicate content in your account after importing content from LJ, we've finished the tool that will help you clean up your account without having to delete each individual post or comment.

This tool will remove all imported content from your account, not just duplicate imported content. (Basically, there's no way for us to tell which version of a post or comment is the 'original' and which is the duplicate, so we need to remove them both and let you re-import from scratch.) Once you've removed all imported content, you can then re-import your content from scratch.

Warnings: This can't be undone. Once you hit the button to delete all imported content, it can't be stopped. It will remove all imported content from your account, no matter how long ago you imported it. If you've deleted posts or comments on LJ since you first imported, they can't be re-imported into your account on Dreamwidth after you remove all imported content.

Reassurances: It will only touch imported content (with the exception of: If an entry was imported, and then someone commented on that entry on Dreamwidth, running this tool will delete that entry and all associated comments, including any comments that were made on Dreamwidth.) We've tested it pretty heavily (although there's always a chance for weird bugs).

If you want to use the tool to clean up your account, it's here.

EDIT: Also, I forgot to mention: if you're one of the people who were affected by the issue with the system telling you that your comments had imported successfully but the comments weren't appearing in your account, you can now try your import again, and it should work this time!

Comments

[sasha_feather]

YAY!!!

[cyprinella]

Can you specify which service to delete? I've got posts imported from Journalfen when that was still a possibility that I wouldn't want to go since we can't import from there any more.

[denise]

I'll have to double-check, but I don't think you can. (You might want to talk to Journalfen's admins -- the thing that is keeping us from re-enabling them as an import source is a really minor thing that they can fix, and one of our team members has given them the instructions, they just need to follow the steps!)

[rowrbazzle]

Thank you.

THANK YOU.

THANK YOU.

[wallflowerish]

Yay! Thank you!

[weber_dubois22]

Is this is a natural glitch that occurs during importing, or something LiveJournal does under the assumption that it's aware its users are moving information from their site to dreamwidth.org?

[denise]

It has to do with LJ having changed how they number and identify posts from their APIs and backend tools (so, affecting all tools that work with that particular data feed from LJ, not just us). It isn't conspiracy. :) (It's just frustrating because it doesn't seem to be consistent -- I'm actually guessing it's a bug, but it makes it really hard for us to work around it.)

[msilverstar]

I'm having a hell of a time trying to import a community, is that even working right now? I know the livejournal password is right, because I can log in with it, but I keep getting errors.

[denise]

Yes, it should be working. If you tell me what the errors are, I can help out a little more. :)

[jordannamorgan]

Just so I'm perfectly clear on this: comments can be imported from LJ without problems now? I'm one of those who was able to import everything else recently, but comments didn't work, so that's what I've been waiting to hear a specific green-light on.

[denise]

The overall problem should have been fixed last week. If it's still not working for you, there's probably something messed up about the account you're trying to import or its comments, and it'll need individual troubleshooting -- the problem is, there are very few people who can do that troubleshooting (it's pretty much just Mark, and he is extremely, extremely overbooked). If you try to import comments again and it still doesn't work, open a support request, but please do understand that it will likely take quite some time for troubleshooting: there's a limit to the amount of our time we can spend in trying to diagnose and work around problems on the LJ end.

[cellio]

Thanks for the good news!

Has there been any progress on the failed import of comments from a week or so back? My profile claims 20k+ comments, but they didn't make it. I know you had a bug report about that (somebody, perhaps you, told me so) and that devs were working on a fix. Is the duplicate-import problem part of that, or are these unrelated?

I'll delete imported content and try again once I hear that comments should work, but I won't bang on your servers before then. (I've been on LJ for 11+ years. I've...got stuff.)

[denise]

The underlying cause of that should have been fixed last week, so try again. If it still doesn't work, you can open a new support request, but we really are getting to the point where I can't guarantee how long we can continue to spend this much of our time working around errors on LJ's end, so it may take some time to get back to you.

[astro_noms]

Yay! I've recently noticed a *ton of doubled entries, so this is very good news.

Is there a notification sent to us after the imported stuff's been removed? I realize there's probably a mile-long queue for it to be done, so I'm not expecting it right away, but I don't want to run the importer again and have the two processes end up conflicting.

[denise]

No, there isn't -- this is just a quick fix to let people clean up, not something that has a ton of bells and whistles and features. Just start it, wait a day or so, check your journal to confirm that imported content is deleted, and then import again.

[ozqueen]

I don't need this tool, but I'm stopping in to give you love, Dreamwidth ♥

[zarhooie]

Dreamwidth <3's you too!

[sholio]

Oh YAYYYYY! Thank you! (And I had assumed that it was just me, because it seemed to be associated with renaming my account. In a weird way, it's a relief to know that my journal wasn't the only one it had happened to.)

[denise]

Actually, renames-on-the-LJ-side may still be A Bit Funky. Unless we fixed that bug and I just don't remember us doing so! Which might also be the case. (Hey, I only run the place, I have no actual idea what the hell goes on around here half the time.)

[heresluck]

I love you guys.

[denise]

It is really awesome that you do, because really, we're ridiculous enough that you have to either love us or just roll your eyes a lot. :)

[elfy]

Question:

If I once imported my LJ and all went fine, but went on posting there (LJ) without crossposting here ... what happens if I import my LJ again? Will I have everything twice except the most recent entries? Or will it sync just the new stuff?

[denise]

Theoretically and in an ideal world, it will just bring over things that are in your LJ that are not in your DW account. (If you'd used the crossposter to crosspost, it will take the comments from the LJ version of the entry and put them on the DW version of the entry. If there are entries in your LJ that aren't in your DW, it will bring the whole entry with its comments.)

In practice, LJ has been acting up lately and giving us funky data, which is why some people have ended up with duplicates, which is why we made the "remove all imported content and try again" override described in this post. :)

[fliegerhorst]

Well, that was fast. I just tried this, and it worked without any problems! Thank you guys so much for this tool!

[nanbungaku]

I don't really need this tool, just passing by to say how much I appreciate all the things you guys do for us!

[girlfmkitty]

Works like a dream! Thank you so much for this tool! <3

[nicefinalbeam]

This will solve the problem I had with duplicates. I'm always so impressed with how you respond to users on your site. Thank you! ♥

[triadruid]

I'm pretty sure I know the answer to this, but... a posted entry that has DW-comments on it will get killed along with those comments, right?

[zarhooie]

Yes. If the entry is imported from LiveJournal, then is commented on once it hits Dreamwidth, the comments will go away once the ICoRT is run. That happens because there's no such thing as a comment without an entry.

Double Trouble Again

[izmeina]

Was delighted to find this post with the cure for those pesky duplicates. But the map turned out to be not quite the territory.

Did the LJ import delete thing in order to get rid of duplicates from May 2009 after the last import which was nearly a year ago (45 weeks according to the import page)
This time the new import did bring all the comments, entries and user pics over and very quickly too but once again all the entries since mid May 2009 are duplicates

Will I need to delete the LJ imports again or is there some other work around?

Re: Double Trouble Again

[denise]

Okay, I looked at this and your situation is a little different: the duplicates aren't happening because of the data coming over weird from LJ. It looks like you've been manually crossposting to LJ, instead of using the Dreamwidth-native crossposting system. Unfortunately, if you manually crosspost, there's no way for the importer to know that the entry on DW and the entry on LJ are the same thing, so it can't merge the comments for the two entries. I'm really sorry. There's no way to merge the existing entries now, but if you start using the Dreamwidth crossposting tool instead of crossposting manually, future imports will understand that the entries were crossposted.

[insomnikat]

Thank you :D

[got_swagger]

I've held off on importing (I like to import every week or so - even moreso now that I tag wrangled and have everything perfect!) since the original notice went up and I'm thrilled to be able to import again!

Thank you so much for the awesome support and transparency you give us.

ETA: I forgot to mention that my import last night went smooth as silk!

[mark]

Glad it went well, thanks for letting us know!

[bluebellanon]

Do you happen to know if anyone else has been having a problem where not all entries are imported over?

Because I just repeated the sherlock_bbc import from sherlockbbc to get the comments and it looks like an entry hasn't transferred. I don't know if there's more than that, but that's one I've noticed straight away.

The entry in question is this: http://sherlockbbc.livejournal.com/5120925.html if that helps at all!

ETA: I just noticed I also have a lot of duplicate entries. Did I need to run the removal tool first? I though DW took already existing entries into account when running new imports? Sorry to be such a pain! I'm new to all this. :)

[venatrix]

Any word on the CDN issue with icons not loading? I had a few that weren't showing up, but they came back and then my default stopped working. It's been a couple of weeks now. I'm using Firefox, and I have cleared cache/cookies, but that only lasts for an hour or so before it stops working again.

[tucuxi]

Thanks! This is awesome, and I appreciate you guys looking into it and fixing it so quickly!

[marilla_pm67]

Great !!!

Question, maybe I understand it wonky, but if someone use the deleting tool, and then want to reimport the content, it's Impossible?? or did I understand this sentence: "they can't be re-imported into your account on Dreamwidth after you remove all imported content." falsely?

BTW, I used the cross posting tool from DW, and wanted to only import the comments, but it seems it doesn't work :( - try again later...

Sam

[denise]

No, if you remove all imported content and then re-import, it will bring over everything that's currently in your LJ account. That sentence in the original post means that if you had (for instance) an entry that you'd posted on LJ, imported into DW, and then deleted on LJ, and you run the tool to delete imported content, the entry will be deleted on DW and then that entry can't be reimported to DW, since it's been deleted on LJ and we use the current contents of your LJ to import.

If your comments won't import, check your inbox for errors, and if there are no errors, open a support request.