The importer is still running, we swear! It's just very busy.

[denise]

People who are worried because the import of their LiveJournal account has been running for a long time with no real signs of progress: please don't be concerned! The import queue is currently a little long. (In the same way that Mt Everest is a little tall and the Mariana Trench is a little deep.) We're limited in how many import jobs we can run simultaneously and how quickly we can start the next import after one finishes: LiveJournal, like all sites, has restrictions on how frequently we can programmatically request data from their site, so the import queue can get very backed up at times like this when more imports are being started than are finishing. If you look at the import queue and the numbers don't seem to be changing much, or are only going up, it doesn't mean that no imports are finishing: it means a lot of additional people have scheduled an import since the last time you reloaded.

As long as you haven't gotten a failure message in your on-site inbox, your import is still running. (Even if you have gotten a failure message, your import may still be running: if the site thinks that the failure is something that might correct itself, like being unable to connect to the remote site, it will retry for a few times before giving up.) If you have gotten a failure message, the error message in your inbox should tell you what went wrong.

The three common problems right now: 1) you mistyped your username and/or password; 2) you need to agree to LiveJournal's new ToS before they'll permit you to access the data in your account; 3) an entry or entries in your LiveJournal account have a text encoding mismatch and you need to follow the link in the error message to fix it on LJ.

If you haven't gotten a failure message, your import is still waiting in the queue, and will run when it makes its way up to the top of the queue.

People keep asking us how long the queue is (by which they mean, how much time will it take for a job just started to successfully finish: length of time, not number of jobs waiting). I would love to be able to give you a definite answer! It's really, really hard for us to predict how long it will take for a job to get up to the top of the queue, though: how long an import takes to complete depends on a lot of things, including how many posts/comments are in the journal. To give you a ballpark figure that might be off by up to 100% on either side: If I personally started a brand new import right now (in my timezone, the early morning of Monday 10 Apr), I would be pleasantly surprised if it finished before Tuesday morning (24 hours or so), would expect it to finish sometime on Tuesday night or maybe even stretch all the way to Wednesday night (36-60 hours), and wouldn't start to wonder if I should poke mark or alierak to doublecheck that something hadn't gotten stuck in such a way that our monitoring didn't alert us about it until Thursday afternoon or evening (100+ hours).

All of those time estimates, by the way, assume a relatively uncomplicated job that succeeds on the first try. When the site tries again after a failure, it includes a delay that increases after each failure in case the failure was due to transient network issues. So, I know some of you started an import at the end of last week and it's still running: some of you are trying to import very large journals, and some of you ran into errors along the way and are in a retry wait loop. Again: if you haven't gotten the final error message in your inbox (and it will tell you it's the final error), it's still chugging along.

You do not have to leave the importer page open or stay logged into Dreamwidth until your import finishes. (You do have to avoid changing your LJ password until the job is done, or it will fail.) You can close the window/tab and go off and explore Dreamwidth; the movers will be along in a little while with your stuff.

The tl;dr version of my usual longwinded babble: IMPORTER VERY BUSY. MANY PEOPLE MOVING IN. LIKE ON DORM OR APARTMENT MOVE-IN DAY, FREIGHT ELEVATORS VERY SLOW. BUILDING OWNERS RUNNING FREIGHT ELEVATORS AS FAST AS POSSIBLE AND APOLOGIZE FOR THE WAIT.

A housewarming glass of champagne/sparkling cider/fancy handmade soda for all! Welcome to the neighborhood.

EDIT, 10 Apr 2017 7:15PM EDT: the importer is not the only thing that is very busy today! I'm trying to get to all the comments here, but keep getting dragged off to handle other stuff (and will be knocking off for the night soon). If you have a technical support problem, it will probably be faster to open a support request, where there's less likelihood that it will get overlooked in the sea of comments.

Comments

Thank you!

[technomom]

As one of those who imported successfully this week, I just wanted to say thank you very much! (No worries about replying - this doesn't need one!)

[drcuriosity]

I am very impressed that the whole thing hasn't caught fire by now. Well done!

I'm also over here now. Hi :-)

[denise]

Oh, hey you! I subscribed to you with my personal account. :)

[malurette]

You know, I never remember whether or not I already told you how great it is that the importer exists at all? Thank you again DW and mods for being so awesome!

[piratice]

I'm not one of the new users that are importing but just wanted to say love the communication from DW staff. Well done!

[denise]

I always love to hear that people like my babble. :)

[sineala]

I have an importer question that doesn't really merit a support request, but--

I imported way back when, when I first joined, and now I'd like to reimport to grab comments and so on, but... uh... you remember that bug the importer used to have where if you had crossposting turned on when you imported it would crosspost every imported entry back to LJ? I was the person who accidentally discovered that and managed to doublepost several thousand entries. So I'm worried that my import was too long ago and somehow (due to the manual cleanup) the entries won't be marked as correctly imported and somehow it will try to post everything twice to DW or something. Is that a thing that could happen?

[denise]

It, uh, is definitely a thing that could happen, yep. My usual advice here would be to run the tool that would clean out all your imported entries and then reimport, but I honestly don't remember what we did to clean up your account and that may result in all of your crossposted entries being imported as duplicates so I don't want to suggest it!

My suggestion for your case:

1) Right now, create another account (sineala-from-lj or sineala-archive or whatever) and import your LJ into it, so you at least have the archive in case of disaster and don't have to worry if step 2 takes a long time.

2) Open a support request in the Importer category. Start your subject line with [mark], or I'll change it for you when I notice that you opened the request, so our usual support volunteers know that this is something that's going to take direct database access to look into. If you remember the number of the support request where we handled the problem last time, stick that in the body of the request somewhere; otherwise, we can dig it up. Summarize the problem (a link to this comment thread is fine) and hang tight.

(There are a few other requests that will need Mark to dig into what's going on, and his time is pretty limited these days and a lot of it this week is going to keeping the site zippy for everybody, so I can't give you any kind of time frame for when we'll be able to get to your request. Hence the suggestion to import into a backup account Just In Case!)

[nerwende]

Hello everyone!

I've started my gradual move from livejournal to here and I quite like what I've seen so far. :)

For anyone wondering about the queue, I just got my entries through after about 48 hours. Comments still haven't been processed though.

[denise]

Yup, entries come first (the comments need a place to go!) and comments will come along in a bit. (We used to not put entries in your journal until the comments had finished importing and were being processed too, but we changed that for this massive influx since that speeds things up just a tidge -- enough to shave down the time noticeably when taken over thousands of imports, at least.)

Welcome!

<3

[catastrophejones]

I just want to say thank you for doing this

feature request

[marmoe]

First off, I'd like to thank you and the rest of the Dreamwidth team for providing a safe haven and an importer tool. :) As the import queue is long, I know I have to be patient.

One (not very creative) idea occured to me, though: Would it be (easily) possible to not just display the length of the queue on the importer page, but the position of the current user in that queue, too? This would be more informative for the user. It's also likely not something you'd want to change in a running system under heavy load. ;-)

Re: feature request

[denise]

Sadly, not without a lot of changes to the system that tracks and schedules jobs, which runs a lot of stuff on the site and therefore one does not want to fuck with it when it's working its ass off, yeah. 99% of the time, it's not an issue, since 99% of the time, the importer is just sitting around bored and waiting for something to do! The other 1% of the time everything's on fire...

(That cartoon cracked me up, heh.)

[jloremaster]

Just the fact that you have established such an easy import process (or any import at all!) makes me giddily happy. Thank you!

[patentdragon]

Thought a chunk of my online life might be gone for good behind a new Iron Curtain, but now it isn't. Thanks for the import, however long it took!

[stardoll]

denise I'm sorry, I feel like I should apologize to Dreamwidth directly since like...I'm importing a 2300+ post blog that spans over 10 years.

But I'll buy paid time to make up for it!

I'M SORRY EVERYONE hi.

[denise]

Aw, don't worry, 2300 posts is nothing! We've done MUCH BIGGER. :D

(I think the record was something like 200,000 posts and over two million comments...)

[autumnpalette]

You are all so amazing for making this whole process much easier. :) I was wondering, though: is it possible to import an external community's content and comments to a personal blog here? The original community has no members and no posts other than the mod's/owner's. If not, is there a way to transfer content and comments from community to personal blog within dreamwidth?

[denise]

You can't import a community to a personal journal, but you can import a community to a community. Just make a DW community for it.

[denise]

Alas, can't really be done without completely rewriting how jobs are scheduled on the backend and how they "report back" to the thing that asked them to start, and that's a high-risk change (could break everything rather messily!) and 99.9% of the time isn't really necessary -- under normal circumstances an import will finish in minutes if not seconds. These are just not normal circumstances!

(The thing that does the actual import is an asynchronous worker, and the code's not really set up for the worker-manager to report back anything but success or failure. The actual "okay let me line up all these jobs and chug through them one by one" is handled in a separate place. It's like if you ask a coworker to do a thing for you and they go off into the other room, do it, and then come back and say "done!" -- if you go into the other room when they're partly finished to interrupt them and ask them how it's going, they'll probably be a bit irked when you break their concentration.)

[codekitten]

I'm importing my LJ to Dreamwidth now. It appears that photos (embedded in my posts) are not coming in. They were photos I had uploaded into my LJ gallery and then linked to.

Thanks!!!

[denise]

By any chance, a) are you using the SSL Everywhere beta or have the SSL Everywhere extension enabled; and b) are the images on the LJ end locked/friends-only?

[rhrsoulmates]

Hi! I successfully imported an LJ comm that I had without any issues - yay! It took such a short amount of time as well! My question is this. In 2009 I imported over what was my then LJ BUT of course I've made entries since that date. If I hit "import" again, will all the entries from 2009 and earlier be duplicated? If so, what do I do about that, aside from having to manually delete each duplicate?

Thanks, in advance!

[denise]

The importer should Just Work to bring over the entries without any duplicates, unless:

* you've been manually crossposting to both sites (via cut and paste or via Semagic/another client) instead of using the on-site crossposter;
* you've been using the on-site crossposter and have ever deleted & re-created your crossposting settings

In those cases, the entries will be duplicated and there's not really much we can do to merge them, and it will be better to create a separate account and import into that as a backup.

In some cases, mostly involving cases where you renamed your LJ account after your last import but sometimes just because the moon is in the seventh house and the importer felt like it, people who first imported their entries when the site first opened will get duplicates when they re-import now. If that happens, you can delete all your imported entries and start a fresh import, or, if you know for sure that there's content that no longer exists on LJ but was imported into DW and you don't want to lose it, open a support request in the Importer category with examples of a few duplicated entries (with the links to the entries both on LJ and DW) and someone will get back to you when they get a chance. It may take a little while, though; these past two weeks have been very hectic!