Denise (![[staff profile]](https://www.dreamwidth.org/img/silk/identity/user_staff.png)
denise) wrote in ![[site community profile]](https://www.dreamwidth.org/img/comm_staff.png)
dw_maintenance2017-04-10 05:53 am
denise) wrote in dw_maintenance2017-04-10 05:53 am
The importer is still running, we swear! It's just very busy.
People who are worried because the import of their LiveJournal account has been running for a long time with no real signs of progress: please don't be concerned! The import queue is currently a little long. (In the same way that Mt Everest is a little tall and the Mariana Trench is a little deep.) We're limited in how many import jobs we can run simultaneously and how quickly we can start the next import after one finishes: LiveJournal, like all sites, has restrictions on how frequently we can programmatically request data from their site, so the import queue can get very backed up at times like this when more imports are being started than are finishing. If you look at the import queue and the numbers don't seem to be changing much, or are only going up, it doesn't mean that no imports are finishing: it means a lot of additional people have scheduled an import since the last time you reloaded.
As long as you haven't gotten a failure message in your on-site inbox, your import is still running. (Even if you have gotten a failure message, your import may still be running: if the site thinks that the failure is something that might correct itself, like being unable to connect to the remote site, it will retry for a few times before giving up.) If you have gotten a failure message, the error message in your inbox should tell you what went wrong.
The three common problems right now: 1) you mistyped your username and/or password; 2) you need to agree to LiveJournal's new ToS before they'll permit you to access the data in your account; 3) an entry or entries in your LiveJournal account have a text encoding mismatch and you need to follow the link in the error message to fix it on LJ.
If you haven't gotten a failure message, your import is still waiting in the queue, and will run when it makes its way up to the top of the queue.
People keep asking us how long the queue is (by which they mean, how much time will it take for a job just started to successfully finish: length of time, not number of jobs waiting). I would love to be able to give you a definite answer! It's really, really hard for us to predict how long it will take for a job to get up to the top of the queue, though: how long an import takes to complete depends on a lot of things, including how many posts/comments are in the journal. To give you a ballpark figure that might be off by up to 100% on either side: If I personally started a brand new import right now (in my timezone, the early morning of Monday 10 Apr), I would be pleasantly surprised if it finished before Tuesday morning (24 hours or so), would expect it to finish sometime on Tuesday night or maybe even stretch all the way to Wednesday night (36-60 hours), and wouldn't start to wonder if I should pokemark or ![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
alierak to doublecheck that something hadn't gotten stuck in such a way that our monitoring didn't alert us about it until Thursday afternoon or evening (100+ hours).
All of those time estimates, by the way, assume a relatively uncomplicated job that succeeds on the first try. When the site tries again after a failure, it includes a delay that increases after each failure in case the failure was due to transient network issues. So, I know some of you started an import at the end of last week and it's still running: some of you are trying to import very large journals, and some of you ran into errors along the way and are in a retry wait loop. Again: if you haven't gotten the final error message in your inbox (and it will tell you it's the final error), it's still chugging along.
You do not have to leave the importer page open or stay logged into Dreamwidth until your import finishes. (You do have to avoid changing your LJ password until the job is done, or it will fail.) You can close the window/tab and go off and explore Dreamwidth; the movers will be along in a little while with your stuff.
The tl;dr version of my usual longwinded babble: IMPORTER VERY BUSY. MANY PEOPLE MOVING IN. LIKE ON DORM OR APARTMENT MOVE-IN DAY, FREIGHT ELEVATORS VERY SLOW. BUILDING OWNERS RUNNING FREIGHT ELEVATORS AS FAST AS POSSIBLE AND APOLOGIZE FOR THE WAIT.
A housewarming glass of champagne/sparkling cider/fancy handmade soda for all! Welcome to the neighborhood.
EDIT, 10 Apr 2017 7:15PM EDT: the importer is not the only thing that is very busy today! I'm trying to get to all the comments here, but keep getting dragged off to handle other stuff (and will be knocking off for the night soon). If you have a technical support problem, it will probably be faster to open a support request, where there's less likelihood that it will get overlooked in the sea of comments.
As long as you haven't gotten a failure message in your on-site inbox, your import is still running. (Even if you have gotten a failure message, your import may still be running: if the site thinks that the failure is something that might correct itself, like being unable to connect to the remote site, it will retry for a few times before giving up.) If you have gotten a failure message, the error message in your inbox should tell you what went wrong.
The three common problems right now: 1) you mistyped your username and/or password; 2) you need to agree to LiveJournal's new ToS before they'll permit you to access the data in your account; 3) an entry or entries in your LiveJournal account have a text encoding mismatch and you need to follow the link in the error message to fix it on LJ.
If you haven't gotten a failure message, your import is still waiting in the queue, and will run when it makes its way up to the top of the queue.
People keep asking us how long the queue is (by which they mean, how much time will it take for a job just started to successfully finish: length of time, not number of jobs waiting). I would love to be able to give you a definite answer! It's really, really hard for us to predict how long it will take for a job to get up to the top of the queue, though: how long an import takes to complete depends on a lot of things, including how many posts/comments are in the journal. To give you a ballpark figure that might be off by up to 100% on either side: If I personally started a brand new import right now (in my timezone, the early morning of Monday 10 Apr), I would be pleasantly surprised if it finished before Tuesday morning (24 hours or so), would expect it to finish sometime on Tuesday night or maybe even stretch all the way to Wednesday night (36-60 hours), and wouldn't start to wonder if I should poke
mark or alierak to doublecheck that something hadn't gotten stuck in such a way that our monitoring didn't alert us about it until Thursday afternoon or evening (100+ hours). All of those time estimates, by the way, assume a relatively uncomplicated job that succeeds on the first try. When the site tries again after a failure, it includes a delay that increases after each failure in case the failure was due to transient network issues. So, I know some of you started an import at the end of last week and it's still running: some of you are trying to import very large journals, and some of you ran into errors along the way and are in a retry wait loop. Again: if you haven't gotten the final error message in your inbox (and it will tell you it's the final error), it's still chugging along.
You do not have to leave the importer page open or stay logged into Dreamwidth until your import finishes. (You do have to avoid changing your LJ password until the job is done, or it will fail.) You can close the window/tab and go off and explore Dreamwidth; the movers will be along in a little while with your stuff.
The tl;dr version of my usual longwinded babble: IMPORTER VERY BUSY. MANY PEOPLE MOVING IN. LIKE ON DORM OR APARTMENT MOVE-IN DAY, FREIGHT ELEVATORS VERY SLOW. BUILDING OWNERS RUNNING FREIGHT ELEVATORS AS FAST AS POSSIBLE AND APOLOGIZE FOR THE WAIT.
A housewarming glass of champagne/sparkling cider/fancy handmade soda for all! Welcome to the neighborhood.
EDIT, 10 Apr 2017 7:15PM EDT: the importer is not the only thing that is very busy today! I'm trying to get to all the comments here, but keep getting dragged off to handle other stuff (and will be knocking off for the night soon). If you have a technical support problem, it will probably be faster to open a support request, where there's less likelihood that it will get overlooked in the sea of comments.
no subject
(Long ago, when I was still working for LJ, we used to have a house party/retreat once a year for one of the volunteer teams: theoretically it was for us all to get together in person and have a week of serious-business meetings about how things were going, since the volunteers lived everywhere across the world. In practice, it was often an excuse for us all to get together, drink heavily, and play Katamari Damacy in a crowd. One year we held it in South Carolina, which had (has?) a law that you can't buy more than a certain amount of liquor at a time without filling out a lot of paperwork, in an attempt to keep you from buying in resale amounts: we had to buy that permit to transport all the booze we were buying back to the rental house from the liquor store. Twice. My liver is very glad I'm not still in my 20s, is all I'm sayin'.)
Here's hoping your comments arrive soon!
no subject
Comments finished this morning - about 36 hours after dropping into the queue. Yay!
Now I'm importing a (very small) community and then I'll be able to delete LJ. Also yay.
Once this migration frenzy simmers down, I hope you'll update us on how many people moved in. The numbers must be impressive!
no subject
You can see them now! With fancy graphs, or the raw data -- you want "newbyday". (With the caveat that newbyday doesn't reflect how many people created accounts, just how many accounts were created; it counts the pseudo-accounts that are created as part of the import in that number, so one person creating an account and then importing their journal might be responsible for dozens or hundreds of "new accounts" if they had a lot of commenters who hadn't also commented in a journal someone had already imported. Like, if someone imports a journal I'd commented on with my LJ account, that wouldn't include an extra "account created", because the pseudo-account for my LJ account was created ages ago, but if someone imports a journal that has a comment by susiefromlj, and their journal is the only place susiefromlj ever commented, that would.)
IT IS A LITTLE BUSY, YES.
no subject
Data point: my tiny comm import is progressing. Entries took 46 hours in queue, comments now queued. Given the insane queue numbers that seems pretty good to me! Thanks to you guys for keeping everything running under this onslaught.
no subject
Ha, yeah. You can kind of map LJ's bad decisions by spikes in our account creation as people import their stuff... :)