Denise (
denise) wrote in
dw_maintenance2017-04-10 05:53 am
![[staff profile]](https://www.dreamwidth.org/img/silk/identity/user_staff.png)
![[site community profile]](https://www.dreamwidth.org/img/comm_staff.png)
The importer is still running, we swear! It's just very busy.
People who are worried because the import of their LiveJournal account has been running for a long time with no real signs of progress: please don't be concerned! The import queue is currently a little long. (In the same way that Mt Everest is a little tall and the Mariana Trench is a little deep.) We're limited in how many import jobs we can run simultaneously and how quickly we can start the next import after one finishes: LiveJournal, like all sites, has restrictions on how frequently we can programmatically request data from their site, so the import queue can get very backed up at times like this when more imports are being started than are finishing. If you look at the import queue and the numbers don't seem to be changing much, or are only going up, it doesn't mean that no imports are finishing: it means a lot of additional people have scheduled an import since the last time you reloaded.
As long as you haven't gotten a failure message in your on-site inbox, your import is still running. (Even if you have gotten a failure message, your import may still be running: if the site thinks that the failure is something that might correct itself, like being unable to connect to the remote site, it will retry for a few times before giving up.) If you have gotten a failure message, the error message in your inbox should tell you what went wrong.
The three common problems right now: 1) you mistyped your username and/or password; 2) you need to agree to LiveJournal's new ToS before they'll permit you to access the data in your account; 3) an entry or entries in your LiveJournal account have a text encoding mismatch and you need to follow the link in the error message to fix it on LJ.
If you haven't gotten a failure message, your import is still waiting in the queue, and will run when it makes its way up to the top of the queue.
People keep asking us how long the queue is (by which they mean, how much time will it take for a job just started to successfully finish: length of time, not number of jobs waiting). I would love to be able to give you a definite answer! It's really, really hard for us to predict how long it will take for a job to get up to the top of the queue, though: how long an import takes to complete depends on a lot of things, including how many posts/comments are in the journal. To give you a ballpark figure that might be off by up to 100% on either side: If I personally started a brand new import right now (in my timezone, the early morning of Monday 10 Apr), I would be pleasantly surprised if it finished before Tuesday morning (24 hours or so), would expect it to finish sometime on Tuesday night or maybe even stretch all the way to Wednesday night (36-60 hours), and wouldn't start to wonder if I should poke
mark or
alierak to doublecheck that something hadn't gotten stuck in such a way that our monitoring didn't alert us about it until Thursday afternoon or evening (100+ hours).
All of those time estimates, by the way, assume a relatively uncomplicated job that succeeds on the first try. When the site tries again after a failure, it includes a delay that increases after each failure in case the failure was due to transient network issues. So, I know some of you started an import at the end of last week and it's still running: some of you are trying to import very large journals, and some of you ran into errors along the way and are in a retry wait loop. Again: if you haven't gotten the final error message in your inbox (and it will tell you it's the final error), it's still chugging along.
You do not have to leave the importer page open or stay logged into Dreamwidth until your import finishes. (You do have to avoid changing your LJ password until the job is done, or it will fail.) You can close the window/tab and go off and explore Dreamwidth; the movers will be along in a little while with your stuff.
The tl;dr version of my usual longwinded babble: IMPORTER VERY BUSY. MANY PEOPLE MOVING IN. LIKE ON DORM OR APARTMENT MOVE-IN DAY, FREIGHT ELEVATORS VERY SLOW. BUILDING OWNERS RUNNING FREIGHT ELEVATORS AS FAST AS POSSIBLE AND APOLOGIZE FOR THE WAIT.
A housewarming glass of champagne/sparkling cider/fancy handmade soda for all! Welcome to the neighborhood.
EDIT, 10 Apr 2017 7:15PM EDT: the importer is not the only thing that is very busy today! I'm trying to get to all the comments here, but keep getting dragged off to handle other stuff (and will be knocking off for the night soon). If you have a technical support problem, it will probably be faster to open a support request, where there's less likelihood that it will get overlooked in the sea of comments.
As long as you haven't gotten a failure message in your on-site inbox, your import is still running. (Even if you have gotten a failure message, your import may still be running: if the site thinks that the failure is something that might correct itself, like being unable to connect to the remote site, it will retry for a few times before giving up.) If you have gotten a failure message, the error message in your inbox should tell you what went wrong.
The three common problems right now: 1) you mistyped your username and/or password; 2) you need to agree to LiveJournal's new ToS before they'll permit you to access the data in your account; 3) an entry or entries in your LiveJournal account have a text encoding mismatch and you need to follow the link in the error message to fix it on LJ.
If you haven't gotten a failure message, your import is still waiting in the queue, and will run when it makes its way up to the top of the queue.
People keep asking us how long the queue is (by which they mean, how much time will it take for a job just started to successfully finish: length of time, not number of jobs waiting). I would love to be able to give you a definite answer! It's really, really hard for us to predict how long it will take for a job to get up to the top of the queue, though: how long an import takes to complete depends on a lot of things, including how many posts/comments are in the journal. To give you a ballpark figure that might be off by up to 100% on either side: If I personally started a brand new import right now (in my timezone, the early morning of Monday 10 Apr), I would be pleasantly surprised if it finished before Tuesday morning (24 hours or so), would expect it to finish sometime on Tuesday night or maybe even stretch all the way to Wednesday night (36-60 hours), and wouldn't start to wonder if I should poke
![[staff profile]](https://www.dreamwidth.org/img/silk/identity/user_staff.png)
![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
All of those time estimates, by the way, assume a relatively uncomplicated job that succeeds on the first try. When the site tries again after a failure, it includes a delay that increases after each failure in case the failure was due to transient network issues. So, I know some of you started an import at the end of last week and it's still running: some of you are trying to import very large journals, and some of you ran into errors along the way and are in a retry wait loop. Again: if you haven't gotten the final error message in your inbox (and it will tell you it's the final error), it's still chugging along.
You do not have to leave the importer page open or stay logged into Dreamwidth until your import finishes. (You do have to avoid changing your LJ password until the job is done, or it will fail.) You can close the window/tab and go off and explore Dreamwidth; the movers will be along in a little while with your stuff.
The tl;dr version of my usual longwinded babble: IMPORTER VERY BUSY. MANY PEOPLE MOVING IN. LIKE ON DORM OR APARTMENT MOVE-IN DAY, FREIGHT ELEVATORS VERY SLOW. BUILDING OWNERS RUNNING FREIGHT ELEVATORS AS FAST AS POSSIBLE AND APOLOGIZE FOR THE WAIT.
A housewarming glass of champagne/sparkling cider/fancy handmade soda for all! Welcome to the neighborhood.
EDIT, 10 Apr 2017 7:15PM EDT: the importer is not the only thing that is very busy today! I'm trying to get to all the comments here, but keep getting dragged off to handle other stuff (and will be knocking off for the night soon). If you have a technical support problem, it will probably be faster to open a support request, where there's less likelihood that it will get overlooked in the sea of comments.
no subject
no subject
no subject
no subject
Thanks for the information!
no subject
You can! You do have to import your entries along with the comments, though, or else there's nothing to stick them on. :)
Imported comments are displayed as coming from "so-and-so.livejournal.com" and, thanks to the magic of OpenID, as long as you still have access to the LJ account you were logged into when you originally made the comment on LJ, you can log into DW using that OpenID account and still have the same level of control (edit, delete, etc) as you had for the LJ version.
Or, you can claim your OpenID with your DW account, and then all the comments you've left in other people's journals with that LJ account will update to display as your DW account instead. For instance, look at the comments to this post from 2008, which was originally made on LJ, and was the first time that we talked about DW in public. I imported the entry, and all its comments, from LJ; the first comment thread, now appearing to be from me and
zorkian (which is
mark's personal account like
synecdochic is my personal account), originally displayed like how many of the others still do. (As xb95.livejournal.com and synecdochic.livejournal.com, respectively.) Then I claimed my synecdochic.livejournal.com OpenID URL with
synecdochic, and he claimed his xb95.livejournal.com OpenID URL with
zorkian, and now anywhere on the site that somebody has imported a comment that I made while logged into LJ as
synecdochic, it shows as
synecdochic instead!
(It's even smart enough that if you have the same icon keywords on DW as you did on LJ, after you claim your OpenID, all the comments will even display the proper icon.)
no subject
If I imported those comments, they'd come with the posts to which they're attached, which would double my posts, wouldn't it?
I'm not going to do anything with this right now, as the import system is understandably overwhelmed, but it's something to think about. It would be nice to keep those comments, if at all possible.
no subject
no subject
I'll do some testing, when things are less chaotic.
Thank you!
no subject
If you were crossposting using Semagic, instead of using the Dreamwidth-native crosspost utility, unfortunately there's no way to import your LJ without duplicates resulting: the only way for the importer to know that a particular entry on both sites is the same actual entry is if the entry was generated through the crossposter. (We don't do things like comparing the actual entry content to see if two entries are the same, because text parsing on that level is annoyingly difficult and slow, and the smallest of differences between the two versions could result in entries not being merged when they should be -- or, worse, being merged when they shouldn't be.) The same thing applies if you were manually copying and pasting your entries into both sites' update pages.
My best suggestion is to create a separate account (something like stasia_from_lj or something) and import the current state of your LJ into it -- it might be annoying to have to look in two places for something you remember, but I'd personally find it less annoying than having two copies of all the entries from 2009-2017.
no subject
Anyway, yeah, I figured that was the issue. Having a second journal just to save those comments might be the best option. I'll wait, though, until the rush dies down.
no subject
no subject
omfg thank you so much for the rec! I force myself to viciously prune them at the end of the day, because omg, but it would be nice to not to have to...
no subject
no subject
Sad high-five of fellow virtual packratness!
no subject