Imports from LJ running again

[denise]

After conversation with LJ's Ops team, and after they made a bunch of changes to their data export function to make sure that it is rate-limited to hell and gone in the event of further DDoS congestion and network traffic, we've been able to re-enable imports from LJ again.

If you previously started an import from LiveJournal, your import will run when it reaches the top of the queue. The queue is still very very long from last week's backlog, and will probably take quite some time to clear out -- my wild-ass-guesstimate right now is about a week to clear out the number of imports we have in queue. (And of course, more imports are being added as the others clear out.) Just be patient -- the imports will be completed in turn!

Right now, it looks like about 75% of imports are completing successfully, while about 25% are failing due to being unable to reach the site at the time the import runs. You will receive a message in your DW inbox when your import succeeds or fails. If it fails, you can put it into the queue again to try again at a later date.

Crossposts to LJ are still hit-or-miss; if your crosspost fails, wait a bit and then edit the entry on the DW side. You don't need to change anything, just save it again; that will force it to try to crosspost again.

Thank you guys so much for being so understanding about this whole mess! Situations like this prove yet again that we have the awesomest userbase in the history of ever.

Comments

[gillo]

You guys and the LJ team are the awesome ones. We're just the whiny, entitled userbase. What really matters is that no spammers are allowed to destroy a blogging platform because they disapprove of the content some of the users post.

[denise]

There has been a significant dearth of whining at us! (I mean, not to say that complaining about things being unavailable is (always) whining. But you know what I mean.) In general, people have been incredibly understanding and supportive, and that makes me so happy to see.

[catsittingstill]

This DDoS attack on LJ is certainly not DW's fault!

Thank you for your efforts to get things sorted out.

[dhamphir]

YAY! Thank for keeping us up to date with what's been going on. :D

[anne_d]

You're awesome, we're just your loyal worshippers!

[sharpiefan]

I second that!

[lexin]

Thanks for keeping us up to date. I'll stop worrying about my comments import, and try again in a couple of weeks.

[denise]

If you put the import into the queue, just forget about it! It will run sooner or later and right now there's about a 75% chance it will succeed.

(If you already got the failure notice, then yeah, give it a bit before you try again. *G*)

[rocquellan]

thanks for the hard work guys, all y'all are awesome-sauce <3

[tanhorn]

Thank You very much
I will notice my friend about this
But I ll also recommend them not to run horses

[vlion]

Well, I haven't imported anything in like a year+ since I originally imported from LJ. So I don't 'care', as it were.

I do care that you are being transparent and are working with the LJ side of things. Thank you.

Thank you a lot.

[jlh]

Seriously, you guys are so fantastic at letting people know what's going on (though I love that everyone's response to "too much traffic in the import queue" is to ask if they can take out their request). And I don't just mean better than LJ--who really has taken a lot of steps to improve in the past year--but in particular over tumblr and a lot of other services who have been around longer than DW.

[owaranai_natsu]

Thank you so much for everything you guys are doing, as well as the LJ team. Thank you, also, for letting us know about the situation.
Much ♥

[oksana107]

как можно импортировать посты из сообщества в LJ в такое же сообщество на dreamwidth.org ?

[denise]

Not yet. We will add this in the future, though.

(I'm sorry I don't speak Russian!)

[illariy]

I love how you're working together with LJ for this. And thank you for keeping us updated on the status of things. :D

[florentinescot]

You guys rock!

And if you really want somebody to whine ..... :-D How about that Journal Backup Thing? :-D

[vlion]

Er, uh, about a year ago I started working on putting that together, but I found myself with grad school, and then with a full-time job.

I got to the point of having a spec done and starting to digest the codebase and bug people when grad school spun up and I had to lay it down.

I am very sorry.

[matopia]

You guys rock! (:

[fightingforward]

Thanks for posting this. I've been trying to get my stuff over here from LJ for 4 days now and I was really wondering what the F was going on.

[denise]

:) It will complete sooner or later! just be patient :) We're able to finish about 200-300 LJ imports per day at this point, so the queue will take a while to drain. But if your job's in the queue, you can forget about it for now; it will run when it gets to the top. (Mind you, it might fail, if LJ is too overwhelmed when it hits the top of queue. But still.)

Thanks for your patience :)

[stephenie_n_lamaina]

Thanks for keeping us in the loop.

[msmaria]

Thank you very much.

[geminianeyes]

Thank you!

Posts like these are one of the reasons why I love DW. :d

[gemfyre]

Well that explains things. Nice to know it's 'not just me'. :)

Death to DDOSers!

[guenievre]

Mine completed MUCH sooner than I expected - thank you!

[denise]

Yeah, the queue is booking this morning! I'm hoping this means that LJ is recovering from the DDoS. (Or that the DDoSers have given up. Fingers crossed.)

[cpolk]

I am confused by the import thing. I've tried a few times and just tried again now, but less than 30 seconds after I put it in the queue, the import aborts? I think i might be doing something wrong, but I can't figure out what. is there a place in the FAQ I should be looking?

[denise]

Check in your Dreamwidth inbox for an error message -- it will tell you what's wrong. Usually, when an import aborts immediately like that, it means that you have your password wrong -- make sure you're putting in your username and password for the other site, not your Dreamwidth username and password!

If the error message in your inbox doesn't help you figure out what the problem is, open a support request with the text of the error message, and somebody will help you out.

[underice]

First off, thank you for being so transparent about the steps you've taken during this situation. I really appreciate that, and it makes me happy to use Dreamwidth.

I have a question, though. Will non-LJ imports still be jumping the queue as they were before, or are they back to normal? My partner has been trying to import from Insanejournal for a couple of days now and when those imports were working, but LJ imports were not, it kept aborting the import and so nothing went through.

That's why I wondered if non-LJ imports were still processing normally, or if they were getting caught up in the backlog as well. Either way, I appreciate your answer; I simply wanted to know.

Thank you for your time.

[denise]

No, we're back to "first in, first out" processing. If your partner's import from InsaneJournal was failing when there weren't any LJ imports in the queue, though, it's likely there's something else wrong! Have hir check hir inbox for the failure message and see if it sheds some light.

The queue is still going, so any retry should hit the top of the queue in a few days or so.

[stellar_raven]

I'm also having an issue like the ones described here by others:

I enter my import into the queue, upon refreshing, I find the the import for my LJ journal entries has been aborted. Everything else imports just fine, just not the entries.

I get no error message in my inbox telling me why it was aborted. It's been doing this for a couple of days now.

I thought I'd mention it here first before opening a support ticket since others seemed to be having the same issues.

[denise]

Please do open a support request! It can be looked into further that way.

A couple of questions

[dimrub]

Hi Denise!

Thanks for the update. A couple of questions if I may (feel free to send me RTFMing if I'm missing something obvious):

1. Where do I report bugs? The one I want to report is that URLs of the form
http://dw-biz.dreamwidth.org/2010/09/ send me to a more or less random month (instead of the one I was asking for).
2. I was curious to see how the number of new comers to DW changes over time, so I took the liberty of building a graph. It looks as if there were 3 peaks since the site got started. The first one (May 2009) probably corresponds to the initial announcement, the last one (that started about a week ago) corresponds to the DDoS attacks on LJ. Do you have an explanation for the second one, starting on September 1st, 2010?

Re: A couple of questions

[denise]

1. You can report them to Support. (In the case of that URL, though, there were no posts made in dw_biz that month, so it will show you the list from beginning in the drop-down -- that could be what's causing your problem.)

2. I will hazard a guess it was when LJ announced Facebook integration, but I'm not sure.

[pne]

I think it's awesome how well DW+LJ can work together!

That DW doesn't thumb its nose and say "We don't need you any more, we're all grown up" or LJ doesn't go "You code-forkers who keep hammering our servers with imports and cross-posts", but that instead, the two ops teams have a friendly relationship and talk to each other regularly.

[primitivepeople]

Indeed, it's pretty impressive.

I'm very loyal to LJ and I hope they get all their DDoS woes resolved - I'm a permanent account holder - but in the short time I've been here, I've felt it's a nice place to be. I hope it stays that way.