denise: Image: Me, facing away from camera, on top of the Castel Sant'Angelo in Rome (Default)
Denise ([staff profile] denise) wrote in [site community profile] dw_maintenance2011-04-10 10:54 am

New imports temporarily stopped; import queue paused

The situation with LJ's DDoS affecting our import queue has only gotten worse since my last post, and delays/slowdowns from LJ means that the queue will not clear anytime soon (due to the length of time each job takes to complete or fail with the extremely slow connections we are experiencing). At this point, it's also possible that the traffic from DW's importer is making LJ's situation worse, which we absolutely do not want to do!

We are therefore temporarily pausing the existing import jobs until the traffic clears up some and we are able to further modify the importer so that it is even less of a load on LJ. We are also temporarily stopping the ability to queue new imports, so that once we bring the import queue back up, it will give the queue the chance to clear a bit. (Also, we'd like to minimize the impact on LJ.)

We will re-enable the existing import jobs in the queue once we are able to make some changes to the importer in order to lower the risk that we're making things worse for LJ even more (reducing the number of times a job will try before it fails, further rate-limiting our queries). At that point, we will let the existing jobs in queue run for a while until we can clear out the queue a bit. Once the queue clears, we will re-enable the starting of new jobs.

It will be at least a few days until we are able to re-enable new imports.

If you already have an import job in the queue, you don't need to do anything. It will be automatically re-started when we bring importing back online. (It's not possible for you to cancel an import once it's started -- sorry!)

If you're importing your journal because you want a backup of your LiveJournal, you may want to try the LJ Archive software, which will give you a local copy of your journal. (You can't upload it into your DW account, but at least you will have the backup!)

If you're trying to import from a source other than LJ, I'm really sorry that the LJ DDoS slowdowns are affecting you! We're working right now on the ability to only pause imports from a single source, rather than all of them. Once we have that, we'll re-enable imports from sites other than LJ, and let the LJ imports sit in the queue until LJ is feeling better.

I'm really sorry about all the inconvenience!

EDIT, 2PM EDT 4/10/11: We have just pushed live a change that will allow us to re-enable the queuing of imports. Imports from sources other than LJ will now be processed as normal. Imports from LJ will be held in the queue and not processed until things calm down a bit on their end.

You can therefore submit an import request to import from LJ and forget about it. We will run the LJ imports once the DDoS and resulting issues clear up a bit on their end -- it could be a day or two, it could be next week; we just don't know!
prozac_poetry: (Default)

[personal profile] prozac_poetry 2011-04-10 03:13 pm (UTC)(link)
I'm really impressed in how you are supporting lj in all of this

(no subject)

[personal profile] yeuxdebleu - 2011-04-11 05:47 (UTC) - Expand
stormy: βͺ ππŽπ“πˆπ‚π„ ❫ 𝑫𝑢 𝑡𝑢𝑻 𝑻𝑨𝑲𝑬 𝑴𝒀 𝑰π‘ͺ𝑢𝑡𝑺 ⊘ (Default)

[personal profile] stormy 2011-04-10 03:14 pm (UTC)(link)
Thanks for the heads up. I'm really surprised [site community profile] dw_maintenance doesn't show up on like [site community profile] dw_news does. If people aren't watching this community or the twitter, they're not going to know.

Even an embedded live twitter feed on that page could be handy if that's accessible enough. I rarely use the page, but it is the first thing people see going to
Edited (more thoughts~) 2011-04-10 15:16 (UTC)

(no subject)

[personal profile] azurelunatic - 2011-04-10 17:11 (UTC) - Expand

(no subject)

[personal profile] brainwane - 2011-04-10 18:26 (UTC) - Expand

(no subject)

[personal profile] azurelunatic - 2011-04-10 18:29 (UTC) - Expand
monanotlisa: (dw yay!)

[personal profile] monanotlisa 2011-04-10 03:15 pm (UTC)(link)
No worries, it's not all your fault; in fact you're just being part of the solution as of now. ;) I'm hoping for the best, for both services.
discodiva76: (FIAD Hug)

[personal profile] discodiva76 2011-04-10 03:18 pm (UTC)(link)
You're doing wonderfully.....I was very lucky to get mine done when I did....the relief is palpable I can tell you!...

Good luck with all the support and hard work you're doing...

Deeds xx
jae: (livejournalgecko)

[personal profile] jae 2011-04-10 03:19 pm (UTC)(link)
Thanks for keeping us up to date! It's really appreciated.

illariy: uhura smiles (uhura: smile)

[personal profile] illariy 2011-04-10 04:02 pm (UTC)(link)
Seconded. :D

(no subject)

[personal profile] shean - 2011-04-12 11:24 (UTC) - Expand
tanhorn: (Default)

[personal profile] tanhorn 2011-04-10 03:23 pm (UTC)(link)
Last night I saw one one or two errors 418 on LJ page ;-)
azurelunatic: "beautiful addiction", electron microscope photo of caffeine (caffeine)

[personal profile] azurelunatic 2011-04-10 05:13 pm (UTC)(link)
And no wonder: I'd imagine they're getting no sleep at all.

(no subject)

[personal profile] tanhorn - 2011-04-10 17:23 (UTC) - Expand

(no subject)

[personal profile] azurelunatic - 2011-04-10 17:28 (UTC) - Expand

(no subject)

[personal profile] tanhorn - 2011-04-10 17:35 (UTC) - Expand

(no subject)

[personal profile] tanhorn - 2011-04-10 17:38 (UTC) - Expand

(no subject)

[personal profile] tanhorn - 2011-04-10 17:36 (UTC) - Expand
sapphire_rose: (Default)

[personal profile] sapphire_rose 2011-04-10 03:23 pm (UTC)(link)
Yay! I was wishing that we could stop import jobs because of overloading LJ, but thank you for pausing the current imports. :-)
archangelbeth: An anthropomorphic feline face, with feathered wing ears, and glasses, in shades of gray. (Default)

[personal profile] archangelbeth 2011-04-11 10:20 am (UTC)(link)
I am hugely impressed that
1) DW paused, and /only/ paused, not just deleted, imports, and
2) fixed it so imports from other places weren't paused as well.

That is classy programming and thought!
holyschist: Image of a medieval crocodile from Herodotus, eating a person, with the caption "om nom nom" (Default)

[personal profile] holyschist 2011-04-10 03:25 pm (UTC)(link)
This sounds like a good plans!
nashipants: illustration by ??? ([seph] sup Cloud)

[personal profile] nashipants 2011-04-10 03:27 pm (UTC)(link)
I have to say how impressed I am by how much you guys are supporting LJ through this. It's worth a lot of respect to see how you've behaved during this technological blogosphere crisis, and it really just makes me love this place all the more.

I wish everyone could be as supportive and respectful about this as you've been.
dhamphir: (Default)

[personal profile] dhamphir 2011-04-10 03:31 pm (UTC)(link)
Thanks for keeping us up to date and for striving to be part of the solution. :D
sollers: me in morris kit (Default)

[personal profile] sollers 2011-04-11 09:43 am (UTC)(link)
Seconded on both counts.
wyldbutterflies: (Default)

[personal profile] wyldbutterflies 2011-04-10 03:39 pm (UTC)(link)
Is there a way that you can state this on the import page itself? I have a feeling that most individuals coming over or trying to import do not read this community and have no idea what is going on. At least they would know what the deal is instead of getting really frustrated at DW.
technoshaman: Londo Mollari having a Memorex Moment (squee)

[personal profile] technoshaman 2011-04-10 03:41 pm (UTC)(link)
Did I mention y'all are awesome? Just sayin'.
habemus: (Default)

[personal profile] habemus 2011-04-10 03:42 pm (UTC)(link)
I thought it would come to this...

I'm sorry we all panicked and started our import at a time when, obviously, LJ didn't need the additional load...

I feel a bit chagrined I had my part in this, I'll admit. I'm glad you took measures to enable LJ to breathe a bit.
goodbyemyfancy: (pic#799423)

[personal profile] goodbyemyfancy 2011-04-10 03:50 pm (UTC)(link)
Thanks for the update with this, and a very good call!!

BtW - LJ Archive used to be great for backing up our LJ journals. Unfortunately the software is out of date and no longer works as the author is no longer maintaining it. I used to use this all the time. I tried using this before setting up my import from LJ to here. Perhaps it works for others though.

(no subject)

[personal profile] shinsetsu - 2011-04-10 16:20 (UTC) - Expand

(no subject)

[personal profile] goodbyemyfancy - 2011-04-10 17:20 (UTC) - Expand

(no subject)

[personal profile] goodbyemyfancy - 2011-04-10 17:26 (UTC) - Expand

(no subject)

[personal profile] faithofone - 2011-04-10 18:24 (UTC) - Expand

(no subject)

[personal profile] ariandar - 2011-04-10 22:44 (UTC) - Expand

(no subject)

[personal profile] faithofone - 2011-04-11 02:31 (UTC) - Expand

(no subject)

[personal profile] sunrisetr - 2011-04-10 15:58 (UTC) - Expand

(no subject)

[personal profile] helens78 - 2011-04-10 16:38 (UTC) - Expand

(no subject)

[personal profile] goodbyemyfancy - 2011-04-10 17:14 (UTC) - Expand

(no subject)

[personal profile] sunrisetr - 2011-04-10 17:46 (UTC) - Expand

(no subject)

[personal profile] brownbetty - 2011-04-10 18:00 (UTC) - Expand

(no subject)

[personal profile] sunrisetr - 2011-04-10 18:14 (UTC) - Expand

(no subject)

[personal profile] yukinojou - 2011-04-11 13:33 (UTC) - Expand

(no subject)

[personal profile] pne - 2011-04-14 08:52 (UTC) - Expand
cathalin: (Default)

[personal profile] cathalin 2011-04-10 03:52 pm (UTC)(link)
Thanks for the head's up, and for working to help reduce the problem.
sunrisetr: (Default)

[personal profile] sunrisetr 2011-04-10 03:57 pm (UTC)(link)
Thank you for the heads up!

[personal profile] romanaorfred 2011-04-10 04:12 pm (UTC)(link)
Thank you for keeping us up to date with all of this, and like others have said, I'm really impressed at how you are helping LJ and how supportive you are. :)
ifaynsh: (Default)

[personal profile] ifaynsh 2011-04-10 04:20 pm (UTC)(link)
I believe that the most effective way to help LJ is to mirror here the pages that are under attack and to cause fire on yourself. This will reduce load on LJ.

(no subject)

[personal profile] ifaynsh - 2011-04-10 16:48 (UTC) - Expand

(no subject)

[personal profile] tatterpixie - 2011-04-10 18:42 (UTC) - Expand

(no subject)

[personal profile] sashura - 2011-04-11 16:29 (UTC) - Expand
havocthecat: the lady of shalott (Default)

[personal profile] havocthecat 2011-04-10 04:39 pm (UTC)(link)
It's fine, of course! I'm happy to wait to back up my LJ.
kiviraat: (Art: Army Doctor)

[personal profile] kiviraat 2011-04-10 04:49 pm (UTC)(link)
Your attitude towards everything that's happening over at LJ is commendable. I backed up my journal a few months ago in a matter of minutes, so it's showing how much strain you guys will be under if you're having to take these steps to make things easier for everyone overall.

Keep working hard guys! :)
phoenix: ink-and-watercolour drawing -- girl looking calmly over her shoulder (Default)

[personal profile] phoenix 2011-04-10 05:02 pm (UTC)(link)
Thanks for the update! I'm glad you're pausing imports while LJ recovers - like you say, it could be affecting the site and it needs no further hassle. And like many of the other commenters, I really appreciate that being supportive towards LJ. I wouldn't expect anything less from you, to be honest. Good people at DW :)
bellakara: (Default)

[personal profile] bellakara 2011-04-10 05:21 pm (UTC)(link)
Hope you don't mind but I've reposted this entire news bulletin on my LJ since quite a few people on my f-list have been importing or setting up new DW accounts and I'm not sure they know about this yet.
anne_d: (WWVD)

[personal profile] anne_d 2011-04-10 05:58 pm (UTC)(link)
Thanks for the heads-up!

It's taken me this long to remember to import my lj entries, so I'm obviously in no hurry. :)

[personal profile] jackiesjottings 2011-04-10 06:50 pm (UTC)(link)
Thank you for being so supportive with all of this. My LJ is in the queue to be backed up. I wish the hackers would go and play somewhere else where they can't do any harm! Like the outer reaches of the universe maybe :)

ps I love the bats mood theme here :)
keris: Keris with guitar (Default)

[personal profile] keris 2011-04-10 07:39 pm (UTC)(link)
Thanks for the updates! As it happens I did my LJ import in the lull after the first attack they had, and it went straight through (and I don't post enough to justify doing another one for a month or so).

I too am impressed by the support you're giving them. It really shows -- not just says -- that there is no animosity in the DW management towards LJ (unfortunately some users of both have been a little too evangelistic (to put it politely) and giving the impression of gloating, which you are not doing at all). As you say, if they decided to attack DW (or IJ etc.) you wouldn't have any better resources to withstand it.
sunnidae: (Default)

[personal profile] sunnidae 2011-04-11 02:39 pm (UTC)(link)
This comment makes me wonder: If someone imports their journal from another site, and then goes back and re-imports it later, is the importer smart enough to compare the two sites & only bring in new content? Or does it replace the entire original import?

Myself, having imported for backup, now I've got it set to crosspost and won't have to import again. I was just wondering.

And ... Thank You Dreamwidth Crew! :D

(no subject)

[personal profile] sunnidae - 2011-04-11 14:52 (UTC) - Expand

(no subject)

[personal profile] sofiaviolet - 2011-04-11 14:46 (UTC) - Expand

(no subject)

[personal profile] keris - 2011-04-11 14:55 (UTC) - Expand

(no subject)

[personal profile] sunnidae - 2011-04-11 14:59 (UTC) - Expand

(no subject)

[personal profile] keris - 2011-04-11 14:50 (UTC) - Expand

(no subject)

[personal profile] sunnidae - 2011-04-11 15:01 (UTC) - Expand

(no subject)

[personal profile] keris - 2011-04-11 15:04 (UTC) - Expand

(no subject)

[personal profile] brooksmoses - 2011-04-12 02:59 (UTC) - Expand

Page 1 of 2

<< [1] [2] >>