Denise (![[staff profile]](https://www.dreamwidth.org/img/silk/identity/user_staff.png)
denise) wrote in ![[site community profile]](https://www.dreamwidth.org/img/comm_staff.png)
dw_maintenance2011-04-08 08:24 pm
denise) wrote in dw_maintenance2011-04-08 08:24 pm
LiveJournal slowdowns/DDoS affecting importer, crossposts
Importing
As I'm sure many people have noticed, the waiting period for our content importer is incredibly long right now. Usually, an import will take minutes; right now, the queue is probably at least 24 hours long, and is only getting longer, not shorter.
This is because most of the import jobs waiting in the queue are requesting data from LiveJournal. LiveJournal has been experiencing a Distributed Denial of Service (DDoS) attack on and off since March 24. This means that the site is often unreachable, either because of the network traffic brought about by the DDoS or because of tactics being taken to block the network traffic brought about by the DDoS. (In other words, sometimes being unable to reach LJ is because of the clogged network, and sometimes it's because of a 'false positive' identification as part of the problem and a temporary block.)
These temporary blocks and timeouts when accessing LJ affect the importer (and the crossposter!) as well. The importer and the crossposter both try multiple times before giving up, and each of those tries takes time. Sometimes an import job will get through, because it hits LJ at a time when traffic is lower. More likely, the import job will loop through its scheduled attempts, then fail.
What this means for you:
* Trying to import your journal right now is extremely delayed, and will likely only get worse. Please consider waiting a few days or a week to try to import your journal from elsewhere.
* If you do start an import job, it will take a very long time to reach the front of the queue. You will not receive any results from your import for at least 24 hours or so. It's okay to close the window or tab where you started the import. You'll receive a notice in your Dreamwidth Inbox when the job runs.
* It is very, very likely that your import job will fail. If your job reaches the top of the queue at a time when LJ is unreachable, the job will retry a few times, then error out. Please consider waiting a few days or a week to try again.
* Each individual step of the import runs under its own separate job. This means that some of your data may import while other bits won't. When the problem calms down, you can re-run your import and only bring in the data that it missed. (You can run an import as many times as you want. Each time will only pick up new content; it won't result in duplicates.)
* Edit, because people have asked: There's no way to cancel an import job once you start it. It's all right, though -- you can close the tab/window and log off, and the process will just stay in the queue until it's your turn.
Crossposting
Crossposts are also occasionally affected by LiveJournal being unreachable. If you try to crosspost to LiveJournal and the job fails, you'll receive a notice in your Dreamwidth Inbox (it should say something about the XML-RPC interface being unavailable). Once the problem calms down, you can "retroactively" crosspost the entry by using the Edit Entry link, making sure the crosspost option's box is ticked off, and save the entry. You don't need to make any changes to the entry itself.
Our fingers are crossed for LJ that they can overcome this problem soon! (Hopefully before their operations team has a nervous breakdown.)
As I'm sure many people have noticed, the waiting period for our content importer is incredibly long right now. Usually, an import will take minutes; right now, the queue is probably at least 24 hours long, and is only getting longer, not shorter.
This is because most of the import jobs waiting in the queue are requesting data from LiveJournal. LiveJournal has been experiencing a Distributed Denial of Service (DDoS) attack on and off since March 24. This means that the site is often unreachable, either because of the network traffic brought about by the DDoS or because of tactics being taken to block the network traffic brought about by the DDoS. (In other words, sometimes being unable to reach LJ is because of the clogged network, and sometimes it's because of a 'false positive' identification as part of the problem and a temporary block.)
These temporary blocks and timeouts when accessing LJ affect the importer (and the crossposter!) as well. The importer and the crossposter both try multiple times before giving up, and each of those tries takes time. Sometimes an import job will get through, because it hits LJ at a time when traffic is lower. More likely, the import job will loop through its scheduled attempts, then fail.
What this means for you:
* Trying to import your journal right now is extremely delayed, and will likely only get worse. Please consider waiting a few days or a week to try to import your journal from elsewhere.
* If you do start an import job, it will take a very long time to reach the front of the queue. You will not receive any results from your import for at least 24 hours or so. It's okay to close the window or tab where you started the import. You'll receive a notice in your Dreamwidth Inbox when the job runs.
* It is very, very likely that your import job will fail. If your job reaches the top of the queue at a time when LJ is unreachable, the job will retry a few times, then error out. Please consider waiting a few days or a week to try again.
* Each individual step of the import runs under its own separate job. This means that some of your data may import while other bits won't. When the problem calms down, you can re-run your import and only bring in the data that it missed. (You can run an import as many times as you want. Each time will only pick up new content; it won't result in duplicates.)
* Edit, because people have asked: There's no way to cancel an import job once you start it. It's all right, though -- you can close the tab/window and log off, and the process will just stay in the queue until it's your turn.
Crossposting
Crossposts are also occasionally affected by LiveJournal being unreachable. If you try to crosspost to LiveJournal and the job fails, you'll receive a notice in your Dreamwidth Inbox (it should say something about the XML-RPC interface being unavailable). Once the problem calms down, you can "retroactively" crosspost the entry by using the Edit Entry link, making sure the crosspost option's box is ticked off, and save the entry. You don't need to make any changes to the entry itself.
Our fingers are crossed for LJ that they can overcome this problem soon! (Hopefully before their operations team has a nervous breakdown.)
no subject
no subject
no subject
I'm trying to bring over one specific post with comments from my LJ, since I can't pick just one post and LJ-Sec doesn't do comments, I was figuring I'd have to do the import thing. But I really don't want duplicate posts from a few years ago. LOL.
Sorry to be a bother. :-)
no subject