Kei suddenly crashed. I got the techs to reboot it, and two minutes later it crashed again.
Rebooted again, and currently working, but disconnected from the net while I investigate.
Annika sez: i heard a rumor about a hacker threat?
Yes. It's a group of Turkish Islamist hacker-wannabe's going after The Jawa Report. They can't actually break in, so they've settled for a Distributed Denial of Service attack which takes us offline.
Unfortunately, the shared filesystem I built to keep us going in these situations decided to choose today to drop off its perch, so I wasn't able to bring everything back up on the other server. Right now, I'm copying everyone's files across, one blog at a time.
Which doesn't work for the Wordpress blogs, unfortunately. Which is why I never wanted to host a large number of Wordpress blogs. They're dead easy to set up, but it's a royal pain to manage them all.
Bah. Need chocolate.
Update: Okay, I think it's all fixed, except for The Jawa Report, which will be back as soon as the attack is over.
I have managed to separate things so that when an attack starts, we can just block it, and only The Jawa Report will go offline.
I finally worked out what was screwing everything up - the private network between the two servers went wonky. Now that that is fixed, we seem to be fine.
Posted by Pixy Misa at June 16, 2006 11:17 PM | TrackBackJawa Report is still down. Any new news?
How long do these things usually last?
It'll be over when it's over.
Thanks again for the hard work, Pixy.
Posted by Vinnie at June 17, 2006 10:58 AMPixy, as much as I love WordPress, I'll switch to Minx when everything is set up... I'd no idea it was a hassle - I'm so sorry!
Anything I can do to help, like backing up my database every day myself, etc...? Let me know, and I'll be happy to do whatever...
Thanks!
Sheesh, I was all ready to call my ISP back and do some hard-core screaming until I found Pixy's alternate site was also down.
There is at least some satisfaction that The Jawa Report is hitting the right nerves, but the important thing is to get the site back up so those people can continue to be pissed off. I doubt there's anything any of us can do that Pixy isn't doing, so again, and always,
Thank you Pixy.
(of course, if there is anything we can do just say the word.)
Posted by Debbye at June 17, 2006 11:40 AMPam - When everything works, WordPress isn't a problem. Infact, it uses a lot less resources than Movable Type.
The problem is, for every WordPress blog, we need another copy of the code and another database, and it gets hard to keep track of them all. That's why some of the WordPress blogs went awry after the first attack.
Posted by Pixy Misa at June 17, 2006 01:37 PMChris & Vinnie
I'll check in with the network guys.
At the moment we have The Jawa Report isolated from the rest of mu.nu. That means that when it gets attacked, we can shut down just The Jawa Report and everything else keeps running.
But if we get attacked and we don't shut down The Jawa Report, the server will die.
So we have to be pretty sure the current attack is over before switching it back on, or we risk losing everything again.
Posted by Pixy Misa at June 17, 2006 01:42 PMOkay: Our hosting company has their budget servers in stock again, so I'm going to grab one and put the Jawa Report on it. Once things settle down we can use it for other stuff as well, but for now, that will at least keep the attacks from wrecking everything else.
Posted by Pixy Misa at June 17, 2006 01:52 PMI think. I just left a query with the network guys to get their advice first.
Posted by Pixy Misa at June 17, 2006 02:16 PMWould it be possible to give Jawa Report an alternate URL that would still work when the main one is being blocked?
Posted by David Boxenhorn at June 17, 2006 06:31 PMDavid, there is one, mypetjawa.blogspot.com.
However, with Rusty gone, no one else has access to post there.
Posted by Vinnie at June 17, 2006 10:27 PMCan you investigate 'Round Robin' hosting on 3 servers to eliminate the possibility of such DDOS Attack?
Good work! Illegitimati Non Carborundum!
E Pluribus Chocolatum!
URLs are not the issue. IPs are the issue. If you give Jawa Report a different URL which resolves to the same IP, then the only effect is to lose Jawa Report all its readers because they can't find it any more.
Posted by Steven Den Beste at June 18, 2006 01:27 AMI've ordered a new server for the Jawa Report, but it could be up to 48 hours before everything is ready.
Hang in there!
Posted by Pixy Misa at June 18, 2006 04:38 AMURLs are not the issue. IPs are the issue.
All (most of?) the munu blogs are on the same IP, and I see that Pixy has managed to keep them up despite the DDOS attack on Jawa. So why not Jawa too, at another subdomain?
When Jawa gets its own server, could it be mirrored on one of the other servers for the inevitable DDOS incident?
Posted by David Boxenhorn at June 18, 2006 06:39 AMI could put the Jawa Report up at another URL, and will do so if Rusty asks. But if the attackers find the new URL and attack there, the server it is on will go down.
So I'd rather hold off until the new server is installed.
Posted by Pixy Misa at June 18, 2006 11:29 AMI thought Rusty was on vacation with a guest in charge. He finally got his fatwa and he missed it. poor guy
Posted by ken at June 19, 2006 07:15 AM