David Rusenko
  • Blog
  • Photography
  • About
  • Contact
David Rusenko

6 Easy Tips to Prevent Downtime

8/24/2007

 

I keep noticing startups (and larger companies, too) making some really basic mistakes that end up leading to a lot of downtime. Here's my list of 6 really easy things you can do to avoid major downtime.

1. Buy backup DNS service
This is so cheap it's a no-brainer. For about $15/year you can get a service that will constantly grab your DNS data and act as a backup if they happen to go down. Otherwise, when your DNS servers go haywire (it's happened to me and I've seen it happen to many others), you'll be stuck helpless for a few hours as people are unable to get to your site. [I've used No-IP Squared Backup, and Chris has used Nettica].

2. Buy a monitoring service
For $5/month, you can purchase a service that pings your servers every few minutes and sends you a text message if they go down. This is absolutely crucial, especially if things go to hell in the middle of the night, or any other time you might not normally be checking. Make sure to buy a service that monitors from at least 3 locations -- there's nothing worse than a few false alarms in the middle of the night, after which you won't get up for the real thing. [I've been happy with WebSitePulse -- their prices are a bit more expensive now, but you should be able to get them down to $5/month on the phone.]

3. Always make database backups before touching the database
It's one of those things you always consider and dismiss right before you bring the whole thing crashing down. Especially if you aren't making very regular backups (note: you should be), make sure to do so before you get your hands dirty. (Ever forget the WHERE clause? Not fun...)

4. Be VERY careful around power cords at Colos
Knocking out a power cord seems to happen consistently if you don't make a very concerted effort not to. It's extremely easy for a cord to jiggle a tiny bit, or for one moving server to pull on another cord in just the right way. Always plan out your server trajectories before you move them, or have someone to hold the power cords in. [Note: Why aren't snap-in power cords standard for rack mountable servers??]

5. Make your site functional in pieces
Even if your database is down, there's no reason your home page shouldn't still show, or any of your other static pages. There's a big difference from a user's point of view in between an otherwise seemingly functional site that shows a nice looking error message, and a site that spits out errors, is not accessible, or won't load at all. If Weebly's database goes down, users will see a polite "Sorry, something is wrong and we're fixing it right now" error message. Our site, blog, and all hosted user sites stay up, so a database crash just means that people can't edit their sites at the moment.

6. Use source management to roll out updates
We use darcs to manage our different source repositories, and it's a flexible and distributed system that works very well for us. Whatever you use, make sure you use some automated process to roll out updates (which doesn't include moving a directory and moving another in it's place, and, God forbid, manually diff'ing files -- there's always more to that than you anticipate). It's quite shameful to see pretty basic sites go down for hours (or days, or weeks) rolling out an update. If you're using Weebly while we push out an update, your session will automatically be refreshed without any loss of data, and you'll be up and running on the new version within seconds (with no downtime). [Darcs can be found at http://darcs.net/]

Those 6 items combined have probably caused over 80% of downtime I've been responsible for. What's your list?

Hovhannes Avoyan link
8/25/2007 05:56:35 am

Hovhannes Avoyan link
8/25/2007 05:58:08 am

Mon.itor.Us (http://mon.itor.us) is free website monitoring service, monitoring from 3 different locations and sending IM, email notifications.

eithkhad link
8/26/2007 06:18:13 pm

I really like this list, the 3rd tip about making a backup of the database before touching it, it's more easy to say than to do. Cron jobs can help also. Make a backup of the whole website regularly, specially if you work only on the server, change the whole code, and forget to back it up. I took it as an habit. Discipline is requested.

Sierra link
8/27/2007 12:03:23 pm

I don't understand a word of what you just typed.

Stephen Hebert
8/28/2007 02:05:37 am

Not that I don't agree with #3 (because I do), but it helps to type the WHERE clause first when you're working live, just to eliminate the possibility that you might accidentally hit ENTER after typing the UPDATE or DELETE bit.

Website Design link
9/15/2007 03:11:07 pm

Good list. I can't believe the amount of downtime some people have. Blows me away. Anyway, good list of things to keep track of and be weary of.

Bob Clarke
11/5/2007 02:17:33 am

This is a great list. You could probably
write a great step-by-step how-to book on doing a web start-up. There are a lot of newbees out there who have no idea about cost, logistics, or implementation -- why not give them some pointers?

adrian tyler link
1/28/2008 09:28:27 pm

ah.. I hate downtimes.. thanks for your post!!

David Ghuerro
6/22/2008 07:47:53 pm

I use www.justuptime.com to monitor my servers. They arn't free but provide a far better service than the free ones and at a resonable price.Never heard of websitepulse.com before.

SEO link
7/24/2008 06:11:51 pm

Very good list, I didn't know there was such thing as a monitoring service, great news and also professional backup solutions.

spenser link
1/23/2009 04:17:43 pm

All very good points in the original article. Each of them should be mandatory in a web environment, just as they are in enterprise environments.

But, the best tip was from Mr. Hebert, typing the WHERE clause before the rest of the query. You have to think about the limits anyways, so why not type it in as you dream it up.

Another way to do this is to start with a BEGIN TRANSACTION. Do the work, then finally, execute a COMMIT or ROLLBACK as required.

Make Money On The Internet link
1/25/2009 09:10:42 am

Making backups is very important. I learned the hard way, as i deleated about 17 post from my blog in the early stages.

I was so upset but was lucky because my host had a backup but charged me $30 to restore it.

I always backup my blog now.

Ppc Optimization Expert link
1/25/2009 09:39:19 am

I learnt the hard way my whole blog got wiped out

this happened years ago and Ive learnt my lesson

It a most that I always backup my PC and Blog now.

Auctions @ Bid U Like link
2/7/2009 02:05:52 am

Our whole server went down but luckily enough my web developer automatically backs our site throughout the day. Very Good Post


Comments are closed.
    Picture
    David co-founded Weebly, an incredibly easy to use tool that helps millions of people create a professional web site, blog or online store.

    He was named to Forbes'  30 under 30 list, is a part-time DJ and has traveled to over 20 countries.

    Investments include Cue, Parse, Exec, Churchkey, Streak, Incident Technologies, Adioso and Zenefits.

    RSS Feed


    Categories

    All
    Bobbyore
    Day To Day
    Misc
    Music
    Open Source
    Product Reviews
    Raising Money
    Rant
    San Francisco
    Scaling
    Startups
    Troubleshooting

    Blogroll

    Jessica Livingston
    Robby Walker
    Adam Smith

    Justin.tv
    Venture Hacks
    Uncrate
    Juno Day

    Flickr Photos

Proudly powered by Weebly