October 2010 IT Business Consulting Newsletter

Backup the Company Jewels!

By Tom K

We've discussed methods to achieve Zero Downtime throughout the last several articles, but at some point you WILL need to resort to Backup Data. It is all but inevitable! So you need to ensure your data sets are being backed up completely and regularly. This is Absolutely Critical!!! Your data sets ARE the Company Jewels!

This month's article shows you all you need to develop a backup strategy that will work for your company, including the Why, What, and How, along with Monitoring and those ever present "gotcha's".

Why Backup?

Regular backups protect your company against Data Loss. The three major data loss probabilities are:

  • Server loss. This can be due to the failure of one of the few non redundant components in your server, a fire or flood in your facility, or a natural catastrophe. This would require restoring all the data to a rebuilt or new server.
  • Data Corruption due to operating system or application malfunction. This would require restoring the corrupt/damaged data, which could be a simple file, or something more complex, like your complete email store, your Property Management data, or your financials.
  • Data Loss due to user error, user malice, or outside influence (viruses and malware). As above, this could be a simple file, or a complex data set.

In all instances, without a recent backup, the affected data is gone forever! With a recent backup, it is a mere inconvenience.

What to Backup?

Ultimately, we want to back up Everything, on All your Servers, Every Night! Our ability to do this is, however, dependent on the amount of data that needs to be backed up, the capacity of your backup system, and the time it takes for your backup system to backup all the data, referred to as the "Backup Window".

If your backup system can't backup everything in one pass, we need to assess if you need an improved backup system (costly), or if we can skip some of your data during the nightly backup. If necessary, we can separate "dynamic" data (operating data that changes frequently - ie: current reservation and financial data) from "static" data (archival data that doesn't change - ie: the 2008 brochure and the 2005 financials). Once separated, we backup the dynamic data nightly, while creating archival backups of the static data.

We also need to insure you are backing up each server's System State, and the Active Directory. If you lose a whole server, having the System State will greatly reduce the time it takes to get a replacement (or rebuilt) server up and running. If you lose your Active Directory, it could take days to re-configure this depending on the size of your company and the complexity of your environment. Most well regarded backup packages include facilities to back up this specialized data.

Types of BU Systems

We have on-site and off-site options, the on-site being most prevalent for core backups and off-site typically being used for specialty or secondary backups.

The on-site systems comprise Removable Media (typically tape and removable hard disks) and fixed backup systems like dedicated servers and Network Storage Systems (NAS).

We recommend Removable Media for your on-site Backups, so you can store backup data off site. Your backups won't help when they are in the same room as your server if a fire or flood occurs.

Removable disks generally could store more data and can run backup and restore jobs quicker than tape, but they are more expensive and tape is catching up. Current tape solutions can store over 1500 GB/tape, and can write a full tape in about 3 hours. This capacity & backup window is typically more than enough for the majority of businesses we've worked with in the Vacation Rentals industry.

As mentioned, we often utilize off-site backups of critical data as insurance. Off-site backups are also automated and send selected data to a vault in the Cloud/Cyber Space every night. This service costs less than $1/GB/yr, so for $100 - $200/yr you can maintain a copy of your critical data off-site, where it is very secure and can be accessed from anywhere.

BU Strategies

Backup Strategies relate to how you rotate your Backup Media, how long you keep backed up data, and where you keep it. As discussed above in "Why Backup", you only need last night's data to recover from a total server failure, but a broader strategy can provide you with data from five days ago before the Customer DB became corrupted, or provide that picture of your data from last September that your auditor needs!

Since most companies need more than simple recovery Backups, we recommend an often used strategy known as Grandfather/Father/Son (GFS) backups (monthly/weekly/daily). In this strategy, we run a separate tape for Mon - Sat (6 tapes) that are overwritten each week, so you have a picture of your data for each of the last 6 days. Then, we run a separate tape for each Sunday in the month (5 tapes), so you have an end-of-week picture of your data for the last 4 or 5 weeks. These tapes are overwritten each month. Finally, we run a separate tape on the last day of each month, providing an end-of-month picture of your data. These tapes are overwritten each year. This strategy requires 23 tapes (@ $30/tape, or $750).

A variation is to rotate one week of tape (7 tapes), along with 12 monthlies (19 total tapes). Another is to rotate 2 weeks of tapes without monthlies (14 tapes). As the cost differential is not extreme and the benefits of having a variety of backup data sets spread over time can be huge, we highly recommend the GFS Backup Strategy.

Once you have decided on a rotation strategy, you need to determine your storage strategy, which needs to be a compromise of keeping BU data close at hand if you need to recover a file, and keeping it safely off-site, away from localized danger. We generally recommend keeping the daily tapes in a fireproof safe in the building, but as far removed from the servers as possible. The weeklies and monthlies should be stored off-site but readily accessible, preferably in a Safety Deposit Box, although some clients store them in a fireproof safe in an owner's or manager's home.

We've seen companies utilize Auto-loaders which can hold and manage all the tapes required of their backup strategy. Each tape is selected and auto-loaded for each day in the strategy, so you never have to load a tape! Pretty cool, except tapes are never removed from the auto-loader, so they are never removed from the premises. As we highly recommend off-site storage, we can't recommend auto-loaders.

Monitor & Test your BUs

As with all of your critical systems, you need to ensure the BU jobs are running and completing as scheduled. Most BU applications can be configured to send status emails at the end of every job to advise the recipients that the job was either completed successfully, with exceptions, or was not successful. Additionally, most BU apps can be configured to email alerts if any BU sub-system fails. We highly recommend you do both.

We also recommend you actually test your BU system once/month. You need to load a recorded tape (I like to use weeklies) and "restore" a random file to a temp directory, just to be sure that the backed up files are, in fact, recoverable.

Final BU Considerations

All of our backup discussions concerned backing up data that lives on your servers, which is where all of your company data (the Company Jewels) should live. BUT, we always find a few Users storing company data in their local My Documents folder, which is the default for most Microsoft Office apps. No matter how hard we work at teaching Users to save company data in user/department/company folders on the server, key data files always end up in their local My Documents folder, and are lost forever when the PC hard drive fails (not uncommon).

So... We continue to help Companies develop centralized document filing systems and teach Users to file and save documents within these systems. But, we also set up automated "My Documents Redirection". This is a neat little trick that moves the User's My Documents folder to her personal storage space on the server, and redirects all files being saved to the User's My Documents to this space on the server. To the User, nothing appears changed. Everything works the same. She can open files from her My Documents and save files to her My Documents. Behind the scenes, however, the files are actually being read from and stored to the server, from where they are backed up every night!

As always, if you have any questions or comments concerning this article, I’d be happy to discuss them with you at your convenience. Feel free to contact me at TomK@TomKConsulting.com, or via my cell 443.310.5110.

Next month's newsletter will detail creating and managing a structured Company Data Storage Strategy, from how to structure the file system, to setting up security, to using Active Directory to simplify granting access to various folders within the file system. We will also describe how best to set up and use User Groups to simplify managing your Users and granting resources to all of your resources.
See "Simplify Data Organization and User Management... with Active Directory!"