OT: Backup Software Recommendations?

[EvanK]

My employer has me looking into centralized network backup software. I'm certainly not the first person to find themselves at this precipice, and I'm hoping I can learn from other monks' experiences. I realize this of course has nothing to do with Perl or even programming generally, but I find the quality of advice here far outweighs any other online portals I frequent. Plus, perhaps the advice here will be helpful to other monks in the same situation :)

Previously, we'd been using Bacula (freeware) from a linux server, backing up other heterogenous servers (Linux, Windows) via a terminal session, but as these things go, we've outgrown it. We're needing to backup not only the servers, but individual PCs now, too, and while they're in use, no less. We even need to preserve permissions now. Rather than tooling around and trying to put something together ourselves, we've convinced the boss to buy *gasp* a retail solution.

Initially, the powers-that-be had me looking at BackupExec, formerly Veritas, and the latest version seems to be laden with issues and falls short on several fronts (can't copy in-use files for the individual PCs, and Symantec's notorious customer support, to name a couple).

So I guess the "must-have" features would be:

• Cross platform (WinXP, Win2K3, Redhat Linux)
• Preserves permissions/ACLs for above OSes
• Central administration for servers and workstations
• Ability to backup files while in use (everyone turns their PCs off at night, so we have to backup during the day)

And of course, the "nice-to-haves":

• Bit-level backup (like rsync)
• Continuous backup of file changes (ie: no having to schedule backup windows)
• Backup/restore of Windows registry data
• Backup/restore for Active Directory
• Backup/restore for Exchange Server (ugh, not my choice to use it)
• Backup/restore for assorted DBMSes (ie: MySQL, MSSQL, Oracle, etc)
• Support for Mac workstations (darn graphic designers...)

I'd asked earlier in the chatterbox, but the question was lost in the fray...If anyone has recommendations, you don't even know how much I'd appreciate it.

[dragonchild]

I don't have much to offer except you don't want to backup files that are in use. That's just asking for trouble, particularly on NTFS. Much better to have the workers leave their PCs on overnight so that you can take a pristine snapshot. (Hibernate doesn't take that much power and a proper backup solution will wake and rehibernate the computer.)

[EvanK]

Normally I'd agree, but some of the apps we use (3rd party stuff) keep a file open and locked even when its read-only. And if a user forgets to close out of the application before they go home, we're up a certain creek...

That's why I think the better solution is something that monitors disk writes and backs up changes at a bit-level. I do appreciate the input :)

[diotalevi]

If you're backing up files that are in use, you'll want backup software that knows how to work with the application that's using it. Fer instance, lots of server-side applications keep stuff open all the time so server-capable backup software can communicate to the running processes to negotiate a safe backup.

[tphyahoo]

Okay, I have the same problem myself and, to be honest, I haven't implemented a solution yet. Just done research. But I'll share my findings.

http://del.icio.us/tphyahoo/backup

My current opinion (to summarize what you get from my bookmarks): if you're okay spending a little money for a commercial service, and want user friendly, windows oriented (tho probably works for nix too): carbonite. Pretty cheap, lotsa buzz, looks pretty good.

http://www.carbonite.com/

If you want a DIY solution,

http://backuppc.sourceforge.net/info.html

good thread on this at

http://www.aaronsw.com/weblog/lazybackup

[glasswalk3r]

I believe Amanda may have the features you're looking for. Here are some links:

• http://www.backupcentral.com/thebook.html
• http://www.amanda.org
• http://sourceforge.net/projects/amanda-win32

[tcf03]

If you have $$ to spend - I would check out Avamar - http://www.avamar.com/ Its by far one of the best I have ever used. It runs on linux and can back up windows and many other Unix flavors. EMC just bought them recently. They also make good use of Perl in their product. That being said the company I work for now - does NOT have $$ to spend, but we do use HP Data protector ( formerly omniback ) We are using version 6.0 on Linux - which is their first version to run ( server side ) linux. I am quite happy w/DP but they have failed miserably in their port to Linux.

[cosimo]

Take a look at Bacula.


"It comes by night and sucks the vital essence from your computers".

I'm using it since last year and it rocks.
It has what you ask and some more. Works on Macs.
Allows to backup Windows workstations with VSS shadow copy, even if I never managed to make that work: my fault.

Main site: http://www.bacula.org
There's a fine users mailing list also.

Ah, it's free and open source.
Or you can look for commercial solutions...

[jkva]

I find BackupPC to suit my needs very well. However, I would not know out of the top of my head if it suits your needs. I'd gather it's worth a look at least, though.

Good luck!

[telcontar]

Just thought I'd add a note about backing up local machines. I think that's generally the wrong approach, and an indication that you might have set up your infrastructure incorrectly. Now I might be wrong, as I do not know your setup exactly, and I do not even know what OS your clients run on.

But, if they are windows, you really should map "my documents" and the desktop to a network drive on a file server. That's best practice in such environments (yaheee, Active Directory), it's done with GPOs and Folder Redirection.

I've set this up more than once and it works quite well. Backups while machines run can be done with Volume Shadow Copy Service. Etc ...

If your clients run on *NIX, use NFS. But data should be stored in a central location, preferably on a server with at least mirroring RAID (depends on the $$$, but you RAID1 is cheaper than data loss ...).

As for your requirements:
• You can script AD backup, but I would generally make sure to have it documented properly and that your domain controllers are redundant. Have you ever tried using a backup to reinstall an Active Directory-enabled domain controller? Ugh ... the pain, the headaches ...
• ntbackup can store registry data and system state
• If you use Volume Shadow Copy Service, you should even be able to back up DBMSs while they are running if the data on disk is in a consistent state, but you'd have to consult the DBMS's docs on this. I'd script this, use ntbackup.
• Exchange server .. tough one. use ntbackup or, if you want to buy something, try veritas (symantec) or tivoli.
• Macs .. you can get those on a windows fileserver! Really! Try it, tell them to store their shit there, and back up only their data ;-)
Try ntbackup for the windows stuff. Do full backups daily and rotate them every week, and do a weekly backup that you'll keep on top of that. That's my suggestions but I don't know your setup well enough. Apologies if it's not applicable to your environment :-) Johannes

[EvanK]

Unfortunately, it was a case where the "infrastructure" (i hesitate to even call it that) was set up long before I came into the picture. You make some very good suggestions though, and I'm going to suggest them to my boss.

I particularly like the (windows) idea of mapping certain system folders to network drives. That would work very well until we can move to a thin-client system, which is my ultimate goal.

[RyuMaou]

As someone else mentioned, if you have $$ to spend, check out SyncSort. They should be able to do everything you're looking to accomplish. I've never actually done all those things myself with SyncSort, but most of them. After it's installed and configured, regular maintenance was pretty simple.

For your convenience, they're at www.syncsort.com

Good luck!

[0xbeef]

All your "must-have" requirements are fullfilled by the heterogeneous enterprise backup solution from IBM called Tivoli Storage Manager.

1. It probably has the best cross-platform support out there of any of the major backup solutions.
2. It preserves permissions & ownership.
3. It has a centralised RDBMS based server console (with SQL support) and a separate GUI-based product called Integrated Server Console.
4. TSM provides several different options for open file backups (it calls this "serialization").

Update: I second Bacula as an open-source solution.

Niel

[shandor]

I am probably quite biased, since this is how I've made my living for the past seven years, but if your company can afford
the licensing fees, I highly recommend EMC (formerly Legato) NetWorker. Of the big three (NetWorker, TSM, NetBackup), NetWorker is
the one I use. Besides that, I find that NetWorker works very well in a large scale, heterogeneous environment.


I've set up backup environments for small data centers (25-50 servers) and large data centers (1500+ servers) and have yet to find an
architecture that NetWorker doesn't support. I'm sure they are out there, but the hardware/OS compatibility of NetWorker is quite vast.


Pro:
- Once you get the hang of the vocabulary and how NetWorker works, it's very simple to set up backups and perform restores.
- The internal scheduling tool works very well. Most Unix admins will start by scheduling jobs manually in CRON, until they're
convinced that NetWorker's scheduling is more efficient... then all those CRON jobs are #'d out forever.
- There is a very active online support community http://listserv.temple.edu/archives/networker.html with some great expertise.
- The official support staff isn't nearly as horrible as it used to be 5 years ago and can actually be helpful (I've heard) now a days.
- There are many modules (and applicable license fee, of course) that allow NetWorker to interact directly with active applications
(Documentum, Oracle, MSSQL, etc...) to allow for little to no downtime.
- Extensive command line abilities making it very easy to write wrapper scripts for tedious tasks or to make very pretty, manager
readable reports.


Cons:
- The learning curve can be steep. I give new team members 6 months before I expect them to be able to fully grasp the complexities
of NetWorker (that time is lenghtened because they have to learn the infinite amount of corporate policies as well).
- I don't think backing up laptops and PCs is very well supported.
- I think the licensing fees can be prohibitive for smaller companies.
- NetWorker is pretty bad at interacting with tape devices. For example, if a person physically moves a tape from a tape drive, it
can some times take several hours of kicking and screaming to get NetWorker to realize that the tape isn't in the drive anymore.
(This is by far my biggest gripe with NetWorker and something I like about NetBackup)


NetWorker is overkill if you're only backing up a few GB per day or a couple hundred GB per week. But, if you have a pretty
heterogeneous data center and your companies is concerned about disaster recovery, this is a very good choice for your backup
solutions.

[wazoox]

Frankly, BackupExec quite sucks. Most commercial softwares work OK, however I found that Bakbone NetVault is the best. It works on virtually all platforms in existence including Mac OS X, z/OS, Linux, Solaris, Windows and whatever; it has plugins to backup anything, Lotus Notes, Oracle, DB/2 or Microsoft Exchange, and many others; it can backups to tapes and disks; and the demo is feature complete and downloadable from the website (time limited demo).

There is one thing to keep in mind however : the setup of a powerful backup system for a couple hundreds computers costs lots of money and takes weeks, so you'll have to buy it from a competent specialist or you'll have lot of trouble (and it will take much, much longer or worse, will appear unusable the day you'll have a really big problem).

[Moron]

It shouldn't be too hard to grow your own * using File::Find and Net::FTP

Update: * insert: "heterogeneous network extension to a single-environment utility such as bacula"