$Id$
d2dbackup is a disk to disk backup utility designed to back up data from one or more source filesystems to a pool of large, cheap IDE disks.
For a conceptual introduction to d2dbackup, see the README.
For in-depth explanation of the perl code, see the docs in d2dbackup itself
For expanations of configuration variables / command-line options, see the docs in d2dbackup.conf.
Some day many of these steps might be automated with an install script or an install Makefile target. For now just copy the 2 files in question by hand.
Aquire some large cheap disks to do backups to. For a production environment, it would be best to use 3 or more disks.
Mount the backup partitions (you want one per disk) on mount points such that a filesystem glob will identify all and only the backup disks, so /backup/disk{1,2,3}/ is OK, but /backup1, /backup2, /mydrive/backup3 will not work. I suppose you could make this example work using symlinks, but we don't recommend it and will raise an eyebrow at you if you do.
WARNING: d2dbackup will push your network hardware, NFS implementation, and IDE controller and drive hardware to extremes most other software will not. Therefore you may find d2dbackup uncovering weaknesses in these areas that you were not aware of. In one case we had to replace a NIC because it was causing NFS errors, which previously had worked fine for a long time.
You will need the following perl modules. They are availble through CPAN (http://www.cpan.org).
Getopt::Mixed
Filesys::Df
File::Find version 1.04
Note: I was unable to find File::Find 1.04 in a distribution separate from Perl 5.8.
Time::HiRes
Note: this is only used for profiling and could be commented out if you prefer
d2dbackup has only been tested with Perl 5.8.0, but we expect it will work under most perl 5 installations. (File::Find v 1.04 might be a problem however. It is possible that d2dbackup will work even with an older version of File::Find, just a little slower. )
Copy the d2dbackup script somewhere convenient, maybe /usr/local/bin.
Copy d2dbackup.conf somewhere, perhaps /etc or /usr/local/etc. Or, if you prefer, for testing purposes d2dbackup will automatically find the configuration file in the current directory.
Edit d2dbackup.conf. Change configuration variables to suit your site. In particular, you MUST set $BDISKSTEM to reflect where your backup disks are.
Use NFS to have the backup and source file systems mounted and available on the machine where you will run d2dbackup.
When mounting the source filesystems, you probably want to use the NFS no_root_squash option. Otherwise, d2dbackup will only be able to read files readable by nobody. In the simplest case, there is no problem with mounting source file systems read-only (but see HOWTO.improve_performance).
You can also NFS-mount a snapshot in place of the directory. See HOWTO.snapshots.
If you are NFS-mounting the backup file systems, you have to do it read/write and with no_root_squash. This is because d2dbackup needs to be able to change the ownership and permissions of backup versions that it writes to the backup disks.
If you want, create a filesystem-specific configuration file for each source filesystem you want to back up. This file can contain any configuration settings normally found in d2dbackup.conf, and will override those. You typically would want to set $SOURCE and perhaps have a mapsrc2backup statement.
The filesystem-specific configuration file can live in any convenient directory, as long as you specify the path to it with the -f option to d2dbackup. Or, d2dbackup will autodetect a filesystem-specific configuration file named .d2dbackup.fs.conf in the top level of the source file system.
Run the first regular backup on each filesystem. Each time you run d2dbackup it operates on a single source file system. So you would run d2dbackup once on each source.
d2dbackup must run as root, because it sets the owner and mode of the backup versions it creates to the original owner and mode.
d2dbackup /sources/datafilesysor, if you set up a filesystem-specific configuration file,
d2dbackup -f datafilesys.confThere is no reason not to run the backup of several filesystems in parallel. So, supposing you have mounted all the source filesystems on mountpoints under /sources, then in bash syntax, you could do something like this:
for i ( /sources/* ); do d2dbackup -q $i & ; doneThe -q option tells d2dbackup to produce no output on STDOUT, instead emailing results (only on errors or warnings) to $ERRORMAIL.
Check the logs (/var/log/d2dbackup.log by default) for errors. Try to resolve any errors you find.
Set up a daily cron(8) job to run daily backups
Typically you would use crontab -e to edit root's crontab. You might put in a line like this:
5 0 * * * /usr/local/bin/d2dbackup -q /sources/datafilesys &Periodically, either from cron or by hand, run garbage collection on each file system:
d2dbackup -g /sources/datafilesysAlex Aminoff, alex_aminoff@alum.mit.edu
Copyright 2002-2004, shared by the National Bureau of Economic Research and Alexander Aminoff
Permission is granted to copy, modify, and use this software under the Gnu General Public License, found in the file LICENSE.