Installation instructions for BackupPC4AFS

The sample commands in this document assume you're running a recent version of Debian. If you're running some other linux distribution or unix flavor, you'll need to substitute the appropriate commands.

Most of these commands will need to be completed as root. Text in blue are commands which would be typed. As always, please do a sanity check that the instructions are correct for your setup before blindly following them.

  1. Disable current AFS backups

    This step is optional, but recommended. BackupPC4AFS performs a "vos backup" of each volume to create a new .backup volume to immediately dump. This ensures that the dump is the freshest possible. If no other process in the cell is creating .backup volumes (such as a "vos backupsys" cron job), then it is also possible to very quickly tell when a volume was last dumped by BackupPC4AFS by simply looking at the Backup time in the output of "vos examine volume".

    If you plan to run BackupPC4AFS alongside another vos dump or afs backup based mechanism, you should take steps to ensure that volume operations (vos backup, vos dump, and backup dump) do not attempt to operate on the same .backup volume at the same time to prevent errors due to volume locking.

  2. Install apache

    Debian offers a version of apache with SSL. It is not advised to send usernames and passwords over a non SSL-encrypted http session.

    vm1: # apt-get install apache-ssl
    Reading package lists... Done
    Building dependency tree... Done
    The following extra packages will be installed:
      apache-common apache2-utils ssl-cert
    Suggested packages:
      apache apache-perl apache-doc
    The following NEW packages will be installed:
      apache-common apache-ssl apache2-utils ssl-cert
    0 upgraded, 4 newly installed, 0 to remove and 4 not upgraded.
    Need to get 1696kB of archives.
    After unpacking 4764kB of additional disk space will be used.
    Do you want to continue [Y/n]? y
    Get:1 http://a-deb-mirror etch/main apache2-utils 2.2.3-4 [341kB]
    Get:2 http://a-deb-mirror etch/main apache-common 1.3.34-4.1 [848kB]
    Get:3 http://a-deb-mirror etch/main ssl-cert 1.0.14 [11.1kB]
    Get:4 http://a-deb-mirror etch/main apache-ssl 1.3.34-4.1 [496kB]
    Fetched 1696kB in 0s (6228kB/s)
    Preconfiguring packages ...
    Selecting previously deselected package apache2-utils.
    (Reading database ... 88007 files and directories currently installed.)
    Unpacking apache2-utils (from .../apache2-utils_2.2.3-4_i386.deb) ...
    Selecting previously deselected package apache-common.
    Unpacking apache-common (from .../apache-common_1.3.34-4.1_i386.deb) ...
    Selecting previously deselected package ssl-cert.
    Unpacking ssl-cert (from .../ssl-cert_1.0.14_all.deb) ...
    Selecting previously deselected package apache-ssl.
    Unpacking apache-ssl (from .../apache-ssl_1.3.34-4.1_i386.deb) ...
    Setting up apache2-utils (2.2.3-4) ...
    Setting up apache-common (1.3.34-4.1) ...
    
    Setting up ssl-cert (1.0.14) ...
    
    Setting up apache-ssl (1.3.34-4.1) ...
    
    Creating config file /etc/apache-ssl/httpd.conf with new version
    
    Creating config file /etc/apache-ssl/srm.conf with new version
    
    Creating config file /etc/apache-ssl/access.conf with new version
    
    Creating config file /etc/apache-ssl/modules.conf with new version
    Starting apache-ssl 1.3 web server....
    
    vm1:~#
    
  3. Stop the apache server (we will customize it shortly)
    vm1:~# /etc/init.d/apache-ssl stop
    Stopping apache-ssl 1.3 web server....
    vm1:~#
    
  4. Create a backuppc user
    vm1:~# adduser --home /home/backuppc4afs --shell /bin/bash backuppc4afs
    Adding user `backuppc4afs' ...
    Adding new group `backuppc4afs' (1000) ...
    Adding new user `backuppc4afs' (1000) with group `backuppc4afs' ...
    Creating home directory `/home/backuppc4afs' ...
    Copying files from `/etc/skel' ...
    Enter new UNIX password: 
    Retype new UNIX password: 
    passwd: password updated successfully
    Changing the user information for backuppc4afs
    Enter the new value, or press ENTER for the default
            Full Name []: BackupPC4AFS User
            Room Number []: 
            Work Phone []: 
            Home Phone []: 
            Other []: 
    Is the information correct? [y/N] y
    vm1:~#
    
  5. Create a directory for data storage and set the ownership

    Feel free to change this location. It is critical that it either have sufficient space for the data you plan to back up (fulls plus incrementals) or that you mount another disk or partition here which does. Mounting a large RAID array at this location is recommended.

    vm1:~# mkdir -p /var/backuppc4afs
    vm1:~# chown backuppc4afs:backuppc4afs /var/backuppc4afs
    vm1:~# chmod 700 /var/backuppc4afs
    vm1:~#
    
  6. Download the BackupPC4AFS source

    Visit http://www.sourceforge.net/projects/backuppc4afs and click the "Download BackupPC4AFS" link. Save the file to a convenient location such as /home/backuppc4afs.

  7. Unpack and install BackupPC4AFS

    As of the date this document was written, the newest release is version 3.0.0.2. Please use the newest version of BackupPC4AFS available.

    Change to the download directory and unpack the distribution

    vm1:/home/backuppc4afs# tar -zxvpf BackupPC4AFS-3.0.0.2.tar.gz 
    BackupPC4AFS-3.0.0/
    BackupPC4AFS-3.0.0/LICENSE
    BackupPC4AFS-3.0.0/cgi-bin/
     < ... full fill listing truncated for brevity ... >
    BackupPC4AFS-3.0.0/images/0011001.gif
    BackupPC4AFS-3.0.0/images/logo-orig.gif
    vm1:/home/backuppc4afs#
    

    Perform the installation. The cgi-bin directory shown below is correct for apache on Debian Etch. Please change it to your distribution's value if it's different.

    vm1:/home/backuppc4afs# cd BackupPC4AFS-3.0.0
    vm1:/home/backuppc4afs/BackupPC4AFS-3.0.0# perl ./configure.pl
    
    Is this a new installation or upgrade for BackupPC?  If this is
    an upgrade please tell me the full path of the existing BackupPC
    configuration file (eg: /etc/BackupPC/config.pl).  Otherwise, just
    hit return.
    
    --> Full path to existing main config.pl []? 
    
    I found the following locations for these programs:
    
        bzip2        => /bin/bzip2
        cat          => /bin/cat
        df           => /bin/df
        gtar/tar     => /bin/tar
        gzip         => /bin/gzip
        hostname     => /bin/hostname
        nmblookup    => 
        par2         => 
        perl         => /usr/bin/perl
        ping         => /bin/ping
        rsync        => /usr/bin/rsync
        sendmail     => /usr/sbin/sendmail
        smbclient    => 
        split        => /usr/bin/split
        ssh/ssh2     => /usr/bin/ssh
    
    --> Are these paths correct? [y]? y
    
    Please tell me the hostname of the machine that BackupPC will run on.
    
    --> BackupPC will run on host [vm1]? vm1
    
    BackupPC should run as a dedicated user with limited privileges.  You
    need to create a user.  This user will need read/write permission on
    the main data directory and read/execute permission on the install
    directory (these directories will be setup shortly).
    
    The primary group for this user should also be chosen carefully.
    The data directories and files will have group read permission,
    so group members can access backup files.
    
    --> BackupPC should run as user [backuppc]? backuppc4afs
    
    Please specify an install directory for BackupPC.  This is where the
    BackupPC scripts, library and documentation will be installed.
    
    --> Install directory (full path) [/usr/local/BackupPC]? /usr/lib/BackupPC4AFS
    
    Please specify a data directory for BackupPC.  This is where all the
    PC backup data is stored.  This file system needs to be big enough to
    accommodate all the PCs you expect to backup (eg: at least several GB
    per machine).
    
    --> Data directory (full path) [/data/BackupPC]? /var/BackupPC4AFS
    
    BackupPC can compress pool files, but it needs the Compress::Zlib
    package installed (see www.cpan.org). Compression will provide around a
    40% reduction in pool size, at the expense of cpu time.  You can leave
    compression off and run BackupPC without compression, in which case you
    should leave the compression level at 0 (which means off).  You could
    install Compress::Zlib and turn compression on later, but read the
    documentation first about how to do this.  Or the better choice is
    to quit, install Compress::Zlib, and re-run configure.pl.
    
    --> Compression level [0]? 0
    
    BackupPC has a powerful CGI perl interface that runs under Apache.
    A single executable needs to be installed in a cgi-bin directory.
    This executable needs to run as set-uid backuppc4afs, or
    it can be run under mod_perl with Apache running as user backuppc4afs.
    
    Leave this path empty if you don't want to install the CGI interface.
    
    --> CGI bin directory (full path) []? /usr/lib/cgi-bin
    
    BackupPC's CGI script needs to display various GIF images that
    should be stored where Apache can serve them.  They should be
    placed somewher under Apache's DocumentRoot.  BackupPC also
    needs to know the URL to access these images.  Example:
    
        Apache image directory:  /usr/local/apache/htdocs/BackupPC
        URL for image directory: /BackupPC
    
    The URL for the image directory should start with a slash.
    
    --> Apache image directory (full path) []? /var/www/BackupPC
    --> URL for image directory (omit http://host; starts with '/') []? /BackupPC
    
    Ok, we're about to:
    
      - install the binaries, lib and docs in /usr/lib/BackupPC4AFS,
      - create the data directory /var/BackupPC4AFS,
      - create/update the config.pl file /etc/BackupPC/config.pl,
      - optionally install the cgi-bin interface.
    
    --> Do you want to continue? [y]? y
    Created /usr/lib/BackupPC4AFS/bin
    Created /usr/lib/BackupPC4AFS/doc
    Created /usr/lib/BackupPC4AFS/lib/BackupPC/CGI
    Created /usr/lib/BackupPC4AFS/lib/BackupPC/Config
    Created /usr/lib/BackupPC4AFS/lib/BackupPC/Lang
    Created /usr/lib/BackupPC4AFS/lib/BackupPC/Storage
    Created /usr/lib/BackupPC4AFS/lib/BackupPC/Xfer
    Created /usr/lib/BackupPC4AFS/lib/BackupPC/Zip
    Created /var/www/BackupPC
    Created /var/BackupPC4AFS
    Created /var/BackupPC4AFS/pool
    Created /var/BackupPC4AFS/cpool
    Created /var/BackupPC4AFS/pc
    Created /var/BackupPC4AFS/trash
    Created /etc/BackupPC
    Created /var/log/BackupPC
    Installing binaries in /usr/lib/BackupPC4AFS/bin
    Installing library in /usr/lib/BackupPC4AFS/lib
    Installing images in /var/www/BackupPC
    Making init.d scripts
    Installing docs in /usr/lib/BackupPC4AFS/doc
    Installing config.pl and hosts in /etc/BackupPC
    PING localhost (127.0.0.1) 56(84) bytes of data.
    64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.026 ms
    
    --- localhost ping statistics ---
    1 packets transmitted, 1 received, 0% packet loss, time 0ms
    rtt min/avg/max/mdev = 0.026/0.026/0.026/0.000 ms
    Installing cgi script BackupPC_Admin in /usr/lib/cgi-bin
    
    Ok, it looks like we are finished.  There are several more things you
    will need to do:
    
      - Browse through the config file, /etc/BackupPC/config.pl,
        and make sure all the settings are correct.  In particular,
        you will need to set $Conf{CgiAdminUsers} so you have
        administration privileges in the CGI interface.
    
      - Edit the list of hosts to backup in /etc/BackupPC/hosts.
    
      - Read the documentation in /usr/lib/BackupPC4AFS/doc/BackupPC.html.
        Please pay special attention to the security section.
    
      - Verify that the CGI script BackupPC_Admin runs correctly.  You might
        need to change the permissions or group ownership of BackupPC_Admin.
        If this is an upgrade and you are using mod_perl, you will need
        to restart Apache.  Otherwise it will have stale code.
    
      - BackupPC should be ready to start.  Don't forget to run it
        as user backuppc4afs!  The installation also contains an
        init.d/backuppc script that can be copied to /etc/init.d
        so that BackupPC can auto-start on boot.  This will also enable
        administrative users to start the server from the CGI interface.
        See init.d/README.
    
      - To use the AFS features, be certain to read the README-AFS file.
    
    Enjoy!
    
  8. Installation final steps

    Edit /etc/BackupPC/config.pl and change/correct the location of voswrap.pl near the end of the file.

    In the example above, BackupPC4AFS was installed into /usr/lib/BackupPC4AFS. If you changed the location, make sure to use the correct value.:

    BEFORE:
      # AFS additions
      # document these
      # Path to the voswrap.pl script on the server
    - $Conf{VosPath} = '$Installdir/bin/voswrap.pl';
    
      # Full command to run voswrap.pl on the server for a backup.
      $Conf{VosCmd} = '$voswrapPath \\
    
    AFTER
      # AFS additions
      # document these
      # Path to the voswrap.pl script on the server
    + $Conf{VosPath} = '/usr/lib/BackupPC4AFS/bin/voswrap.pl';
    
      # Full command to run voswrap.pl on the server for a backup.
      $Conf{VosCmd} = '$voswrapPath \\
    
  9. Do some post-install cleanup

    Debian's apache doesn't support suid cgi-bins.

    vm1:/home/backuppc4afs/BackupPC4AFS-3.0.0# chmod u-s /usr/lib/cgi-bin/BackupPC_Admin 
    vm1:/home/backuppc4afs/BackupPC4AFS-3.0.0# ls -la /usr/lib/cgi-bin/
    total 48
    drwxr-xr-x  2 root         root          4096 2007-05-24 00:01 .
    drwxr-xr-x 83 root         root         40960 2007-05-24 00:01 ..
    -r-xr-xr--  1 backuppc4afs backuppc4afs  3997 2007-05-24 00:01 BackupPC_Admin
    

    Make apache run as the backuppc4afs user. Edit /etc/apache-ssl/httpd.conf and change the User and Group.

    262,263c262,263
    - User www-data
    - Group www-data
    ---
    + User backuppc4afs
    + Group backuppc4afs
    

    BackupPC4AFS uses apache's REMOTE_USER variable to control access. This can be set using any existing authentication method, but is normally set via an .htaccess file. Enable .htaccess files for the apache cgi-bin directory by adding an AllowOverride to the cgi-bin Directory entry:

    BEFORE:
          <Directory /usr/lib/cgi-bin/>
    -         AllowOverride None
              Options ExecCGI -MultiViews +SymLinksIfOwnerMatch
              Order allow,deny
              Allow from all
          </Directory>
    
    AFTER
          <Directory /usr/lib/cgi-bin/>
    +         AllowOverride All
              Options ExecCGI -MultiViews +SymLinksIfOwnerMatch
              Order allow,deny
              Allow from all
          </Directory>
    
  10. Create an appropriate .htaccess file in the cgi-bin directory

    Review the instructions for setting up a generic or kerberized .htaccess file.

  11. Add BackupPC4AFS admin users

    Add one or more users as a BackupPC4AFS admin. This is done by adding them to the $Conf{CgiAdminUsers} variable in /etc/BackupPC/config.pl. Separate multiple users with spaces.

    non-kerberized example

    $Conf{CgiAdminUsers} = 'backuppc4afs otheruser';
    

    kerberized example

    $Conf{CgiAdminUsers} = 'someuser@MY.KRB5REALM.COM someotheruser@MY.KRB5REALM.COM ';
    
  12. Start apache
    vm1:~# /etc/init.d/apache-ssl start
    Starting apache-ssl 1.3 web server....
    vm1:~#
    
  13. Activate backuppc4afs init script and start the daemon
    vm1:~# cp /home/backuppc4afs/BackupPC4AFS-3.0.0/init.d/debian-backuppc /etc/init.d/backuppc
    vm1:~# update-rc.d backuppc4afs defaults
     Adding system startup for /etc/init.d/backuppc4afs ...
        /etc/rc0.d/K20backuppc4afs -> ../init.d/backuppc4afs
        /etc/rc1.d/K20backuppc4afs -> ../init.d/backuppc4afs
        /etc/rc6.d/K20backuppc4afs -> ../init.d/backuppc4afs
        /etc/rc2.d/S20backuppc4afs -> ../init.d/backuppc4afs
        /etc/rc3.d/S20backuppc4afs -> ../init.d/backuppc4afs
        /etc/rc4.d/S20backuppc4afs -> ../init.d/backuppc4afs
        /etc/rc5.d/S20backuppc4afs -> ../init.d/backuppc4afs
    vm1:~# /bin/sh /etc/init.d/backuppc4afs start
    Starting backuppc: ok.
    vm1:~# 
    
  14. Access the web interface

    Visit https://your.host.name.dom/cgi-bin/BackupPC_Admin and log in using the username and password you previously added to your .htaccess file.

    If you do not see the prompt for username and password, double-check your .htaccess setup and file permissions. The user running apache (backuppc4afs in this example) must be able to read the .htaccess file and the .htpasswd file if it exists.

    Once logged in, if you don't see all 12 links under "Server", including "Edit Config" and "Edit Hosts", double-check that the username you're using is one of the listed CgiAdminUsers in config.pl and that you restarted backuppc4afs after making any changes to config.pl.

  15. Add AFS cell keyfile

    In order to perform vos dump operations without tokens, we need to copy the AFS cell's keyfile to the backuppc4afs server. This keyfile must be protected. Only the backuppc4afs user should be able to read it. This means that the backuppc4afs server should be kept as secure as your AFS fileservers. This install document won't devolve into a security lecture, but suffice it to say you should turn off all unnecessary services and wrap or firewall the remaining services.

    vm1:~# mkdir -p /etc/openafs/server
    vm1:~# chmod 700 /etc/openafs/server
    vm1:~# cd /etc/openafs
    vm1:/etc/openafs# scp -R root@fs1:/etc/openafs/server server/
    root@fs1's password:
    KeyFile                                       100%   16     0.0KB/s   00:00    
    ThisCell                                      100%   16     0.0KB/s   00:00    
    CellServDB                                    100%   16     0.0KB/s   00:00    
    UserList                                      100%   16     0.0KB/s   00:00    
    vm1:/etc/openafs/server# chown -R backuppc4afs:backuppc4afs .
    vm1:/etc/openafs/server# chmod 700 KeyFile
    
  16. Add AFS volumeset(s)

    The OpenAFS documentation has more information on doing this, but the general idea is to create one or more AFS volumesets (a list of volumes to backed up together at the same time and according to the same dump schedule). To these volumesets you add one or more volentries specifying the volumes to backup. These volume entries are server/partition/volumename triads and each may be be exact or a regular expression. Since they may be regular expressions, any of the fields can also contain wildcards. Remember, all of the volumes in a volumeset will be dumped at the same time.

    Just as with the original, old-school AFS backups, a single volume may belong to more than one volumeset--accidentally or by clever design. Backing up the same volume in multiple volumesets will increase the amount of disk space required.

    Under normal circumstances, you want to backup the AFS .backup volumes rather than the readwrite volumes. To create a volumeset named "testvolumeset1" which will back up volumes on any servers, any partition, and with volumename matching the regular expression "user.<anything>.backup" issue the following commands. Notice that "." represents any single character, ".*" represents zero or more of any single character. Literal periods must be escaped with a backslash: "\.".

    vm1:/etc/openafs# backup -localauth
    backup> addvolset -name testvolumeset1
    backup> addvolentry -name testvolumeset1 -server .* -partition .* -volumes test\..*\.backup
    
    And to verify the result
    backup> listvolset -name testvolumeset1
    Volume set testvolumeset1:
        Entry   1: server .*, partition .*, volumes: test\..*\.backup
    backup> quit
    
  17. Verify AFS backup configuration

    Once configured correctly, the BackupPC4AFS "getvols.pl" script should yield a list of volumes when given a volumeset.

    vm1:/etc/openafs# /usr/lib/BackupPC4AFS/bin/getvols.pl testvolumeset1
    volumeset testvolumeset1
    backup: waiting for job termination
        Entry   1: server .*, partition .*, volumes: test\..*\.backup
    VOLUME:test.stephen.backup
    VOLUME:test.www.backup
    vm1:/etc/openafs# 
    

    The volumes listed above, test.stephen.backup and test.www.backup, are the volumes which will be dumped when a backup of the "host" afs_testvolumeset1 triggers.

  18. Set sensible backup defaults

    At a minimum, you'll want to configure the following settings. Suggested values are shown, but please read /etc/backuppc/config.pl or the BackupPC configuration documentation for full details. These settings can be changed via the web by a user previously added to CgiAdminUsers by clicking on "Edit Config", or by editing config.pl directly.

    Server | Wakeup Schedule | WakeupSchedule: 23
    Server | Concurrent Jobs | MaxBackups: 3 (preliminary tests on my server (dual 2.8GHz HT on gigE) reveal diminishing throughput returns at 4 simultaneous vos dumps. YMMV.)
    Schedule | Full Backups | FullPeriod: 20.67
    Schedule | Full Backups | FullKeepCnt: 3
    Schedule | Full Backups | FullKeepCntMin: 3
    Schedule | Full Backups | FullAgeMax: 90
    Schedule | Incremental Backups | IncrPeriod: 0.67
    Schedule | Incremental Backups | IncrKeepCnt: 42
    Schedule | Incremental Backups | IncrKeepCntMin: 42
    Schedule | Incremental Backups | IncrAgeMax: 90
    Schedule | Incremental Backups | IncrLevels: 3,2,5,4,7,6,9,8,1,3,2,5,4,7,6,9,8,1,3,2
    Xfer | Xfer Settings | XferMethod: vos
    

    Note that IncrLevels does not include the level 0 (full) dump. The levels list presented here is based on the Towers of Hanoi dump sequence. A full discussion of backup rotations is beyond the scope of this installation doc. For background on designing backup schedules, I recommend UNIX(R) System Administration Handbook by Nemeth, Snyder, SeeBass, and Hein. (it's old, but good.)

    It is CRITICAL that the XferMethod be set to "vos" either globally as shown above, or on a per-host basis for each afs volumeset if BackupPC4AFS is going to backup non-afs hosts too. Don't forget to Save your changes.

  19. Add the volumeset to BackupPC4AFS

    Add each volumeset to BackupPC4AFS as a host. The hostname for an afs volumeset is "afs_volumesetname"

    Hosts | Hosts | Add
     host: afs_testvolumeset1
     dhcp: no/unchecked
     user: backuppc4afs
     moreUsers: blank
    

    Repeat as necessary for additional volumesets. After changing any settings, you must restart the backuppc process by doing one of the following:

  20. Perform a test backup

    Test the setup by performing a backup. Click "Host Summary", "afs_testvolumeset1" (or whatever you called your host), ", then "Start Full Backup" (twice in a row).