Backup scripts for DAR

Nowadays you can buy small network attached boxes to function as small home-office disk servers so cheaply, that most of the time I take backups by having a couple of those lying around the house. But for many years I used to backup my personal Linux desktop by burning CD's and then DVD's.

The challenge with backing up to a CD/DVD is that you easily have more data to back up than fits on one disc. The TAR utility was created for tapes and completely lacks any capability of splitting itself into parts. What's more, with TAR you create the archive first, and then compress it, so it is not possible to know in the TAR phase when you've actually reached the size of your CD or DVD, and in the compression phase it is generally too late.

You could do hacks like using the split command line utility, but that seems both unreliable and is also inconvenient: To restore you would first need to append together all the pieces, and for a large backup that means you'd need a lot of empty space somewhere.

It is surprising how little attention it has gotten, but I've been very happy to use DAR (Disk ARchiver) for these backup tasks.

DAR is like TAR but of CD/DVD's. It creates an archive that is split into slices of the size you choose. If you are low on disk space you can save them one by one, and also when restoring you only need the slice(s) that contains your file(s). It works kind of opposite to what tar|gzip does. It first compresses each individual file, then combines them into one archive. This works better for this use case.

I've come to run the following scripts from my cron, on a monthly and daily basis. I still use them to make backups to a network disk. It makes me feel good when I know that if needed, I could still easily burn my backups to CD's or DVD's.

(This was posted simply so I can copy paste the scripts from here as I need them.)

$ cat fullbackup.sh 
#!/bin/sh
# Create full DAR backup archive onto network drive
# Henrik Ingo, henrik.ingo [at] avoinelama.fi, Feb 2008
#
# License: GPLv2

#### SETTINGS ####
#
#
DARRC="/usr/local/etc/backup.darrc"
DISKSERVER=diskserver
HOSTNAME=`hostname`

# Directory for backups
BACKUPDIR="/tmp"
BACKUPDIRFINAL="/media/$DISKSERVER/backup/$HOSTNAME"

# Filename (body) for archive files
# Chosen logic: Do full backup monthly
TIMESTAMP="`date +%Y-%m`"
FILENAME="`hostname`-$TIMESTAMP-backup-full"

#
#### SETTINGS END ####

# Do not do anything unless the specified directory is available 
# (ie we are on the road and it's not mounted)
if test ! -d "$BACKUPDIRFINAL"
then
  echo "Backup directory $BACKUPDIRFINAL not available (not mounted, no network)."
  exit 1
fi

if test ! -w "$BACKUPDIRFINAL"
then
  echo "Backup directory $BACKUPDIRFINAL not writable."
  exit 2
fi

#create directory for this months backups
BACKUPDIRMONTH="$BACKUPDIR/$TIMESTAMP"
if test ! -d "$BACKUPDIRMONTH"
then
  mkdir "$BACKUPDIRMONTH"
elif test ! -w "$BACKUPDIRMONTH"
then
  echo "$BACKUPDIRMONTH exists but is not writable."
  exit 3
fi

#do dar backup
BACKUPPATHFILE="$BACKUPDIRMONTH/$FILENAME"
dar --create "$BACKUPPATHFILE" -B "$DARRC" -v

echo "Testing integrity of created archive."

dar -t "$BACKUPPATHFILE"


if test $BACKUPDIR eq $BACKUPDIRFINAL
then
  # Already wrote files to final dest, nothing left to do
  echo Backup complete.
  exit 0
fi

BACKUPDIRFINALMONTH="$BACKUPDIRFINAL/$TIMESTAMP"
#create directory for this months backups
if test ! -d "$BACKUPDIRFINALMONTH"
then
  mkdir "$BACKUPDIRFINALMONTH"
elif test ! -w "$BACKUPDIRFINALMONTH"
then
  echo "$BACKUPDIRFINALMONTH exists but is not writable."
  exit 3
fi

echo
echo "Now copying backups $BACKUPPATHFILE* to $BACKUPDIRFINALMONTH."
echo
echo
cp "$BACKUPPATHFILE"* "$BACKUPDIRFINALMONTH"


echo end of $0 reached




$ cat diffbackup.sh
#!/bin/sh
# Create differential DAR backup archive onto network drive
# Depends on a full backup archive to exist, see fullbackup.sh
# Henrik Ingo, henrik.ingo [at] avoinelama.fi, Feb 2008
#
# License: GPLv2


#debugging...
echo "$0 being run by `whoami` at `date`" >> /var/log/diffbackup

#### SETTINGS ####
#
#
DARRC="/usr/local/etc/backup.darrc"
DISKSERVER=diskserver
HOSTNAME=`hostname`

# Directory for backups
BACKUPDIR="/media/$DISKSERVER/backup/$HOSTNAME"

# Script that creates the full backup if needed
FULLBACKUPSH="fullbackup.sh"

# Chosen logic: Do full backup monthly, maintain diff backup each day for a week
# (eg, next monday we'll overwrite this mondays backup)
TIMESTAMP="`date +%Y-%m`"
WDAY="`date +%A`"
DAY="`date +%d`"
FILENAMEFULL="`hostname`-$TIMESTAMP-backup-full"
FILENAMEDIFF="`hostname`-$TIMESTAMP-$DAY-backup-diff"

#
#### SETTINGS END ####


# Do not do anything unless the specified directory is available 
# (ie we are on the road and it's not mounted)
if test ! -d "$BACKUPDIR"
then
  echo "Backup directory not available (not mounted, no network)."
  exit 1
fi

if test ! -w "$BACKUPDIR"
then
  echo "Backup directory not writable."
  exit 2
fi

# directory for this months backups
BACKUPDIRMONTH="$BACKUPDIR/$TIMESTAMP"
# and for this weekday
BACKUPDIRWDAY="$BACKUPDIRMONTH/$WDAY"
# and the dar files (diff to create, full to read)
BACKUPPATHFILEDIFF="$BACKUPDIRWDAY/$FILENAMEDIFF"
BACKUPPATHFILEFULL="$BACKUPDIRMONTH/$FILENAMEFULL"

# full reference backup must exist. If not, create it.
if test ! -f "$BACKUPPATHFILEFULL.1.dar"
then
  echo "Full reference backup (for this month) doesn't exist. Creating it first (this will take time)."
  echo Running $FULLBACKUPSH
  echo
  $FULLBACKUPSH
  echo
  echo Now returning to $0
fi


if test ! -d "$BACKUPDIRWDAY"
then
  mkdir "$BACKUPDIRWDAY"
elif test ! -w "$BACKUPDIRWDAY"
then
  echo "$BACKUPDIRWDAY exists but is not writable."
  exit 3
else
  # WDAY directory exists, so remove old backups
#  echo removing old backups from "$BACKUPDIRWDAY"
#  rm $BACKUPDIRWDAY/*
  echo
fi


echo removing old backups from "$BACKUPDIRWDAY"
rm $BACKUPDIRWDAY/*


#do dar backup
dar --create "$BACKUPPATHFILEDIFF" --ref "$BACKUPPATHFILEFULL" -B "$DARRC" -v

echo
echo "Testing integrity of created archive."
echo 

dar -t "$BACKUPPATHFILEDIFF"


echo end of $0 reached
#debugging...
echo "end of $0 reached at `date`" >> /var/log/diffbackup



$ cat /usr/local/etc/backup.darrc
#Preface
#--------
#
#Here follows a sample batch file submited by Henrik Ingo (Thanks Henrik ;-) ).
#It is complete for backup but does not use conditional syntax. Except comments
#(lines staring with #) all commands can also take place on the command-line.
#Thus, this is a nice way to discover DAR's features.
#
#                                                            Denis Corbin
###########################################################################

# I used to run this simply as 'dar -B thisfile'. I now use the fullbackup.sh
# and diffbackup.sh scripts to do a cycle of incremental backups too. I have
# therefore commented out things that are done from those scripts instead.
#
#  -- henrik


#Execution file for dar (Disc Archiver)
#Simply use 'dar -B thisfile' to backup
#This backs up my home machine

#Where to place the backup (somewhere with lots of space)
#--create /home/hingo/nobackup/darbackups/backup_May2008


#General settings

#size of an archive (one slice). 650M fits nicely on CD-R (and RW?)
-s 699M
#compress using bzip
-y
#verbose
-v

#Files not to compress
-Z "*.mp3" -Z "*.avi" -Z "*.mpg" -Z "*.mpeg"
-Z "*.divx" -Z "*.rm" -Z "*.wmv" -Z "*.wma"
-Z "*.asf" -Z "*.ra"

-Z "*.gif" -Z "*.jpg" -Z "*.jpeg" -Z "*.png"

-Z "*.zip" -Z "*.tgz" -Z "*.gzip" -Z "*.bzip"
-Z "*.bzip2" -Z "*.rar" -Z "*.Z"

#Define directories to be backed up
#First give a root
--fs-root /

#Then list directories to back up (relative to fs-root)
#prepended with option -g
#Exclude directories/files with the --prune option
#--prune not/this

-g etc
-g var/lib
-g var/backups
-g var/games
-g var/local
-g var/www
-g var/ftp
-g usr/local

-g root
--prune root/RPMS
--prune root/tmp
--prune root/kino
--prune root/Desktop/Trash
--prune root/.Trash

-g home/hingo
--prune home/hingo/tmp
--prune home/hingo/.local/share/Trash/
--prune home/hingo/nobackup
--prune home/hingo/downloads
--prune home/hingo/mnt


#End of file
#Use something like this to restore everything:
# dar -x /home/hingo/nobackup/darbackups/backup_May2008 -R /
#something like this to restore something (etc-subtree):
# dar -x /home/hingo/nobackup/darbackups/backup_May2008 -R / etc
#And something like this to retrieve a single file to temp
# dar -x /home/hingo/nobackup/darbackups/backup_May2008 -R /tmp/ etc/httpd/conf/httpd2.conf --flat




$ cat /etc/cron.daily/darbackup 
#!/bin/sh

/usr/local/bin/diffbackup.sh

 

About the bookAbout this siteAcademicAccordAmazonAppleBeginnersBooksBuildBotBusiness modelsbzrCassandraCloudcloud computingclsCommunitycommunityleadershipsummitConsistencycoodiaryCopyrightCreative CommonscssDatabasesdataminingDatastaxDevOpsDistributed ConsensusDrizzleDrupalEconomyelectronEthicsEurovisionFacebookFrosconFunnyGaleraGISgithubGnomeGovernanceHandlerSocketHigh AvailabilityimpressionistimpressjsInkscapeInternetJavaScriptjsonKDEKubuntuLicensingLinuxMaidanMaker cultureMariaDBmarkdownMEAN stackMepSQLMicrosoftMobileMongoDBMontyProgramMusicMySQLMySQL ClusterNerdsNodeNoSQLNyrkiöodbaOpen ContentOpen SourceOpenSQLCampOracleOSConPAMPParkinsonPatentsPerconaperformancePersonalPhilosophyPHPPiratesPlanetDrupalPoliticsPostgreSQLPresalespresentationsPress releasesProgrammingRed HatReplicationSeveralninesSillySkySQLSolonStartupsSunSybaseSymbiansysbenchtalksTechnicalTechnologyThe making ofTransactionsTungstenTwitterUbuntuvolcanoWeb2.0WikipediaWork from HomexmlYouTube