HOWTO: installing rsnapshot - incrementa backup utility using rsync

[Aramaki]

Quoting freshmeat.net

rsnapshot is a filesystem snapshot utility based on rsync. It makes it easy to make periodic snapshots of local machines, and remote machines over ssh. It uses hard links whenever possible, to greatly reduce the disk space required.

Actual manual version 1.1 (version 1.0)

Russian version

[Aramaki]

create backup dir - for example

$ mkdir /tmp/harddisk/backups

edit /opt/etc/rsnapshot.conf

change
snapshot_root /tmp/harddisk/backups/

uncomment if you like
no_create_root 1

uncomment
cmd_cp /bin/cp

uncomment
cmd_ssh /usr/bin/ssh


uncomment
cmd_du /usr/bin/du



uncomment and change
cmd_rsnapshot_diff /opt/bin/rsnapshot-diff



change to your needs (optional). in this example to distribute backups evently through the day you should run it every 4 hours (6x4=24). If you run it hourly you'll have only last 6 hours saved.

interval hourly 6
interval daily 7
interval weekly 4
#interval monthly 3

uncomment and change
logfile /opt/var/log/rsnapshot.log

may be need to be changed (i didn't) - ask guru
lockfile /var/run/rsnapshot.pid

edit to your needs after (my example follows)

###############################
### BACKUP POINTS / SCRIPTS ###
###############################

backup /opt/ asus/
backup /usr/ asus/
backup /tmp/harddisk/exchange/ asus/


save and exit;


rsnapshot configtest

output must be
Syntax OK


test mode

[admin@(none) rsnapshot-1.3.0]$ rsnapshot -t hourly
echo 25893 > /var/run/rsnapshot.pid
mkdir -m 0755 -p /tmp/harddisk/backups/hourly.0/
/opt/bin/rsync -a --delete --numeric-ids --relative --delete-excluded /opt \
/tmp/harddisk/backups/hourly.0/opt/
mkdir -m 0755 -p /tmp/harddisk/backups/hourly.0/
/opt/bin/rsync -a --delete --numeric-ids --relative --delete-excluded /usr \
/tmp/harddisk/backups/hourly.0/usr/
mkdir -m 0755 -p /tmp/harddisk/backups/hourly.0/
/opt/bin/rsync -a --delete --numeric-ids --relative --delete-excluded \
/tmp/harddisk/exchange /tmp/harddisk/backups/hourly.0/exchange/
touch /tmp/harddisk/backups/hourly.0/


next you can edit crontab directly or use cron.hourly (or cron.daily, cron.weekly, cron.monthly) folders created as described at http://www.macsat.com/cron.php

0 */4 * * * /usr/local/bin/rsnapshot hourly #runs every 4 hours (for interval hourly 6: 6x4=24 hours)
30 23 * * * /usr/local/bin/rsnapshot daily #runs every day at 23.30

IMPORTANT (from http://www.rsnapshot.org/howto/1.2/r...he_config_file)

It is usually a good idea to schedule the larger intervals to run a bit before the lower ones. For example, in the crontab above, notice that daily runs 30 minutes before hourly. This helps prevent race conditions where the daily would try to run before the hourly job had finished. This same strategy should be extended so that a weekly entry would run before the daily and so on.

Our ASUS is not a professional power server so take this into consideration.


for testing I created script in cron.5min

[admin@(none) cron.5mins]$ touch /opt/etc/cron.5mins/rsnapshot.sh
[admin@(none) cron.5mins]$ chmod +x /opt/etc/cron.5mins/rsnapshot.sh

edit it with your favourite editor (I use mc)

content should be

#!/bin/sh
/opt/bin/rsnapshot hourly

save and wait for snapshots to appear after 5 mins

don't forget to move rsnapshot.sh to hourly dir

create files for daily and weekly backups

=============

about backup from http://www.rsnapshot.org/howto/1.2/r...he_config_file

4.3.8. backup
Please note that the destination paths specified here are based on the assumption that the --relative flag is being passed to rsync via the rsync_long_args parameter. If you are installing for the first time, this is the default setting. If you upgraded from a previous version, please read the INSTALL file that came with the source distribution for more information.

This is the section where you tell rsnapshot what files you actually want to back up. You put a “backup� parameter first, followed by the full path to the directory or network path you're backing up. The third column is the relative path you want to back up to inside the snapshot root. Let's look at an example:

backup /etc/ localhost/
In this example, backup tells us it's a backup point. /etc/ is the full path to the directory we want to take snapshots of, and localhost/ is a directory inside the snapshot_root we're going to put them in. Using the word localhost as the destination directory is just a convention. You might also choose to use the server's fully qualified domain name instead of localhost. If you are taking snapshots of several machines on one dedicated backup server, it's a good idea to use their various hostnames as directories to keep track of which files came from which server.

In addition to full paths on the local filesystem, you can also backup remote systems using rsync over ssh. If you have ssh installed and enabled (via the cmd_ssh parameter), you can specify a path like:

backup root@example.com:/etc/ example.com/
This behaves fundamentally the same way, but you must take a few extra things into account.

The ssh daemon must be running on example.com

You must have access to the account you specify the remote machine, in this case the root user on example.com.

You must have key-based logins enabled for the root user at example.com, without passphrases. If you wanted to perform backups as another user, you could specify the other user instead of root for the source (i.e. user@domain.com). Please note that allowing remote logins with no passphrase is a security risk that may or may not be acceptable in your situation. Make sure you guard access to the backup server very carefully! For more information on how to set this up, please consult the ssh man page, or a tutorial on using ssh public and private keys. You will find that the key based logins are better in many ways, not just for rsnapshot but for convenience and security in general. One thing you can do to mitigate the potential damage from a backup server breach is to create alternate users on the client machines with uid and gid set to 0, but with a more restrictive shell such as scponly.

This backup occurs over the network, so it may be slower. Since this uses rsync, this is most noticeable during the first backup. Depending on how much your data changes, subsequent backups should go much, much faster since rsync only sends the differences between files.

4.3.9. backup_script
With this parameter, the second column is the full path to an executable backup script, and the third column is the local path you want to store it in (just like with the "backup" parameter). For example:

backup_script /usr/local/bin/backup_pgsql.sh localhost/postgres/
In this example, rsnapshot will run the script /usr/local/bin/backup_pgsql.sh in a temp directory, then sync the results into the localhost/postgres/ directory under the snapshot root. You can find the backup_pgsql.sh example script in the utils/ directory of the source distribution. Feel free to modify it for your system.

Your backup script simply needs to dump out the contents of whatever it does into it's current working directory. It can create as many files and/or directories as necessary, but it should not put its files in any pre-determined path. The reason for this is that rsnapshot creates a temp directory, changes to that directory, runs the backup script, and then syncs the contents of the temp directory to the local path you specified in the third column. A typical backup script would be one that archives the contents of a database. It might look like this:

#!/bin/sh

/usr/bin/mysqldump -uroot mydatabase > mydatabase.sql
/bin/chmod 644 mydatabase.sql
There are several example scripts in the utils/ directory of the rsnapshot source distribution to give you more ideas.

Make sure the destination path you specify is unique. The backup script will completely overwrite anything in the destination path, so if you tried to specify the same destination twice, you would be left with only the files from the last script. Fortunately, rsnapshot will try to prevent you from doing this when it reads the config file.

Please remember that these backup scripts will be invoked as the user running rsnapshot. In our example, this is root. Make sure your backup scripts are owned by root, and not writable by anyone else. If you fail to do this, anyone with write access to these backup scripts will be able to put commands in them that will be run as the root user. If they are malicious, they could take over your server.




===========

useful links

howto with explanation of parameter and configs http://www.rsnapshot.org/howto/1.2/r...-HOWTO.en.html

[Aramaki]

For minimum installation you'll need perl (9.7M), rsync (222K), gawk (327K) and rsnapshot 1.3 (191K).
I recomment to use the following repository for ipkg

http://ipkg.nslu2-linux.org/feeds/optware/oleg/cross/stable

Run the following script in shell (can take time while perl is downloaded - about 5 mins at 2200kbps for everything)

ipkg update
ipkg install rsync
ipkg install gawk
ipkg install perl
mkdir /tmp/harddisk/rsnapshot
cd /tmp/harddisk/rsnapshot
wget http://www.rsnapshot.org/downloads/rsnapshot-1.3.0.tar.gz
tar -xzvf rsnapshot-1.3.0.tar.gz
cd rsnapshot-1.3.0
./configure --sysconfdir=/opt/etc --prefix=/opt --bindir=/opt/bin --mandir=/opt/man  --with-perl=/opt/bin/perl --with-rsync=/opt/bin/rsync
make test
make install
#cp /opt/etc/rsnapshot.conf.default /opt/etc/rsnapshot.conf
mkdir /tmp/harddisk/backups

Open /opt/etc/rsnapshot.conf with your favourite editor

Important!!!
1. Between variable_name and variable_value must be a tabulation, not spaces
2. Path to dir must end with slash "/"

Right: /tmp/harddisk/opt/
Wrong: /tmp/harddisk/opt

Change standard values with the following

ñòàðàÿ ñòðîêà > íîâàÿ ñòðîêà
snapshot_root /.snapshots/ > snapshot_root /tmp/harddisk/backups/
#cmd_ssh /usr/bin/ssh > cmd_ssh /usr/bin/ssh
cmd_logger /usr/bin/logger > #cmd_logger /usr/bin/logger
#cmd_du /usr/bin/du > cmd_du /usr/bin/du
#cmd_rsnapshot_diff /usr/local/bin/rsnapshot-diff > cmd_rsnapshot_diff /opt/bin/rsnapshot-diff
interval hourly 6 > interval hourly 24

#logfile /var/log/rsnapshot > logfile /opt/var/log/rsnapshot.log
#du_args -csh > du_args -csH

In this section set up rules for exclusions and backups (my example)

exclude *.tmp
exclude *.avi
exclude /~*
exclude /private/*

###############################
### BACKUP POINTS / SCRIPTS ###
###############################
backup /opt/bin/ asus/
backup /usr/ asus/

Save changes and exit.

Run

rsnapshot configtest

Answer should be

Syntax OK

Run program in test mode

rsnapshot -t hourly

Answer should be

echo 25893 > /var/run/rsnapshot.pid
mkdir -m 0755 -p /tmp/harddisk/backups/hourly.0/
/opt/bin/rsync -a --delete --numeric-ids --relative --delete-excluded /opt \
/tmp/harddisk/backups/hourly.0/opt/
mkdir -m 0755 -p /tmp/harddisk/backups/hourly.0/
/opt/bin/rsync -a --delete --numeric-ids --relative --delete-excluded /usr \
/tmp/harddisk/backups/hourly.0/usr/
mkdir -m 0755 -p /tmp/harddisk/backups/hourly.0/
/opt/bin/rsync -a --delete --numeric-ids --relative --delete-excluded \
/tmp/harddisk/exchange /tmp/harddisk/backups/hourly.0/exchange/
touch /tmp/harddisk/backups/hourly.0/

Create a test script, run it (it takes a while) and look what you get

echo "#!/bin/sh" >> /opt/tmp/test.sh
echo "rsnapshot hourly" >> /opt/tmp/test.sh
chmod +x /opt/tmp/test.sh
/opt/tmp/test.sh
ls -l /tmp/harddisk/backups

In /tmp/harddisk/backups should appear folder hourly.0

drwxr-xr-x 3 admin root 4096 Oct 27 19:11 .
drwxr-xr-x 11 admin root 4096 Oct 27 18:07 ..
drwxr-xr-x 3 admin root 4096 Oct 27 19:12 hourly.0

Now take a look what was logged

cat /opt/var/log/rsnapshot.log

Answer should be

[27/Oct/2007:19:11:53] /opt/bin/rsnapshot hourly: started
[27/Oct/2007:19:11:53] echo 30847 > /var/run/rsnapshot.pid
[27/Oct/2007:19:11:53] mkdir -m 0755 -p /tmp/harddisk/backups/hourly.0/
[27/Oct/2007:19:11:53] /opt/bin/rsync -a --delete --numeric-ids --relative --delete-excluded /opt/bin /tmp/harddisk/backups/hourly.0/asus/
[27/Oct/2007:19:12:22] /opt/bin/rsync -a --delete --numeric-ids --relative --delete-excluded /usr /tmp/harddisk/backups/hourly.0/asus/
[27/Oct/2007:19:12:31] touch /tmp/harddisk/backups/hourly.0/
[27/Oct/2007:19:12:31] rm -f /var/run/rsnapshot.pid
[27/Oct/2007:19:12:31] /opt/bin/rsnapshot hourly: completed successfully

If folders already existed answer would be

[27/Oct/2007:19:06:08] /opt/bin/rsnapshot hourly: started
[27/Oct/2007:19:06:08] echo 30726 > /var/run/rsnapshot.pid
[27/Oct/2007:19:06:08] mv /tmp/harddisk/backups/hourly.4/ /tmp/harddisk/backups/hourly.5/
[27/Oct/2007:19:06:08] mv /tmp/harddisk/backups/hourly.3/ /tmp/harddisk/backups/hourly.4/
[27/Oct/2007:19:06:08] mv /tmp/harddisk/backups/hourly.2/ /tmp/harddisk/backups/hourly.3/
[27/Oct/2007:19:06:08] mv /tmp/harddisk/backups/hourly.1/ /tmp/harddisk/backups/hourly.2/
[27/Oct/2007:19:06:08] native_cp_al("/tmp/harddisk/backups/hourly.0", "/tmp/harddisk/backups/hourly.1")
[27/Oct/2007:19:06:10] /opt/bin/rsync -a --delete --numeric-ids /tmp/harddisk/backups/hourly.0/ /tmp/harddisk/backups/hourly.1/
[27/Oct/2007:19:06:10] /opt/bin/rsync -a --delete --numeric-ids --relative --delete-excluded /usr /tmp/harddisk/backups/hourly.0/asus/
[27/Oct/2007:19:06:10] /opt/bin/rsync -a --delete --numeric-ids --relative --delete-excluded /tmp/harddisk/exchange /tmp/harddisk/backups/hourly.0/asus/
[27/Oct/2007:19:06:11] touch /tmp/harddisk/backups/hourly.0/
[27/Oct/2007:19:06:11] rm -f /var/run/rsnapshot.pid
[27/Oct/2007:19:06:11] /opt/bin/rsnapshot hourly: completed successfully

Now create cron rules to automate the process. Open /opt/etc/crontab and add lines

0 * * * * admin /opt/bin/rsnapshot hourly 
30 23 * * * admin /opt/bin/rsnapshot daily

It is usually a good idea to schedule the larger intervals to run a bit before the lower ones. For example, in the crontab above, notice that daily runs 30 minutes before hourly. This helps prevent race conditions where the daily would try to run before the hourly job had finished. This same strategy should be extended so that a weekly entry would run before the daily and so on.

Read more at HOWTO rsnapshot (íà àíãëèéñêîì))

If you configured cron as described in manual by macsat and you have folders cron.hourly, cron.daily etc just put scripts with appropriate commands to folders.
Example:

echo "#!/bin/sh" >> /opt/etc/cron.hourly/rsnapshot_hourly.sh
echo "rsnapshot hourly" >> /opt/etc/cron.hourly/rsnapshot_hourly.sh
chmod +x /opt/etc/cron.hourly/rsnapshot_hourly.sh

To find out how much space backups consume use built in command beacuse simple calculationg folders sizes gives wrong result because of symlinks.

rsnapshot du

Result will be like this

204652 /tmp/harddisk/backups/hourly.0
3040 /tmp/harddisk/backups/hourly.1
207692 total

Actual copy takes 204652 kb and previous versions take 3040 kb for deleted files.


Useful links

• Actual repository
• rsnapshot HOWTO