rsync is a tool used for synchronize files or directories minimizing the data transfer, due to its algorithm based in the difference of the files and the data compression. rsync allow us work in a local machine and with remote servers using ssh server or using rsync as a server listening the port number 873/tcp.
This tutorial is a brief introduction to the use of rsync and the main options and uses.
Basic use of rsync
The basic way of use rsync is with the -av options:
-a (archive mode) : same options as -rlptgoD (-r recursively, -l copy the symbolic links as symbolic links, -p keep the permissions, -t keep the modify time, -g keep the group, -o keep the owner, -D keep the devices files)
[email protected]:~# ls dir1 dir2 [email protected]:~# rsync -av dir1/ dir2/ sending incremental file list ./ bar fic1 foo dira/ dirb/
sent 239 bytes received 80 bytes 638.00 bytes/sec total size is 0 speedup is 0.00
If one file is changed rsync by default use the option -W for local copy. This causes the entire file is copied:
[email protected]:~# echo "one small step for a man, one giant leap for mankind" >> dir1/foo [email protected]:~# rsync -av dir1/ dir2/ sending incremental file list ./ foo
<strong>sent 4871 bytes</strong> received 36 bytes 9814.00 bytes/sec total size is 4712 speedup is 0.96
To avoid this behavior we can use the option –no-whole-file:
[email protected]:~# rsync -av --no-whole-file dir1/ dir2/ sending incremental file list ./ foo
<strong>sent 695 bytes</strong> received 78 bytes 1546.00 bytes/sec total size is 4712 speedup is 6.10
With rsync we can make incremental backups saving the changed files in a different directory with the -b option:
[email protected]:~# rsync -avvb --backup-dir=~/BackupDir_$(date +%H%M%m%d%y) dir1/ dir2/ backup_dir is ~/BackupDir_2015082612/ sending incremental file list delta-transmission disabled for local transfer or --whole-file ./ bar foo is uptodate <strong>backed up bar to ~/BackupDir_2015082612/bar</strong> total: matches=0 hash_hits=0 false_alarms=0 data=21
sent 166 bytes received 45 bytes 422.00 bytes/sec total size is 4733 speedup is 22.43
Including and excluding files for the backup
We can include and exclude by the file or directory name with the options –include or –exclude:
[email protected]:~# rsync -av --exclude='bar' dir1/ dir2/ sending incremental file list ./
sent 81 bytes received 17 bytes 196.00 bytes/sec total size is 4712 speedup is 48.08
[email protected]:~# rsync -av --include='bar' dir1/ dir2/ sending incremental file list bar sent 154 bytes received 33 bytes 374.00 bytes/sec total size is 4733 speedup is 25.31
Also we can create filters in a text file for rsync for include or exclude directories:
[email protected]:~# cat dir.txt + */ + /var/log/** + /usr/share/doc/procmail/** - *
- + : match.
- – : discard.
- */ : is necessary for match the own directories (/var, /var/log and /usr/, /usr/share/ …) .
- – * :determine all that doesn’t match with the pattern will exclude for the backup.
- ** : all files and directories will match with the pattern.
[email protected]:~# rsync -av --prune-empty-dirs --include-from=dir.txt / backup/ building file list ... done (...) var/log/news/ var/log/news/news.crit var/log/news/news.err var/log/news/news.notice
sent 1702534 bytes received 1148 bytes 3407364.00 bytes/sec total size is 1612678 speedup is 0.95
The –prune-empty-dirs option is used for don’t copy the empty directories.
Other way without –prune-empty-dirs option specifying all the directories:
[email protected]:~# cat dir.txt + /var/ + /var/log/ + /usr/ + /usr/share/ + /usr/share/doc/ + /usr/share/doc/procmail/ + /var/log/** + /usr/share/doc/procmail/** - *
This are two valid patterns, the first is more simple than the second, but It will be more slow for search the directories recursively.
We can use rsync for remote backups using ssh. For do this more easy, I’ll copy my key to the other server for connect with the server without using a password. First we have to generate a rsa key pair in our servers:
# ssh-keygen -t rsa
Next copy our public key to the destination backup server as authorized key:
# scp /root/.ssh/id_rsa.pub [email protected]:/root/.ssh/authorized_keys
Now we can make a backup script to syncronize two directories:
[email protected]:~# cat syncbackup.sh
export source=/root/backup/ export destination=192.168.0.110:/backup export logfile=/var/log/syncbackup.log
echo -e "n** Begin at $(date +%H:%M:%S_%m/%d/%y) **" >> $logfile nice -n19 ionice -n2 rsync -avz --delete --no-whole-file $source [email protected]$destination 2>&1 >> $logfile echo -e "n** Finish at $(date +%H:%M:%S_%m/%d/%y) **" >> $logfile
nice and ionice command are used for the best performance for the server and the backup don’t interrupt the work of the server. nice specify the priority for the process between the range -20 (High priority) and 19 (low priority). ionice is used to determine the priority that one process access to the disk.[/sourcecode]
And execute every hour with cron:
# crontab -e
# m h dom mon dow command 0 * * * * /root/syncbackup.sh