Register a free account to unlock additional features at BleepingComputer.com
Welcome to BleepingComputer, a free community where people like yourself come together to discuss and learn how to use their computers. Using the site is easy and fun. As a guest, you can browse and view the various discussions in the forums, but can not create a new topic or reply to an existing one unless you are logged in. Other benefits of registering an account are subscribing to topics and forums, creating a blog, and having no ads shown anywhere on the site.


Click here to Register a free account now! or read our Welcome Guide to learn how to use this site.

Generic User Avatar
- - - - -

A Guide to Backing Up Your Home Directory Using rsync.


  • Please log in to reply
1 reply to this topic

#1 Naught McNoone

Naught McNoone

  •  Avatar image
  • Members
  • 804 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:The Great White North
  • Local time:12:02 AM

Posted 06 April 2021 - 03:51 PM

A Guide to Backing Up Your Home Directory Using rsync.



Guide Overview

The purpose of this guide is to show you how the rsync command can be used to backup your /home directory.

 

This guide is aimed at inexperienced users.

This guide is based on material found the official rsync home page:

 

https://rsync.samba.org/

 

References to other commands can be found on your systems' individual man pages:

 

man commandname

 

Introduction

 

The rsync command is typically used to transfer files through a network.  However, when used in local mode it becomes a very powerful backup utility.

By default, all of your personal files and your program configuration files are in your "/home" directory.  Each user has their own separate sub directory.  They will appear as "/home/username1", "/home/username2", &c.

In most cases, recovering from a hard drive failure would involve reinstalling Linux on a new drive, recreating the user(s) name(s), and then restoring the "/home" directory from the backup.

 

Some Linux backup programs are based on rsync such as Time Vault, Flyback, and Grsync

In this guide we will look at the rsync command and it's practical use.

All commands are done in a Linux Terminal.

To enter Linux Terminal, from your desktop, press the following key combination simultaneously:

 

 <ctl>+<alt>+t

 

To exit the Linux Terminal, simply enter the command exit.


Tools Needed

 

A working Linux system.
A backup media device.

 

For this the purpose of this guide, I am using 3.5" USB 3 SATA enclosure with a separate 12 volt power supply.  It has a 2TB drive.  My notebook has Xubuntu 20.04 installed on a 1TB drive.



#######################################

 

Part :step1:    Using the basic rsync command:

If you are using Ubuntu, or one of its' derivatives, then rsync is installed by default.

To verify that you have rsync installed, simply enter the command “rsync” in the terminal, with no added switches or strings.

If rsync is installed, it will print it’s own help file to your screen.

If rsync is not installed, then you can use the following terminal commands to install it:

 

Debian or Ubuntu Based            sudo apt install rsync
Red Hat, Fedora and CentOS    yum install rsync


This is the rsync command in it's simplest form:

 

rsync -av /home/username /destination
 

Where:
rsync                        =     basic command
-a                             =     archive switch
-v                             =     verbose switch
/home/username     =     source directory
/destination             =     target directory


The two basic switches are -a and -v.

The -a switch tells rsync to use archive mode.  That is, rsync will preserve the original files names, but also ensures  that  symbolic  links,  devices,  attributes,  permissions, ownership, etc. are preserved in the transfer.

The -v switch tells rsync to be verbose.  That is, it will display what it is doing on the screen.  Otherwise you would just see a blank line on the screen, until the job was finished.

The directory /home/username, is the source. It can be an entire directory, or just a single file name.

The directory /destination, is the target where the source will be copied to.  

It is important to note that there is a space between the source and the target.
In the above example, the entire "username" directory will be copied into the "destination" directory.

###############################################################

What happens when we alter the source slightly.  What if we use the command:

    rsync -av /home/username/ /destination

Note the trailing slash after username.

By adding the trailing slash after username, we a telling rsync not to copy the directory of username, but rather only the contents of the directory.
 
The result would be that /destination/username would not be created, but rather all the files and sub directories would appear in /destination.

###############################################################

If we use the command this way:

    rsync -av /home/username/ /destination/some_other_name

The rsync command will create the /destination/some_other_name directory and transfer the contents of the /home/username directory to it.  

You cannot multi tier the target.  If you were to use /destination/target/some_other_name/ then the directory /destination/target must already exist.


############################################################

 

Part :step2:        Practical use of rsync and more examples of how it works.


The program first examines the source and target, to see what is there.  It will find out what files are already on the target, and what the difference, if any, there is between them and the same file on the source.

It will then transfer only the changed parts of any files that are already in the source, and the complete file for any new files.

Once the initial backup is done, then the incremental backups run very quickly, as only the changed data is sent to the source.

The time required to do a backup is dependent on the speed of your system, the amount of data, and the speed of the connection to the backup device.

I ran the following write speed test on my enclosure.
 

dd if=/dev/zero of=./speedTestFile bs=20M count=5 oflag=direct
5+0 records in
5+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 0.908418 s, 115 MB/s


I estimated the initial back up of my home directory, 372GB of data, to take about 55 minutes.
It actually took just over 58 minutes.
Subsequent backups take less than a minute.  Somewhat longer, if I have downloaded new iso images.

###############################################################

Removable Storage Devices

Removable storage devices should automatically mount it in the /media directory, under the users name.

The mounted device may appear as something like this:

    /media/username/0cxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/

Sometimes there will be a string of numbers and letters appearing as the sub directory in the users media mount point.

That alpha numeric is the UUID number of the partition on the removable drive.  
This typically occurs because when the media was formatted, it was not given a volume name.  

In order to identify it to the user, Linux uses the UUID number of the partition.

To change this, and provide a more human readable volume name, you can use gparted or gnome-disks.
 
They both give you the ability to add or change a volume name.  You can also use the command line.

Here are the commands to change partition labels:

 

e2lable /dev/sdxn
ntfslable /dev/sdxn
mlable /dev/sdxn


e2lable         =     ext2\3\4 native Linux file systems
ntfslable       =     Windows native ntfs file systems
mlable          =     DOS fat file systems
/dev/sdxn     =     The device, physical drive letter, and partition number

 

To obtain the physical drive letter and partition number of a removable drive, use the lsblk command, with the -f switch.

 

lsblk -f

 

This will print all of your block devices to your screen, showing their file system information.

###############################################################

A typical rsync command for removable media would be:

 

rsync -av /home/username /media/username/backup_device/some_sub_directory

 

I have my backup device labelled BUD, with a directory called Backups.  My command looks like this:

 

rsync -av /home /media/naught/BUD/Backups

 

This backs up all users to my external drive into the Backups directory.

A total system backup can be done with this command (All one line!):


sudo rsync -aAXv --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*",

"/media/*","/lost+found"} /* /media/naught/BUD/Backups


This would include virtually everything in the / Root partition.  

The -A and -X switches preserve ownership and extended file attributes.

Notice that I used the --exclude= switch to omit certain directories.  

Note the way they are listed between the { } parenthesizes.   
Full path, enclosed in “ “ double quotation marks, with the /* trailing slash and wild card symbol, and separated by commas.

They are not essential to a system recovery.  They would be recreated by the new system install.
As well as that, I omit my mount points, otherwise my server drive would be included.  It has its own back up routine.  
Also, if I tried to backup /media to /media/naught/BUD, then I would have created a loop.

For more advanced information on these, and other switches, see the rsync man page.
Use this command:

    man rsync

Note:  Use the letter q to exit from any man page.

###############################################################

To back up to an NAS server on your home network, it must be mounted.

Although you can view and access a local NAS server using a file manager like Thunar, they normally don't mount the drive permanently.

To do that, you have to use the mount command, as super user.

The mount command would be similar to this:

 

sudo mount //some_networkshare/shared_name /mnt/a_mount_point

 

First, make the mount point in your file system.  For example:

 

sudo mkdir /mnt/nas
sudo chown username:username -r /mnt/nas

 

Then mount the share.  For example:

 

sudo mount //192.168.nnn.nnn/Share_Name /mnt/nas

 

The network share now appears as a regular directory in your file system.

The rsync command can be used to backup from or to that share.

When used in a network, you should add the -z switch to the rsync command.
It would look like this:

 

rsync -avz /home/username /mnt/nas/some_sub_directory

 

The -z switch tells rsync to use data compression when sending the files through the network.  It does not actually compress the files at the other end.  It compresses the data packets.
It is simply there to increase the speed of the transfer.

One of the best things about using rsync in conjunction with an NAS is that you can set it up as a cron job.
You can have it run every day, at ZeroDark30, to do a backup of your system.

For more advanced information, see the man pages for those commands.

 

man mkdir
man chown
man mount
man crontab

 

I suggest you read the documentation first, and encourage you to try it on your own.  If you need help, post a question in the regular Linux Users forum.

In conclusion, if you want to take more personal control of your system, then you may want to start with rsync.

With older computers, using the command line in terminal reduces the overhead put on by GUI programs.

The more things you do from the terminal, the more you will realize how powerful Linux is as an operating system.

Cheers!

Naught :busy:

 



BC AdBot (Login to Remove)

 


#2 Naught McNoone

Naught McNoone
  • Topic Starter

  •  Avatar image
  • Members
  • 804 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:The Great White North
  • Local time:12:02 AM

Posted 07 April 2021 - 08:52 AM

Addendum to Original Guide

 

Mounting an NAS server in your local network

 

To mount a local network drive you must first install the smbclient.

 

User the following terminal commands in Debian/Ubuntu based systems:

 

Step :step1:

Install smbclient

 

sudo apt update

sudo apt install smbclient

smbclient -L //nnn.nnn.nnn.nnn

 

Where //nnn.nnn.nnn.nnn is the IP address of the NAS server

 

Step :step2:

Create the mount point

 

sudo mkdir /mnt/share_name

sudo chown username:username -R /mnt/share_name

 

Where share_name is the share listed by the NAS from smbclient -L

chown username:username -R  changes the ownership of the mount point recursively. 

Use your user name as it appears on your command line prompt.

 

NOTE:  You do not have to use the share actual server name.  You can use any label you wish for the mount point.

 

Step :step3:

Mount the drive

 

sudo mount -t cifs -o username=nas_user,password=nas_password //nnn.nnn.nnn.nnn/share_name /mnt/share_name

 

Where -t cifs tells the mount command to use the Common Internet File System.

-o is the options switch

username=nas_user is the username that the NAS requires for access

password=nas_password is the password the NAS requires for access

NOTE:  both strings must be joined with a "," comma, and have no spaces between them.

 

 

Cheers!

 

Naught :busy:

 

 


Edited by Naught McNoone, 07 April 2021 - 08:54 AM.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users