Author Topic: How To - Fsarchiver and the Root File System  (Read 7260 times)

VastOne

  •      v-ger
  • Posts: 3861
    • View Profile
    • VSIDO Community
How To - Fsarchiver and the Root File System
« on: March 01, 2013, 04:36:58 AM »
fsarchiver (File System archiver) IMO is the absolute best for backing up and restoring not only the File System, but anything that you have on a partition

In this How To I am going to explain how I use it to backup the VSIDO build partition and use it on multiple machines

One of the greatest things about fsarchiver is that it does not care how big the partition is that you restore to, as long as it is big enough to handle the size of the original file system.  My build FS is right at the 5 gig mark and fsarchiver compresses that down to 1 gig... So you can put your FS on a thumb drive and take it anywhere and restore it.  Since my build process is pretty much hardware agnostic (except that it is 64 bit), you should be able to download the link I will provide and restore exactly the build machine / FS that I use to build VSIDO

Here we go.  fsarchiver a part of VSIDO already.  In this How To, I am going to describe exactly the steps I used on my machine

Backup the file system with fsarchiver

I am booted to sda1

Make sure that sdc2 is not mounted

1: Create the following directory in /mnt

Code: [Select]
sudo mkdir /mnt/iso
2: Run the savefs portion of fsarchiver

single-dual core PC with 1GB or less RAM,
Code: [Select]
sudo fsarchiver -z7 savefs /mnt/iso/buildos.fsa /dev/sdc2
single-dual core PC with more than 1GB of RAM,
Code: [Select]
sudo fsarchiver -z8 savefs /mnt/iso/buildos.fsa /dev/sdc2
quod core PC with 4GB or less RAM,
Code: [Select]
sudo fsarchiver -j3 -z8 savefs /mnt/iso/buildos.fsa /dev/sdc2
quod core PC with more than 4GB of RAM,
Code: [Select]
sudo fsarchiver -j3 -z9 savefs /mnt/iso/buildos.fsa /dev/sdc2
PC with more than four cores and a plenty of RAM,
Code: [Select]
sudo fsarchiver -j4 -z9 savefs /mnt/iso/buildos.fsa /dev/sdc2
buildos.fsa is the file name being built, saved to /mnt/iso.  You can change this name to anything you want as long as it is with the .fsa extension

The save will take 1-3 minutes and when done you will see something like this:

Code: [Select]
vastone@vsido:~$ sudo fsarchiver savefs /mnt/iso/buildos.fsa /dev/sdc2
Statistics for filesystem 0
* files successfully processed:....regfiles=92361, directories=10723, symlinks=13608, hardlinks=21, specials=44
* files with errors:...............regfiles=0, directories=0, symlinks=0, hardlinks=0, specials=0

You can now take that .fsa file anywhere you need to for restore

Restoring a fsarchiver fsa file

I am booted to sda1 and I want to restore /mnt/iso/buildos.fsa to sdc8 and it cannot be mounted, if it is, unmount it

Step 1: cd to where the buildos.fsa is

Code: [Select]
cd /mnt/iso
Step 2: Run the restore process with this command

Code: [Select]
sudo fsarchiver -j 4 restfs buildos.fsa id=0,dest=/dev/sdc8
This will take 1-3 minutes to restore and when it is done, you will see something like this

Code: [Select]
vastone@vsido:/mnt/iso$ sudo fsarchiver restfs buildos.fsa id=0,dest=/dev/sdc8
[sudo] password for vastone:
Statistics for filesystem 0
* files successfully processed:....regfiles=90272, directories=10191, symlinks=12503, hardlinks=21, specials=44
* files with errors:...............regfiles=0, directories=0, symlinks=0, hardlinks=0, specials=0

Step 3: Change the uuid of your swap partition on the new restored device

Note - Do this only if you have a swap section in your fstab

First in terminal run:

Code: [Select]
sudo blkid
You will see something like this as one of the entries

Code: [Select]
/dev/sdb8: UUID="c462e964-62a3-4119-ac3b-1a0367905b95" TYPE="swap"
In SpaceFM mount the new drive you just created

Edit /etc/fstab and make sure the entry for swap has the same drive and uuid that you got from blkid

Step 4: Now make sure grub sees the new partition

Code: [Select]
sudo update-grub
Reboot and login into the build OS...

Profit!

NOTE - If you run this restore on the same machine, there will be a uuid conflict and it will not boot.  In the next message I will write up how to get around that by making a grub tweak
VSIDO      VSIDO Twitter     VSIDO FB     VSIDO Google+   

I dev VSIDO

VastOne

  •      v-ger
  • Posts: 3861
    • View Profile
    • VSIDO Community
Re: How To - Fsarchiver and the Root File System
« Reply #1 on: March 01, 2013, 05:06:07 AM »
Part II

How To - Correct UUID When Installing to Same Machine

This will show you what to do when restoring the fsarchive to same machine. Because fsarchive preserves the UUID you must change it or you will not be able to boot to the partition  This is how you correct it

Step 1: First you need to change the uuid of the new restored partition. To do that run this:

Code: [Select]
sudo tune2fs -U random /dev/sdxx
where xx is the actual partition letter and number you are restoring to. In my example it is sdc8 so it is:

Code: [Select]
sudo tune2fs -U random /dev/sdc8
Now find out what the new uuid is

Code: [Select]
sudo blkid
that shows my new uuid on sdc8 now as:

Code: [Select]
f20b1310-fb1a-46b9-abb0-0f04f0a1a254
Now in SpaceFM mount and go to the partition that you created, in my example it is sdc8 and edit /etc/fstab as root

You will see the first entry UUID as something similar to this

Code: [Select]
#Entry for /dev/sdc3 :
UUID=fe400ca1-e2d6-4d7a-b0e2-4cc01c821fc8 / ext4 errors=remount-ro 0 1

You need to change it to the new uuid for sdc8 from what blkid told us

Code: [Select]
#Entry for /dev/sdc8 :
UUID=f20b1310-fb1a-46b9-abb0-0f04f0a1a254 / ext4 errors=remount-ro 0 1

Step 2: Getting grub to recognize the restored fsarchive

Make sure sdc8 is unmounted and now run all of this one after the other:

Code: [Select]
sudo mount /dev/sdc8 /mnt

sudo mount --bind /dev /mnt/dev

sudo mount --bind /sys /mnt/sys

sudo mount --bind /proc /mnt/proc

sudo chroot /mnt

sudo update-grub

exit

ctrl d

When you are finished unmount by executing these from terminal (You do not need to do this if running this from LiveCD)

Code: [Select]
sudo umount /mnt/dev

sudo umount /mnt/sys

sudo umount /mnt/proc

sudo umount /mnt/

Now from your normal partition (mbr grub partition or whatever you boot to) make sure you update-grub so it can see the new partition:

Code: [Select]
sudo update-grub
and when you reboot again the the normal sdc8 VSIDO grub entry will boot correctly

Profit!
VSIDO      VSIDO Twitter     VSIDO FB     VSIDO Google+   

I dev VSIDO

VastOne

  •      v-ger
  • Posts: 3861
    • View Profile
    • VSIDO Community
Re: How To - Fsarchiver and the Root File System
« Reply #2 on: March 01, 2013, 05:07:11 AM »
It is my birthday today, and the above is my gift to you all

Enjoy!  :)
VSIDO      VSIDO Twitter     VSIDO FB     VSIDO Google+   

I dev VSIDO

tlinsley

  • Posts: 47
    • View Profile
Re: How To - Fsarchiver and the Root File System
« Reply #3 on: March 01, 2013, 06:55:31 AM »
Happy Birthday VastOne!

Very nice tutorial.

You probably already knew this, but just in case you didn't, the "-j" option can be added to the fsarchiver command to enable multi-threading of compression/decompression.  So if a person had a snazzy four core machine, they could add "-j 3" to their fsarchiver commands thereby utilizing three of the four available cores and ultimately speeding things up a bit.  Said person could even specify "-j 4" as long as they weren't planning on doing anything else while fsarchiver was running.  ;)

VastOne

  •      v-ger
  • Posts: 3861
    • View Profile
    • VSIDO Community
Re: How To - Fsarchiver and the Root File System
« Reply #4 on: March 01, 2013, 03:23:21 PM »
Thank you tlinsley!

I think the -j option defaults to using all 4 of my cores without that flag.  When I run this, my cpu is maxed at 100% and it only takes 2 minutes to back up and even less time to restore without using that option

It is good to know that flag in case you need it to use half of your cpu power, but when I do this I shut down everything and let fsarchiver have the whole show

I will test it with -j4 enabled to see if it is different..  Thanks for the tip.  ;D
VSIDO      VSIDO Twitter     VSIDO FB     VSIDO Google+   

I dev VSIDO

Sector11

  • Posts: 870
    • View Profile
Re: How To - Fsarchiver and the Root File System
« Reply #5 on: March 01, 2013, 05:55:22 PM »
Feliz Cumpleaños
Happy Birthday in español

Thanks for this VastOne, hope you had a great day ... did you manage to blow out all the candles without passing out?

That's looks easy enough.  I'll have to play with it ...
I should have learned this years ago.

Hmmm I can -j 2 it and still play.
Yup, gotta play with this...

Edit:  Did I miss something?

you should be able to download the link I will provide and restore exactly the build machine / FS

Stay Home

VastOne

  •      v-ger
  • Posts: 3861
    • View Profile
    • VSIDO Community
Re: How To - Fsarchiver and the Root File System
« Reply #6 on: March 01, 2013, 06:47:42 PM »
Wow... Was I ever wrong...

I just did a restore with the -j 4 option and it finished a 1 gig restore in 24 seconds...

Incredible..

Later on I will do a savefs with it to see how fast that is...

Incredible..

Thanks tlinsley!
VSIDO      VSIDO Twitter     VSIDO FB     VSIDO Google+   

I dev VSIDO

VastOne

  •      v-ger
  • Posts: 3861
    • View Profile
    • VSIDO Community
Re: How To - Fsarchiver and the Root File System
« Reply #7 on: March 01, 2013, 08:13:33 PM »
I did the backup portion with -j 4 as well.  It took less than a minute for the backup from 5 gig to a 959 MiB fsa of the build system

As long as I have been using fsarchive, I always took for granted that it was 'fast enough'

Boy was I wrong...

VSIDO      VSIDO Twitter     VSIDO FB     VSIDO Google+   

I dev VSIDO