Formatting a large drive and reclaim thy reserved space (max drive size)

[statmonkey]

Since tomorrow I head to the track and probably won't be on for a while I wanted to post at least one more of these fwiw how to things.  This one is as much for myself as a reminder as anything else.

If you are using gparted or fdisk to format large drives you are going to have problems. This is due to fdisk having a 2TB partition limitation. There are several different ways to overcome this Debian can handle larger partitions, the Linux kernel and GNU tools can use the larger partitions once created, but to me the easiest method for getting that initial partition created requires using parted.  This method works because parted allows the gpt partition table that fdisk does not.

Why would you do this?

Fixing this issue using the command line is fast and relatively easy, it also helps you to be familiar with the command line tools for partitioning should you have drive issues.  I also feel like I have more control, the command line offers several options that GUI disk managers do not.

How technical is this job?

There is nothing in here that should give even the most basic user a problem.  A little common sense and some patience to make sure you get it right.  This method does involve working with disks so you need to be careful to get the right disk and understand that you will be deleting whatever is on the disk since you are formatting it.

What is covered?

We will walk through using parted and some of its commands, creating labels and mkfs all from the command line.

What do you need?

A command line, a hard disk connected externally, parted and a terminal.

What are the steps?

• Get parted


• Find the disk


• Understanding using parted


• Format the disk


• Get our reserved space back

Steps

Get parted

We will be using parted for this operation so fire up your cli and run:

Code Select Expand

sudo apt-get parted

Find the right disk

I use a fairly standard External SATA USB Cradle to attach and communicate to the system.  Once the drive is in the bay and the USB connected we need to make sure we know where the system has placed the device.  Since we'll need to be root to run parted open up a terminal and login in as root.  There are several ways to check where the drive is but the simplest to me is just run dmesg and you get a list of drive activity as part of the summary (assuming you just plugged the drive in).  In my case the newest attached drive is /dev/sdm.

Code Select Expand

dmesg | tail -5

You should see something like this:

[69229.559950]  sdm: unknown partition table
[69229.560655] sd 27:0:0:0: [sdm] Very big device. Trying to use READ CAPACITY(16).
[69229.562168] sd 27:0:0:0: [sdm] No Caching mode page present
[69229.562172] sd 27:0:0:0: [sdm] Assuming drive cache: write through
[69229.562175] sd 27:0:0:0: [sdm] Attached SCSI disk

So now that we know where our drive is we can open parted.

Using Parted

Before starting with parted it doesn't hurt to refresh what  parted is.  Parted works similarly but not the same as fdisk and if you are familiar with fdisk you are certainly going to have no issues with parted.  Just running the command parted will give you the parted environment and you can see a listing of "help" which shows commands that you can run with parted. 

For example:

Code Select Expand

parted -a optimal /dev/sdm

would show the partitions on device sdm and the command print would list out the physical information on the drive.  Unlike fdisk parted doesn't need a separate tool set to format and label it's all included.

Code Select Expand

mkfs #partition #fs-type

will tell parted to create whatever file system you have for #fs-type on #partition and if you make any mistakes you run

Code Select Expand

rm #partition

and it will remove #partition.  To exit parted all you have to do is type:

Code Select Expand

quit

One of the flaws in the parted documentation is that it doesn't lay out using gpt partition tables or even using ext4, ext3 or btrfs. This is annoying since there is a simple work around but then again if it was corrected we would not need this how to. All of these things can be done with parted.  The actual partition type you use is incidental to the process, for our example we will use ext4 but recognize it can be about any file-system we want.

Formatting the disk

To create a single, large partition from the command line using parted and our disk on /dev/sdm we will start by creating the gpt label. Due to the issues about parted not knowing internally about ext4 we will do this outside of the parted environment.  In other words we will call parted from the command line as part of the executable.

Code Select Expand

parted -a optimal /dev/sde mklabel gpt

Parted will helpfully warn you:

Warning: The existing disk label on /dev/sdm will be destroyed and all data on
this disk will be lost. Do you want to continue?
Yes/No?

Select yes, we want to rewrite the disk label. Next I need to set up the partition with

Code Select Expand

parted -a optimal /dev/sdm mkpart -- primary ext4 1 -1

Parted will now tell you that you need to update your fstab. This is fine information but not relevant for our external drive. If you are working with a drive on your system just make note that you need to fix your fstab later and move on. What the above command is saying is that we want a primary partition on partition one of /dev/sdm. What you need to recognize is that we now have a partition to work with and not just a device. That means that the remaining commands will use /sdm1 which is the partition and not /sdm which is the device. It's a subtle but important difference. We can now format our partition.

Code Select Expand

mkfs.ext4 -L 'External' /dev/sdm1

What we are saying here is format to ext4 and label External our device at /dev/sdm1. Our system responds with:

mke2fs 1.42.8 (20-Jun-2013)
Filesystem label=External
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
183148544 inodes, 732566272 blocks
36628313 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
22357 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848, 512000000, 550731776, 644972544

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

All well and good except ... see that line "36628313 blocks (5.00%) reserved for the super user"?  On a small drive it's not important but on a 3 TB drive 5% is a significant amount of space.  This default 5% is reserved for the system as a failsafe.  It was originally intended (when we had small drives) to stop the drive becoming unusable when full.  It's never stopped me from over filling a drive by the way and seems pretty worthless.  Since this is an external drive only used for storage I don't really care.  I probably would never use the whole drive but it makes me crazy to be forfeiting so much space, so let's fix that.

Get our reserved space back

We don't need to have 5% reserved, 1% would be plenty and it's doubtful we are going to fill this whole thing up anyway.  So we have one more command to run:

Code Select Expand

sudo tune2fs -m 1 /dev/sdm1

Our terminal tells us:

Setting reserved blocks percentage to 1% (7325662 blocks)

So we know we now have 1% reserved and not 5%.

Summary

This should all go pretty fast, I think it takes me about 5 minutes to do the step by step process, being careful.  Since I use Spacefm it sees the drive automatically and I can begin dropping things into it from the moment I click on it. Given how easy this is I don't ever waste my time with the GUI apps like disk manager or gparted, I just fire up the cli and go. Enjoy your new large drive!

[VastOne]

Incredible and helpful How To StatMan!

parted will now be on VSIDO going forward, sorry about that oversight...

[statmonkey]

That was a short nap.  There was a typo which is now fixed.  My editing skills are a little weak.

Parted is a nice little app and didn't consider that an oversight by any means.