Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a 22TB disk on /dev/sdb. How do I create a 22TB partition? Don't really care about the file system - ext4 or zfs is fine.

Running CentOS 6.2 - Partition will be used as a data dump. Only a single stream of data so being picky/choosy over what file system isn't really a concern right now. The disk is formed from 12x2TB nearline sas drives and a Dell Perc controller.

I just want a 22tb partition.

share|improve this question
1  
a few more pieces of information would be helpful. what OS are you using? what kind of performance do you expect or need to get from it? what's the underlying hardware? it is already a raid set? does it have redundancy? do you need the extra data protection features of zfs? –  Tim Kennedy Jan 14 '12 at 1:46
3  
Which tries have failed so far? fdisk, cfdisk? –  Nils Jan 14 '12 at 20:39

6 Answers 6

up vote 23 down vote accepted

The simplest solution is to use GPT partitioning, a 64-bit version of Linux, and XFS:

  • GPT is necessary because the MS-DOS-style MBR partition table created by fdisk is limited to 2 TiB disks. So, you need to use parted or another GPT-aware partitioning program instead of fdisk. (gdisk, gparted, etc.)

  • A 64-bit kernel is necessary because 32-bit kernels limit you to filesystems smaller than you're asking for. You either hit some limit based on 32-bit integers or end up not being able to address enough RAM to support the filesystem properly.

  • XFS is not the only solution, but in my opinion it is the easiest one for RHEL systems.

    You cannot use ext4 for this in RHEL 6. Although the filesystem was designed to support 1 EiB filesystems, there is an artificial 16 TiB volume size limit in the version of e2fsprogs included in RHEL 6 and its derivatives. Both Red Hat and CentOS call this out in their docs. (The ext4 16 TiB limit was raised considerably in RHEL 7 to 50 TiB.)

    ZFS may not be practical in your situation. Because of its several legal and technical restrictions, I can't outright recommend it unless you need something only ZFS gives you.

    Having ruled out your two chosen filesystems, I suggest XFS. It is the default filesystem in RHEL 7, it was available as a supported filesystem in all RHEL 6 versions, and was backported to the later RHEL 5 releases after RHEL 6 came out.

Bottom line, here's what you have to do:

  1. Check whether you have mkfs.xfs installed by running it without arguments. If it's not present, install the userland XFS tools:

    # yum install xfsprogs
    

    If that failed, it's probably because you're on an older OS that doesn't have this in its default package repository. You really should upgrade, but if that is impossible, you can get this from CentOSPlus or EPEL. You may also need to install the kmod_xfs package.

  2. Create the partition:

    If the 22 TiB volume is on /dev/sdb, the commands for parted are:

    # parted /dev/sdb mklabel gpt
    # parted /dev/sdb mkpart primary xfs 1 -1
    

    That causes it to take over the entire volume with a single partition. Actually, it ignores the first 1 MiB of the volume, to achieve the 4 KiB alignment required to get the full performance from Advanced Format HDDs and SSDs.

    You could skip this step, and format the entire volume with XFS. That is, you would use /dev/sdb in the example below instead of /dev/sdb1. This avoids the problem of sector alignment. In the case of a RAID array, there are no downsides worth speaking about, but I'd caution doing this on a single removable disk, since some OSes (Windows and OS X, for instance) will offer to format a partitionless hard drive for you every time you plug it in. Putting the filesystem on a partition solves this.

  3. Format the partition:

    # mkfs.xfs -L somelabel /dev/sdb1
    
  4. Add the /etc/fstab entry:

    LABEL=somelabel    /some/mount/point    xfs     defaults   0 0
    
  5. Mount up!

     # mount /some/mount/point
    

If you want to go down the LVM path, the above steps are basically just a more detailed version of the second set of commands in bdowning's answer. You have to do bdowning's first set of commands before the ones above.

LVM has certain advantages, at a complexity cost. For instance, you can later "grow" an LVM volume group by adding more physical volumes to it, thereby making space to grow the logical volume ("partition" kinda, sorta), which in turn lets you grow the filesystem living on the logical volume. (See what I mean about complexity? :))

share|improve this answer
2  
The ZFS part is debatable. The Linux port has come a long way since 2010 and compared to XFS it has many advantages –  TheLQ Jan 18 '12 at 18:03
    
Regarding GPT/MBR, doesn't that only really apply if /boot is part of /? The MBR shouldn't care how big / is if it only has to mount a small /boot right? I could be wrong. –  jonescb Jan 20 '12 at 4:39
1  
@jonescb: The location of /boot doesn't have any bearing on the limitations of MBR. If you need a partition over 2 TB, you can't use MBR partitioning. It is true, however, that it is possible to work around lack of BIOS support for booting from GPT by putting /boot on a smaller MBR-partitioned disk. Once the kernel is up, you don't have to worry about BIOS limitations because it knows how to interpret the GPT partition table. If your machine is EFI-based, you don't need to do this dance, because EFI understands GPT. –  Warren Young Jan 20 '12 at 6:32

Just as an alternative to the other suggestions.
You don't have to partition a disk at all.
You could simple create a Volume Group, with one or more Logical Volumes.

pvcreate /dev/sdb
vgcreate data /dev/sdb
lvcreate --name dump -L '100%VG' data

Now you have a logical volume that you can format with any filesystem type you wish.

mkfs.XXXX /dev/mapper/data-dump #<- XXXX can be ext4, xfs, btrfs, reiser
mount /dev/mapper/data-dump /mntpt
share|improve this answer
    
LVM is basically an advanced form of partitioning. It is pointless to use LVM if you are just going to create a single LV using all of the space, so you may as well just mkfs directly on the whole disk device. –  psusi Jan 17 '12 at 18:42
1  
no LV's add tools for snapshots of live data, resizable, multiple copies of metadata; far more flexible than a simple fs on a device. –  bdowning Jan 18 '12 at 3:03
    
You don't need multiple copies of metadata when you have no metadata ( partition table ) in the first place. Snapshots require free space, as does adding/expanding volumes, hence why it is rendered pointless if you just create a single logical volume using all of the space right off the bat. If you want the features of LVM, then you should start with a smaller volume so you have plenty of free space to use later. –  psusi Jan 18 '12 at 16:39
    
I wasn't recommending LVM in this individual's case, simply listing it as an alternative to "other" partitioning, non-partitioning answers/solutions. It's up to her to read all the responses, follow up with her own research and then decide which course of action best suits her needs. –  bdowning Jan 18 '12 at 17:57

Question to the question: You asked 'how to partition 22TB disk' and then in the question again, you said, you just wanted a 22TB partition. So this is ambiguos in first place.

If you already have a single block device which can support 22TB of space on it, then you already posses whole 22TB partition. All you need is a filesystem on top of it, which will make the device mountable and usable for reading/writing by system processes. More ever, you need to have a Linux kernel running in 64-bit mode with a filesystem module/driver that supports and scales to 22TB of data growth, can handle the ins and outs of managing the data on the (single) block device with ease. Performance is altogether another dimension to it. In such case, I would opt to choose XFS as my filesystem, for the reason that it is a 64-bit filesystem and capable of handling filesystems as large as a million terabytes. It supports upto 9 EXABYTES.

2^63  = 9 x 1018 = 9 exabytes 

For more details on XFS: http://oss.sgi.com/projects/xfs/

If you are looking for further partitioning the huge 22TB block device, then use gparted to split the device into usable partitions and then format them with the filesystems to make them mountable.

It seems that you have got hardware RAID controller, since you mention that you have got DELL perc RAID controller -- which will mean that, you have to tell which RAID configuration (precisely which RAID level are you using?) and in most cases, you are not going to get complete 22TB of space for use, I could be wrong though.

share|improve this answer
    
I was going to suggest xfs as well, but xfs_check will take a large amount of memory, and a long time to run w/22G fs. Between physical and swap one would need at least 32G to check, that is if he ever cares to check the fs on a 'datadump' (whatever that is ;) –  bdowning Jan 14 '12 at 11:43
    
@bdowning, you might want to correct it to 22T fs :) –  Nikhil Mulley Jan 14 '12 at 11:44
    
It is 22TB Raid 5. I have a lot of disks :) –  LVLAaron Jan 14 '12 at 13:20
    
That was a typo(brain fart). I have the same device, 10TB, with LSI Megaraid (same card as Dell Perc) –  bdowning Jan 14 '12 at 13:27
    
@bdowning: xfs_check does indeed use a lot of memory, but the manual page(8) mentions: "Note that using xfs_check is NOT recommended. Please use xfs_repair -n instead, for better scalability and speed.". –  Cristian Ciupitu Nov 29 '12 at 6:10

I'm not sure this is currently possible using the standard partition table. In the standard partition table scheme, volumes are limited to 232 sectors. With 512 bytes per sector, you'd simply run out numbers to assign to sectors around 2TB.

However, you ought to be able to do this if you use a GUID Partition Table instead of a standard one. GUID partition tables allow for volumes to extend into the zettabyte range. Most Linux distros are bootable from a GUID volume, however no version of Windows (except for Windows 7 on EFI) currently is.

Some tools like fdisk can't work with GUID volumes, however other tools like GParted can. Once you create your GUID partition table, you ought to be able to create a volume using one of the several common filesystems which support a volume of that size (e.g. EXT4.)

share|improve this answer
    
On a 32-bit system may be. I have been able to create a 3.5TB partition on a 64-bit Ubuntu server supported under version 8.04 and higher. Speaking of cyberciti.biz/tips/… –  Karlson Jan 14 '12 at 4:20
    
To my knowledge you may not even need a partition table to put a filesystem on a harddrive. Plus according to the OP it's not a boot drive just a large dump space. –  Karlson Jan 14 '12 at 5:47

You shouldn't need any partitioning when using ZFS, just create a ZFS pool on your 22 TB device and a file system in it if you don't want to use the default one and that's it. If for some reason, zpool doesn't support using the whole disk, first create an EFI label and a partition using the whole space available inside then use that partition to create the pool.

For several reasons, I wouldn't recommend using anything but ZFS for such a large file system. The most obvious is if you have a brutal power off (eg: kernel panic or power shortage), fsck can take a painful length of time to recover traditional file systems. ZFS on the other hand doesn't need fsck so will import the pool instantly.

share|improve this answer

If you are not looking for the redundancy or ability to back it up you can probably do:

mkfs -t ext4 /dev/sdb
share|improve this answer
4  
Have you tried it? I changed the mkfs to specify the filesystem –  Karlson Jan 14 '12 at 4:05
1  
@AaronJAnderson an explanation would help. –  maxmackie Jan 14 '12 at 5:26
    
That command will not work if the device is over 2TB –  LVLAaron Jan 14 '12 at 13:26
1  
@AaronJAnderson I have created 3.5TB volume using reiserfs and ext3 so if the ext4 claims to have maximum size of the volume of 16 exabytes I don't see any reason that 22TB won't work. –  Karlson Jan 14 '12 at 20:04
1  
the version of e2fsutils that ships w/ this version of linux (and most others at the moment do not support over 16TB) –  LVLAaron Jan 17 '12 at 8:00

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.