Our Goal is to use huge storage as 200TB for Kafka machine

In order to achieve this Goal , we create volume using LVM over 4 physical disks in RAID10 , when each disk size is 50T ,

Note - each disk as sdb/sdc/sdd/sde is actually RAID10 , when each RAID10 is holding 12 disks

In our kafka server we have 4 disks ( example from lsblk )

sdb                  8:16   0   50T  0 disk 
sdc                  8:32   0   50T  0 disk 
sdd                  8:48   0   50T  0 disk 
sde                  8:64   0   50T  0 disk 

So we wrote the following procedure:

We combine the disks /dev/sdb and /dev/sdc. and /dev/sdd and /dev/sde , by adding them to the same volume group.- DB_vg

vgcreate DB_vg /dev/sdb /dev/sdc /dev/sdd /dev/sde

we create logical volumes ( we are using here 200T size )

lvcreate -L200T -n DB_lv DB_vg

We create the XFS filesystem

mkfs.ext4  -j -m 0  /dev/DB_vg/DB_lv -F

We create folder , in order to mount the LV to this folder

mkdir /var/lib/kafka_broker

mount /dev/DB_vg/DB_lv /var/lib/kafka_broker

from df -h , its should looks like this ( example )

/dev/mapper/DB_vg-DB_lv  200T   61M  200T   1%  /var/lib/kafka_broker

from lsblk

└─DB_vg-DB_lv 253:4    
└─DB_vg-DB_lv 253:4   
└─DB_vg-DB_lv 253:4    
└─DB_vg-DB_lv 253:4  

from blkid

blkid | grep LVM2_member
/dev/sda2: UUID="Y5MbyB-C5NN-hcPA-wd9R-jmdI-02ML-W9qIiu" TYPE="LVM2_member"    <-- sda2 is the OS
/dev/sdc: UUID="mtg8CY-b3j9-K1qZ-CRTV-SZum-gDx3-lbm4ZX" TYPE="LVM2_member"
/dev/sdb: UUID="MvfzqO-PV8N-dror-psl4-PfUf-7coa-6cb8lQ" TYPE="LVM2_member"
/dev/sdd: UUID="C4n63l-Uk3E-D65G-WcgI-xic2-cJLi-eSTUAa" TYPE="LVM2_member"
/dev/sde: UUID="d743Xp-eDxr-Dygk-HGgy-Dh9c-K3cx-kHtyqo" TYPE="LVM2_member"

So now we can use the LV /dev/mapper/DB_vg-DB_lv that mount to /var/lib/kafka_broker folder

and we can use the folder /var/lib/kafka_broker as storage with 200T for kafka broker machine

My Questions are:

  1. is it safe? to use huge storage that created by LVM using 4 Huge disks?

  2. can we create XFS filesystem when storage is around 200T size?

  3. am I right about my procedure or maybe I am missing something?

  • The disks as sdb/sdc/sdd/sde are new scratch disks
    – yael
    Nov 1, 2023 at 19:00
  • in my procedure I not use pvcreate , shall I use it?
    – yael
    Nov 1, 2023 at 19:33
  • You need the pvcreate before you can create an LVM volume group. I suggest that if you've not tried this before, then you set up some (small) scratch disks. See the first part of this answer Nov 1, 2023 at 19:48
  • ok, I accept your comment , second - do you think my Questions are reasonable enough? , or I need to add some additional info ?
    – yael
    Nov 1, 2023 at 19:52
  • LVM used this way gives you one big filesystem, with zero load distribution. You can do that but I would prefer smaller volumes and let kafka itself handle the distribution. Kafka documentation even suggests letting it handle replication instead of relying on raid. kafka.apache.org/documentation/#diskandfs Nov 2, 2023 at 7:13

1 Answer 1

  1. As with any storage, this is as safe as your backups.

  2. XFS is supported up to 500TB, so this is fine.

  3. As far as I can tell, nothing is missing (vgcreate and vgextend can initialize PVs, so you don’t need to run pvcreate).

  • another thing if I can, as already mentioned each disk is actually represented the RAID10 , do you see a problem with that? ( I asked this because Kafka service need high disk performance )
    – yael
    Nov 1, 2023 at 20:11
  • 1
    I answered your questions, taking all the information therein into account. Nov 1, 2023 at 22:02
  • 1
    If the devices were not previously intialized as PVs with pvcreate(8), vgcreate will inititialize them, making them PVs. The pvcreate options for initializing devices are also available with vgcreate. Nov 2, 2023 at 6:15
  • 1
    @StéphaneChazelas wow, since October 2009! Thanks, I wasn’t aware of that! Nov 2, 2023 at 6:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .