23

I had asked a question about ext4lazyinit here. From all I have read and understood, it simply allows you to start using the hdd without creating all the inodes for your file system.

The only way that you know or can measure is monitoring ext4lazyinit in iotop. Is there a way to get its progress as a percentage?

5
  • 2
    check this patch discussion Jun 14 '17 at 7:06
  • Did you try bar [1] or pv [2] utilities? [1]: [archlinux.org/packages/community/any/bar/] [2]: [archlinux.org/packages/community/x86_64/pv/]
    – mattia.b89
    Jun 16 '17 at 21:17
  • 1
    @mattia.b89 ext4lazyinit isn't a userspace program, it's a background kernel process. Check the link in the question pointing to shirish's original question about this for info. Aug 17 '17 at 17:05
  • Also, I would tend to agree with Ted T'so's assessment in the patch discussion that @KrzysztofStasiak linked, if you are in a situation where you have to wait for this to finish, you probably should not be using it. Aug 17 '17 at 17:11
  • @KrzysztofStasiak could you make it an answer, I think this is the nearest I can get to the answer, FWIW Ted Tso is synonymous with ext* so whatever his beliefs are, they are important.
    – shirish
    Aug 18 '17 at 3:40
4

I seem to have found a way to approximate the progress of ext4lazyinit.

TL;DR: see Script below.

This method assumes that the disk has never been disconnected (nor has the system been rebooted) since the very first time the partition was mounted, and that you have written exactly as much data to the partition as is in use on it (so no deletion or modification of files).

Step 1: Compare the partition size in fdisk (converted to kiB) to the number of 1K-blocks shown in df. Subtract (number of 1K-blocks) from (partition size in kiB) to get (approx size of inode table).

EDIT: Example, fdisk:

Sector size (logical/physical): 512 bytes / 4096 bytes
(...omitted...)
Device     Start         End     Sectors  Size Type
/dev/sdd1   2048 11720978398 11720976351  5.5T Linux filesystem

df:

Filesystem              1K-blocks       Used  Available Use% Mounted on
/dev/mapper/workbackup 5813233164 1217095176 4596121604  21% /mnt/backup_work

11720976351 sectors * 512 bytes / sector / 1024 = 5860488175.5 kiB (nearly 5.5 TiB, like fdisk says). Minus df's 5813233164 equals 47255011.5 kiB (some 45 GiB) for the approximate inode table size.

Step 2: Get (total kiB written to partition):

awk '{ print $3"\t"$10 }' /proc/diskstats

Pick the right line for your partition and convert it to kiB.

EDIT: example:

sdb     260040
sdb1    260040
sdd     2530109116
sdd1    2530108940

Using sdd1 in my case, total kiB written = 2530108940 sectors * 512 bytes / sector / 1024 = 1265054470 kiB (nearly 1.2 TiB).

Step 3: Only needed if you have already written any data to the filesystem. Subtract (number of 1K-blocks USED, shown in df) from (total kiB written to partition) to get (approx kiB written to inode table).

EDIT: example: approximate kiB written to inode table = 1265054470 (from step 2) - 1217095176 (see df output in step 1) = 47959294 kiB (45.7 GiB)

Step 4: Divide (approx kiB written to inode table) by (approximate size of inode table in kiB) and multiply by 100 to get the progress as a percentage.

EDIT: example: approximate progress = 47959294 / 47255011.5 * 100% = 101.5%

Script

Or to write that as a partial script (where I refuse to script a call to fdisk, for safety reasons):

let sectorsize=$(cat /sys/block/sda/queue/hw_sector_size)
let partsize=$2*$sectorsize/1024
let fssize=$(df -- "$3" | tail -n -1 | awk '{print $2}')
let approxinodetablesize=$partsize-$fssize
let tkw=$(awk "/$1/"' {print $10}' /proc/diskstats | head -n 1)*$sectorsize/1024
let used=$(df -- "$3" | tail -n -1 | awk '{print $3}')
let tkw_inodetable=$tkw-$used
echo "Approximate progress: $(bc -l <<< "$tkw_inodetable*100.0/$approxinodetablesize") %"

Call with $1 = "name of partition" (e.g. sdd1), $2 = "sectors of partition according to fdisk", $3 = "mount point without trailing slash"

Test results

I have only tested my method once. Setup:

  • 6 TB partition

  • encryption using cryptsetup

  • filesystem created with default parameters except -m 0

  • 279 GiB of files written to the partition before ext4lazyinit completed.

Result: a reading of 99.7 % at the time of completion :-)

EDIT: same disk, after writing almost another TiB of data to it, now yields an estimate of 101.5%. Accurate enough to be useful, I think.

2
  • it would have been nicer to have an example given so we have an idea as to what exactly we would be looking at.
    – shirish
    Aug 11 '19 at 1:47
  • @shirish Thank you for your suggestion. Aug 11 '19 at 22:22
2

Check this patch discussion. You can initialize system without lazyinit, but can't measure it. If you have time to wait, just wait. You can try patch from discussion, but according to Patchwork it has state "rejected".

2

(you will see, english is not my mother language)

At least in Ubuntu-Based systems(I use ZorinOS 15) is indirect mesurement of progress possible as follows:

  1. You can get information about device and current sector which is processing by ext4lazyinit.

a) Therefore you have to write "1" in /proc/sys/vm/block_dump. (Imortant: You have to set block_dump back to "0" if you are ready with work):

   echo 1 > /proc/sys/vm/block_dump

b) Then you get the information by

tail -f /var/log/syslog | grep ext4

output will be like(maybe you have to wait, till ext4lazyinit access to device):

May  7 13:19:59 xxx kernel: [ 1130.643118] ext4lazyinit(1070): WRITE block 9235888384 on md0 (2048 sectors)
May  7 13:20:01 xxx kernel: [ 1132.594961] ext4lazyinit(1070): WRITE block 9235890432 on md0 (2048 sectors)
May  7 13:20:01 xxx kernel: [ 1132.632755] ext4lazyinit(1070): WRITE block 9235892480 on md0 (2048 sectors)

You see current sector (block 9235892480) and device (md0) on wich ext4lazyinit is working.

  1. You can get information, what is the count of sectors on your device(in my case /dev/md0). This is posible e.g. with graphical tool "gparted" or by command "fdisk -l ". In my case:

     sudo fdisk -l /dev/md0
    

Output will be like that:

Festplatte /dev/md0: 5,5 TiB, 5990282952704 Bytes, 11699771392 Sektoren
...
  1. Now you have the number of sector wich is currently on processing and device size in sectors, so you can calculate the progress of ext4lazyinit.

  2. set block_dump back to "0"

    echo 0 > /proc/sys/vm/block_dump
    

See also

Für unser eins

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.