Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm currently engaged in a non-work related homework exercise. I have an ext4 filesystem sitting on a logical volume. I'm testing different performance tuning strategies and this idea occurred to me. Since pvmove can move individual and ranges of extents, is there a way to map out what physical extents hold a particular file (in theory it can be backing files for a database, or a large commonly accessed file share) and move them to a particular storage device (for example I have a regular HDD and an SSD drive in the same LVM Volume Group)?

I thought of using "filefrag" but then it occurred to me that I wasn't 100% on whether the extent numbers would necessarily be used in sequential order (so knowing how many sectors in ext4 sees a file isn't necessarily going to let me figure out what which extent numbers/volumes the file is physically sitting on.

Any ideas?

share|improve this question

1 Answer 1

up vote 9 down vote accepted

The two main ingredients are hdparm --fibmap file, which tells you where the file is physically located within the LV, and lvs -o +seg_pe_ranges,vg_extent_size which tells you where the LV is physically located on your device(s).

The rest is math.

So, for example:

# hdparm --fibmap linux-3.8.tar.bz2 

linux-3.8.tar.bz2:
 filesystem blocksize 4096, begins at LBA 0; assuming 512 byte sectors.
 byte_offset  begin_LBA    end_LBA    sectors
           0     288776     298511       9736
     4984832     298520     298623        104
     5038080     298640     298695         56
     5066752     298736     298799         64
     5099520     298824     298895         72
     [...]

I don't know why this is so fragmented - downloaded with wget. May be a good example because as you see, you get a headache without scripting this somehow, at least for fragmented files. I'll just take the first segment 288776-298511 (9736 sectors). The count is wrong since it's not 512 byte sectors, but anyhow.

First check that this data is actually correct:

# dd if=linux-3.8.tar.bz2 bs=512 skip=0 count=9736 | md5sum
9736+0 records in
9736+0 records out
4984832 bytes (5.0 MB) copied, 0.0506548 s, 98.4 MB/s
7ac1bb05a8c95d10b97982b07aceafa3  -

# dd if=/dev/lvm/src bs=512 skip=288776 count=9736 | md5sum
9736+0 records in
9736+0 records out
4984832 bytes (5.0 MB) copied, 0.123292 s, 40.4 MB/s
7ac1bb05a8c95d10b97982b07aceafa3  -

Wheeee.That's identical so we are reading the LV-src at the right place. Now where's the source-LV located?

# lvs -o +seg_pe_ranges,vg_extent_size
  LV          VG   Attr      LSize   Pool Origin Data%  Move Log Copy%  Convert PE Ranges             Ext   

[...]
 src         lvm  -wi-ao---   4.00g                                            /dev/dm-1:5920-6047   32.00m
[...]

Now that's boring, this LV isn't fragmented. No headache here. Anyway.

It says src is on /dev/dm-1 and starts at PE 5920 and ends at PE 6047. And PE size is 32 MiB.

So lets see if we can read the same thing from /dev/dm-1 directly. Math-wise, this is a little bleargh since we used 512 byte blocksize earlier... :-/ but I'm lazy so I'll just calculate the MiB and then divide by 512! Ha! :-D

# dd if=/dev/dm-1 bs=512 skip=$((1024*1024/512 * 32 * 5920 + 288776)) count=9736 | md5sum
9736+0 records in
9736+0 records out
4984832 bytes (5.0 MB) copied, 0.0884709 s, 56.3 MB/s
3858a4cd75b1cf6f52ae2d403b94a685  -

Boo-boo. This isn't what we're looking for. What went wrong? Ah! We forgot to add the offset that's occupied by LVM at the beginning of a PV for storing LVM metadata and crap. Usually this is MiB-aligned, so just add another MiB:

# dd if=/dev/dm-1 bs=512 skip=$((1024*1024/512 * 32 * 5920 + 288776 + 1024*1024/512)) count=9736 | md5sum
9736+0 records in
9736+0 records out
4984832 bytes (5.0 MB) copied, 0.0107592 s, 463 MB/s
7ac1bb05a8c95d10b97982b07aceafa3  -

There you have it.

share|improve this answer
1  
Some day they will build statues to your honor. –  Bratchley Apr 5 '13 at 20:05
    
One thing though, any idea why my hdparm invocation is segfaulting? –  Bratchley Apr 5 '13 at 20:09
    
Actually, strike that, looks like I need to improve on my google-fu. It's a new RHEL bug pertaining to SSD's and LVM. I'll take this to mean that file's already on the SSD. Ha –  Bratchley Apr 5 '13 at 20:23
    
Is there another utility to determine the file's position in the LV until they fix this? –  Bratchley Apr 5 '13 at 20:25
    
You already mentioned it - filefrag. Otherwise google if any other tool does fibmap or fiemap. –  frostschutz Apr 5 '13 at 21:00

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.