1

I have a question I'm struggling to research an answer for. I am building a Ubuntu media server at home and decided to carve up my 2TB bulk storage disk into various partitions.

I created a 700GB partition for Music and formatted EXT4. Then migrating about 370GB from an NTFS external HD, using Beets to build my music library and copy the files. Problem is the total directory size is exceeding 730GB!

Why is this? Is there an error in the way Beets is processing the files? Or is this something to do with the filesystem choice?

Beets is fetching album art and some other metadata, but this definitely would not increase the overall footprint by a factor of two?

Any advice appreciated! I'm new to Linux filesystems so probably missing something elementary about inodes, journaling?

dumep2fs shows:

chris@MICROSERVER:~$ sudo dumpe2fs /dev/mapper/MICROSERVER--vg-music 
[sudo] password for chris: 
dumpe2fs 1.42.9 (4-Feb-2014)
Filesystem volume name:   <none>
Last mounted on:          /media/ubuntu/62c278e1-4c1e-470a-b68c-f0a5d777c165
Filesystem UUID:          62c278e1-4c1e-470a-b68c-f0a5d777c165
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
cFilesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
Filesystem flags:         signed_directory_hash 
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              1966080
Block count:              7864320
Reserved block count:     393216
Free blocks:              7681951
Free inodes:              1966057
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1022
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
Filesystem created:       Sat Jul 25 10:22:35 2015
Last mount time:          Sun Aug  2 21:28:44 2015
Last write time:          Sun Aug  2 21:28:44 2015
Mount count:              23
Maximum mount count:      -1
Last checked:             Sat Jul 25 10:22:35 2015
Check interval:           0 (<none>)
Lifetime writes:          121 MB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:           256
Required extra isize:     28
Desired extra isize:      28
Journal inode:            8
Default directory hash:   half_md4
Directory Hash Seed:      f7ccf1c6-402b-4666-bdd1-2a10d7fa1948
Journal backup:           inode blocks
Journal features:         journal_incompat_revoke
Journal size:             128M
Journal length:           32768
Journal sequence:         0x00000023
Journal start:            0

df -h

chris@MICROSERVER:~$ df -h
Filesystem                          Size  Used Avail Use% Mounted on
/dev/mapper/MICROSERVER--vg-root     20G   12G  7.5G  60% /
none                                4.0K     0  4.0K   0% /sys/fs/cgroup
udev                                3.9G  4.0K  3.9G   1% /dev
tmpfs                               796M  1.2M  795M   1% /run
none                                5.0M     0  5.0M   0% /run/lock
none                                3.9G     0  3.9G   0% /run/shm
none                                100M     0  100M   0% /run/user
/dev/sda1                           236M   98M  126M  44% /boot
/dev/mapper/MICROSERVER--vg-music   673G  641G  372K 100% /media/music
/dev/mapper/MICROSERVER--vg-backup   50G   38G  9.3G  81% /media/backup
/dev/sdb2                           673G   33G  606G   6% /music

df -i

Filesystem                           Inodes  IUsed    IFree IUse% Mounted on
/dev/mapper/MICROSERVER--vg-root    1310720 267682  1043038   21% /
none                                1017972      2  1017970    1% /sys/fs/cgroup
udev                                1006939    502  1006437    1% /dev
tmpfs                               1017972    514  1017458    1% /run
none                                1017972      5  1017967    1% /run/lock
none                                1017972      1  1017971    1% /run/shm
none                                1017972      2  1017970    1% /run/user
/dev/sda1                             62248    311    61937    1% /boot
/dev/mapper/MICROSERVER--vg-music  44802048  97110 44704938    1% /media/music
/dev/mapper/MICROSERVER--vg-backup  3276800   1434  3275366    1% /media/backup
/dev/sdb2                          44802048  11710 44790338    1% /music

lsblk -b

NAME                              MAJ:MIN RM          SIZE RO TYPE MOUNTPOINT
sda                                 8:0    0  160041885696  0 disk 
├─sda1                              8:1    0     254803968  0 part /boot
├─sda2                              8:2    0          1024  0 part 
└─sda5                              8:5    0  159784108032  0 part 
  ├─MICROSERVER--vg-root (dm-0)   252:0    0   21441282048  0 lvm  /
  ├─MICROSERVER--vg-swap_1 (dm-1) 252:1    0    8552185856  0 lvm  [SWAP]
  ├─MICROSERVER--vg-music (dm-2)  252:2    0   32212254720  0 lvm  
  └─MICROSERVER--vg-backup (dm-3) 252:3    0   53687091200  0 lvm  /media/backup
sdb                                 8:16   0 2000398934016  0 disk 
├─sdb1                              8:17   0  734003200000  0 part /media/music
└─sdb2                              8:18   0  734003200000  0 part /music

mount

/dev/mapper/MICROSERVER--vg-root on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/cgroup type tmpfs (rw)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
none on /run/user type tmpfs (rw,noexec,nosuid,nodev,size=104857600,mode=0755)
none on /sys/fs/pstore type pstore (rw)
/dev/sda1 on /boot type ext2 (rw)
/dev/mapper/MICROSERVER--vg-music on /media/music type ext3 (rw)
/dev/mapper/MICROSERVER--vg-backup on /media/backup type ext3 (rw)
/dev/sdb2 on /music type ext4 (rw)
/dev/sdb1 on /media/music type ext4 (rw)
systemd on /sys/fs/cgroup/systemd type cgroup (rw,noexec,nosuid,nodev,none,name=systemd)

sudo lvdisplay

--- Logical volume ---
LV Path                /dev/MICROSERVER-vg/root
LV Name                root
VG Name                MICROSERVER-vg
LV UUID                Lwx28M-m2k9-XIpB-fUEC-HiWa-5QsN-SlNOt3
LV Write Access        read/write
LV Creation host, time MICROSERVER, 2015-07-24 20:55:58 +0100
LV Status              available
# open                 1
LV Size                19.97 GiB
Current LE             5112
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           252:0

--- Logical volume ---
LV Path                /dev/MICROSERVER-vg/swap_1
LV Name                swap_1
VG Name                MICROSERVER-vg
LV UUID                vFQxWC-g7xw-Ilc0-Eixj-51Fv-HaXV-t3uAuq
LV Write Access        read/write
LV Creation host, time MICROSERVER, 2015-07-24 20:55:58 +0100
LV Status              available
# open                 2
LV Size                7.96 GiB
Current LE             2039
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           252:1

--- Logical volume ---
LV Path                /dev/MICROSERVER-vg/music
LV Name                music
VG Name                MICROSERVER-vg
LV UUID                AA9x7O-ZBRd-uBr4-6hPO-jMeS-3ien-r6zvew
LV Write Access        read/write
LV Creation host, time MICROSERVER, 2015-07-25 10:19:33 +0100
LV Status              available
# open                 1
LV Size                30.00 GiB
Current LE             7680
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           252:2

--- Logical volume ---
LV Path                /dev/MICROSERVER-vg/backup
LV Name                backup
VG Name                MICROSERVER-vg
LV UUID                o6tyxv-eTik-OcpX-KhdE-rI4h-z4Rq-tG2NUS
LV Write Access        read/write
LV Creation host, time MICROSERVER, 2015-07-25 20:58:27 +0100
LV Status              available
# open                 1
LV Size                50.00 GiB
Current LE             12800
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           252:3

To add some more context, I am moving the data using beets from /music to /media/music. The original music library size was ~370GB.

  • What kinds of files are on the disk other than music files? find /media/ubuntu/62c278e1-4c1e-470a-b68c-f0a5d777c165 \! \( -name \*.mp3 -o \*.ogg <other music-file extension tests> \) – cxw Aug 3 '15 at 9:47
  • that does not look like 700 GB – coteyr Aug 3 '15 at 9:49
  • I may be totally wrong, but that looks like 30G, Even if I am wrong it would be 300G. No matter how I do it I don't get 700G – coteyr Aug 3 '15 at 9:53
  • Can you run dh -h and paste the output. I don't have dumep2fs installed and I can't figure out why your BS * BC doesn't equal 700Gigs. – coteyr Aug 3 '15 at 9:56
  • Please post the output of lsblk -b, mount, and sudo lvdisplay. – Vincent Yu Aug 3 '15 at 11:17
1

While I have never used beets, I can give you some general info. First you are correct, in one regard at least. Generally speaking 300 GB of data is 300 GB of data. The file system should not matter too much. There are a few things you can check to make sure things are going smoothly.

First is free inodes. When you run df -h you should get free space in a nice format. If you use df -i you get inode stats. It is possible (but not likely) that you are running out of inodes and not actual space. This can happen with many small files.

Second, and this is more likely, embedded album art can really take up that much space. It has to be embedded in each MP3. So a album with 15 songs and album art with front, back, and insert images, each at 700k just added 31.5 Megs.

Third make sure beets is not copying, but moving. It beets tries to copy from /music to /music/processed and /music is the mount point, then you will easily more then double your space. This is true even if it moves from /old_music to /music then as part of processing copies from /music to /music/processed.

  • Thanks for your insight, fair point about album art, I might have to consider that! Updated the question with the df outputs. – Chris Finlayson Aug 3 '15 at 10:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.