Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

At the moment I am using ArchiveMount to mount a 123,000 kb archive that contains more then 3 million files inside. So far it has been mounting for 5+ hours and still isn't finished.

Is there a better way to mount a .tar.gz file? I am trying to mount to a folder, and uncompressed it takes a few gigs. I don't even need write mode, just read-only is sufficient.

share|improve this question
    
There's also AVFS; I have no idea if it'll perform better. –  Gilles Nov 6 '11 at 16:25
5  
If your files were compressed as a squashfs module instead of as a tarball, then read-only access would be very quick - you just (loop) mount the squashfs module. Requires the squashfs-tools package. –  dru8274 Nov 7 '11 at 6:05
add comment

2 Answers

The problem here is with the format, the TAR (Tape ARchive) format is designed for sequential access, not random access. And gzip is a good complement to tar, since it is a stream based compression format, also not for random access.

So a high level tool that does not interact with the compressed blocks directly, will have to parse through the entire file every time it needs to read anything, first to get you the list of files, then perhaps the cache invalidates and it reads it again, and then for each file you copy off it might read through it again. You can make a tool that remembers the position of each file, and what blocks it needs to decompress to get it, but it seems that few have bothered with that.

If you want this to go faster, do a tar tzf file.tar.gz > filelist, open that file list in vim, gedit or whatever, remove the lines of files you do not need, save, and then extract them with tar xzf file.tar.gz -T filelist -C extracted/.

To get random access to a compressed file, you should use perhaps zip with posix extensions, rar, or as dru8274 suggested, squashfs, or even ZFS with compression turned on, or btrfs if btrfs has gotten compression to work at the time of reading.

share|improve this answer
add comment

You could also create a compressed squashfs image

mksquashfs /etc squashfs.img -comp xz
mkdir img
mount -o squashfs,ro squashfs.img img

In order to do this you'll need to extract your tar.gz archvie.

The advantage is also that the image has better fault tolerancy than gz.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.