If speed is important and compression is not needed, you can hook the syscall wrappers used by
LD_PRELOAD, to change
tar to calculate it for us. By reimplementing a few of these functions to suit our needs (calculating the size of potential output tar data), we are able eliminate a lot of
write that is performed in normal operation of
tar. This makes
tar much faster as it doesn't need to context switch back and forth into the kernel anywhere near as much and only the
stat of the requested input file/folder(s) needs to be read from disk instead of the actual file data.
The code below includes implementations of the
write POSIX functions. The macro
OUT_FD controls which file descriptor we expect
tar to use as the output file. Currently it is set to stdout.
read was changed to just return the success value of
count bytes instead of filling buf with the data, given that the actual data wasn't read buf would not contain valid data for passing on to compression, and thus if compression was used we would calculate an incorrect size.
write was changed to sum the input
count bytes into the global variable
total and return the success value of
count bytes only if file descriptor matches
OUT_FD, otherwise it calls the original wrapper acquired via
dlsym to perform the syscall of the same name.
close still preforms all of its original functionality, but if the file descriptor matches OUT_FD, it knows that
tar is done attempting to write a tar file, so the
total number is final and it prints it to stdout.
#define OUT_FD 1
uint64_t total = 0;
ssize_t (*original_write)(int, const void *, size_t) = NULL;
int (*original_close)(int) = NULL;
printf("%" PRIu64 "\n", total);
int close(int fd)
original_close = dlsym(RTLD_NEXT, "close");
if(fd == OUT_FD)
ssize_t read(int fd, void *buf, size_t count)
ssize_t write(int fd, const void *buf, size_t count)
original_write = dlsym(RTLD_NEXT, "write");
if(fd == OUT_FD)
total += count;
return original_write(fd, buf, count);
Benchmark comparing a solution where the read disk access and all the syscalls of normal tar operation is performed against the
$ time tar -c /media/storage/music/Macintosh\ Plus-\ Floral\ Shoppe\ \(2011\)\ \[Flac\]/ | wc -c
tarsize$ time ./tarsize.sh -c /media/storage/music/Macintosh\ Plus-\ Floral\ Shoppe\ \(2011\)\ \[Flac\]/
The code above, a basic build script to build the above as a shared library, and a script with the "
LD_PRELOAD technique" using it is provided in the repo:
Some info on using LD_PRELOAD: https://rafalcieslak.wordpress.com/2013/04/02/dynamic-linker-tricks-using-ld_preload-to-cheat-inject-features-and-investigate-programs/