Is it possible to generate a separate file list when creating a tarball?

Question

When using tar for backups I'd like to create a separate list of the files that is included in the archive for storing alongside the backup tarball.

I also want to be able to store the error messages from tar in a log.

Solutions that's not so good:

• tar --verbose ... &> filelist.txt But this would give errors + file list in one file.
• tar --list after the tarball has been created. But this takes a long time.

Is there some way to get the file list and store it in a separate file at the same time as I'm rolling the ball for the first time?

Answer 1

You mention you don't like tar --list because it's slow. I'm guessing this is because it's a large tarball, and it has to re-scan the whole thing. If this is indeed the case, you can get better performance out of this by scanning as it's being created:

tar -c /input/directory | tee output.tar | tar -t > filelist.txt

This uses tee to split the resulting tarball, one going to a file, the other going to tar -t.

If you want to gzip the tarball:

tar -c /input/directory | tee >(gzip > output.tar.gz) | tar -t filelist.txt

We don't have the first tar do the gzip as it'll waste CPU cycles when the second tar has to decompress it again. Instead we only gzip the tarball as it's being written to disk.

Answer 2

You could append each file to the tar file:

for f in file ...; do 
    echo "$f"
    tar rf result.tar $f
done

Answer 3

This would put the error messages (stderr) in one file and file list (stdout) in one file:

$ tar -cvf archive.tar test nonexisting 1> stdout.txt 2> stderr.txt

$ cat stdout.txt
test/
test/05.txt
test/06.txt
test/00.txt
test/01.txt
test/10.txt
test/07.txt
test/03.txt
test/02.txt
test/04.txt
test/08.txt
test/09.txt

$ cat stderr.txt
tar: nonexisting: Funktion "stat" misslyckades: Filen eller katalogen finns inte
tar: Avslutar med felstatus på grund av tidigare fel

Using --index-file as suggested by Johan in the comment to my question would also work good:

tar --create \
    --verbose \
    --index-file=$fileList \
    --file $archiveFile \
    $filesystemToArchive \
    &>> $logFilename

From tar manpages:

--index-file=FILE

send verbose output to FILE