How to tar files from multiple paths without including path names?

Question

At the end of an automated job, I want to tar up some logfiles and deposit them somewhere in case they need to be analyzed. The log files are at multiple different locations on the host, and some of them can be quite large.

Suppose the following files exist and I want all of them:

/app/app_stderr.log
/app/app_stdout.log
/var/log/service/service.log
/var/lib/service/service_backend.log
/var/log/messages.log

However I don't want any paths in the tarfile. I just want the five files as-is:

app_stderr.log
app_stdout.log
service.log
service_backend.log
messages.log

I also want the tar archive gzip compressed. Ideally I'd like to do this in the most efficient way possible - i.e. not having to make copies of the logfiles or store an uncompressed tarfile temporarily before gzipping it.

The -C option that is often given as a solution to similar questions is not applicable to me since I want to tar files from multiple directories in different parts of the filesystem while still stripping off the path names.

If I just run:

tar -czf /drop/run_logs.tar.gz /app/app_*.log /var/log/service/*.log /var/lib/service/*.log /var/log/messages.log

then I'm going to get a tarfile with the paths included.

Possible to accomplish stripping the paths?

Answer 1

The tar --transform option allow you to modify the filename before writing to the tar file. This option will run the filenames through a sed expression before they get written to the tar file. See here for the full details.

Here is a quick example.

First create a few files to add to a tar file

$ mkdir test-tar
$ cd test-tar

$ mkdir -p alpha/beta/gamma

$ touch alpha/one.log
$ touch alpha/beta/two.log
$ touch alpha/beta/gamma/three.log

use a standard tar to reproduce the tar file with the full relative paths.

$ tar cf test.tar alpha/*.log alpha/beta/*.log alpha/beta/gamma/*.log
$ tar tf test.tar
alpha/one.log
alpha/beta/two.log
alpha/beta/gamma/three.log

Now redo the command but using the --transform option. In the example I'm using the sed expression s#.*/## to delete all leading characters in the filename up to the rightmost /.

$ tar cf test-transform.tar alpha/*.log alpha/beta/*.log alpha/beta/gamma/*.log --transform='s#.*/##'

now see what was written to the tar file

$ tar tf test-transform.tar 
one.log
two.log
three.log

no leading paths present.