If they are on the same filesystem, you could hard-link those to a common directory and tar that directory.
Alternatively, if you are using GNU tar, you could (with a little more flexibility) soft-link those to a common directory and using its -h
option, tar the files which the soft-links point to.
The manual page for the latter shows:
-h
, --dereference
follow symlinks; archive and dump the files they point to
FreeBSD tar supports equivalent options, but names them like pax
(see below).
There is of course no POSIX tar to use for comparison. If you can use pax
, it has a similar -L
option:
-L
If a symbolic link referencing a file of type directory is specified on the command line or encountered during the traversal of a file hierarchy, pax
shall archive the file hierarchy rooted in the file referenced by the link, using the name of the link as the root of the file hierarchy. Otherwise, if a symbolic link referencing a file of any other file type which pax
can normally archive is specified on the command line or encountered during the traversal of a file hierarchy, pax
shall archive the file referenced by the link, using the name of the link. The default behavior, when neither -H
or -L
are specified, shall be to archive the symbolic link itself.
Whether hard-linking, or soft-linking, the result is that you do not have to move your existing files. Hard-linking changes the ctime
(timestamp) of your files, while soft-linking does not. But soft-linking (though it reduces the necessity of being on the same filesystem) is not universally supported by tar implementations using identical options.
OP's original statement appeared to indicate that the reason why it was inconvenient to put all files together in one directory was because of some limitation by the application which generates them. Clarification makes it apparent that the problem is that they all have the same filename. While linking into one directory (and for instance encoding their original directory name into the common location) is certainly doable, there are other options for simply collecting the files into a single archive, retaining their existing directory names. One drawback to doing this is that it limits the ability to restore into arbitrary locations. However: the simplest way of passing some ~20 names to tar would be on the command-line, e.g.,
`tar czf myoutput.tar.gz $(find . -type f -name bigfile.dat)`
For only 20 files that is not likely to be a problem with command-line length. If it were a large number of files (or very long pathnames), that would make it harder, since tar
implementations as a rule lack the ability to pass a list of pathnames other than as separate command-line parameters (see Solaris for instance). For those, one might try to work around the limitation by creating an archive in steps — but then that would not work with compression. Some implementations provide options for reading a list of filenames from a file. The GNU tar -T
(also FreeBSD) option does this. Other programs may provide a third alternative, reading the list of filenames from the standard input (as done by pax, which was influenced by cpio), but generally tar does not: it may read the file content from standard input.
index1_index2/subfolder/bigfile.dat
, orsubfolder/bigfile.dat
, orbigfile.dat
?bigfile.dat
? Are there other files that are bigger? I would try it withfind options | xargs tar czf bigtar
. The options depend on your answer.tar
, which would create one file containing only as many files as the last timetar
was run. Perhaps not what was anticipated.xargs
results in exact one invocation oftar
.tar c
creates a new archive or overwrites an existing one, no matter what happended on an earlier run. Have you overlooked thatxargs
?