How to pin certain files when using sort?

Question

Normally I am more creative with Unix commands and pipes, but not today.

So what I have is this:

collage.jpg
DCIM001.jpg
DCIM002.jpg

or this:

001.jpg
002.jpg
boston-collage.jpg

or sometimes

panama-collage.jpg
DCIM001.jpg
DCIM002.jpg

So now I use this command, which sorts the images properly.

find "Image Folder" -print0 | sort -z | tar cv --no-recursion --null -T -

But I would like to fix files with *collage.* as the first files of the sorting results to add them first to the tar archive I am creating off that list. Any idea?

Answer 1

If the end goal is to put the files in the archive in the desired order, then a simple solution is to create an archive with the files to sort first, then add the files to sort last to the existing archive. This can of course be generalized to more than two sort segments. This requires creating an archive file, you can't append to an archive in a pipeline.

tar cf foo.tar /images/*collage.*
find /images ! -name '*collage.*' -print0 | sort -z | tar rf foo.tar --null -T -

If you want to perform a custom sort with just basic utilities, one possible method is to add a prefix to the line indicating which group the item is a part of. Arrange for the prefix tags to be sorted in the order you want the groups to be sorted, and strip those prefixes afterwards.

find /images -print0 |
sed -z -e 's/.*collage\./1&/' -e 't' -e 's/^/2/' |
sort -z |
sed -z 's/^.//' |
tar …

Another approach would be to do the sorting in another language such as Perl, Python or Ruby where you can express a custom sort. If you do that, you could do the filename gathering and even the archive production in that language. Here's a Perl example that just does the sorting:

perl -e '$,=$\="\0"; print sort {
    $a =~ /collage\./ ? $b =~ /collage\./ ? $a cmp $b : 1 :
                        $b =~ /collage\./ ? -1 : $a cmp $b
  } @ARGV' /images/* | tar …

And another example that does the sorting via a temporary rewrite (it's called “Schwarztian transform” in the Perl community):

perl -e '$,=$\="\0"; print
    map {substr($_,1)}
    sort
    map {$_ = (/collage\./ ? "1" : "2") . $_}
    @ARGV' /images/* | tar …

Both of these examples will fail if the combined length of the file names exceeds the command line length limit. To avoid that, let Perl generate the file names.

perl -e '$,=$\="\0"; print
    map {substr($_,1)}
    sort
    map {$_ = (/collage\./ ? "1" : "2") . $_}
    glob("/images/*")' /images/* | tar …

If you need to generate file names recursively or apply some filtering, you can use File::Find. If you want to generate the archive in Perl, you can use Archive::Tar.

Answer 2

Here's my suggestion, using a temporary file:

tmpdir=$(mktemp)
find "Image Folder" -print0  > $tmpdir/temp
{ grep -z collage $tempdir/temp | sort -z; 
  grep -z -v collage $tempdir/tmp | sort -z; } | \
  tar cv --no-recursion --null -T -
rm -rf $tmpdir

You could use something simpler for the temporary file, as long as you don't put it inside "Image Folder".

The basic idea behind this solution is to save find's results to a temporary file so that we can run two greps against it, sorting each grep result individually before combining them into the pipe for tar.