1

I have many, many files of the form [a-zA-Z]+\d\.\d{2}\_\d.\d{4}.end and want to find the file(s) with the smallest 4 digit number right before .end. (in case of collision I want all files)

How can you do that using Bash? sort alone doesn't work, obviously, due to lexicographic ordering.

1

Using Bash, find and the sort -t|--field-separator and -k|--key options:

find_files.sh:

#!/bin/bash
first_file=$(find . -iname '*.end' | sort -t '.' -k 4.1 | head -1)
IFS='.'
fields=($first_file)
unset IFS
find . -iname "*${fields[3]}.end"

The -k|--key options expects a KEYDEF of the form field_number.character_number. When -t|--field-separator is in effect, the field numbers are calculated based on the specified separator (in this case .) rather than a whitespace.

We then extract the wanted pattern from the first file using Bash's input-field separator, and search the directory with find again for files matching that pattern in case several files share the same 4-digit number.

Example:

$ ls -1
abc0.03_1.1921.end
def0.03_9.0311.end
ghi0.03_1.1966.end
jkl1.04_1.1916.end
mno2.04_4.9540.end
pab9.04_1.1994.end
uvx7.04_3.2002.end
yyy1.05_8.0311.end
zzz4.04_1.2097.end
$ ./find_files.sh
./yyy1.05_8.0311.end
./def0.03_9.0311.end
2

With GNU tools, you could do something like:

find . -regextype posix-extended \
  -regex '.*/[a-zA-Z]+[0-9]\.[0-9]{2}_[0-9]\.[0-9]{4}\.end' -print0 |
  awk -v RS='\0' -F . '
    NR == 1 || $(NF-1) < min {files=$0; min = $(NF-1); next}
    $(NF-1) == min {files = files "\n" $0}
    END {if (NR) print files}'

That's an example of a common pattern: we print the list of matching files NUL-delimited (as NUL is the only character that can't occur in a file path) and process that output with awk where the record separator has been set to NUL (not all awk implementations support that though).

The field separator is set to .. NF is the number of fields, so $(NF-1) is the last before last field. awk finds the lowest number by comparing that with the last known value, and stores the corresponding files in the files awk variable.

As we store the list newline-separated as opposed to NUL separated, that's only intended for user output. If you wanted to post-process it reliably, you'd want to use NUL ("\0" in awk) instead.

1

sort has -t and -k, which would allow you to sort based on one of the dot-separated parts of the file name, which should do it in this case.

-t, --field-separator=SEP
    use SEP instead of non-blank to blank transition

-k, --key=KEYDEF
    sort via a key; KEYDEF gives location and type

(quote from the man page of GNU sort.)

  • Note that while -k/-t are standard, the long options are GNU extensions (now also supported by the sort of some BSDs though for compatibility with GNU sort). Using sort -t . -k ... can only be used here if those .end files are located in directories that don't contain dot or newline characters. – Stéphane Chazelas Mar 16 '17 at 13:07
  • Yah, I forgot to add the link to the quote. And yes, I assumed the simple case where the files are in a single list/directory. – ilkkachu Mar 16 '17 at 13:29
0
perl -le '
   for ( grep { /^[a-zA-Z]+\d\.\d{2}\_\d.\d{4}\.end$/ } <*.end> ) {
      ($n) = /(\d{4})\.end$/;
      $min //= $n;
      $n <= $min and $min = $n, push @{$h{$min}}, $_;
   }
   print for @{$h{$min}};
'

<*> collects all the relevant files using the globbing syntax, out of which, the proper syntax files shall be selected by the for.

Inside the for we first extract the last 4 digits to determine & compare it against the running minimum and if found , pushed onto a hash.

At the end, we simply print the contents of the hash with the key $min.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.