6

I'm writing a Bash script to test USB flash memory for errors using f3 tool. I have text like this (created by grepping logs from f3read program):

2017-10-25_09:30:22/sdf.log:Data LOST: 4.00 KB (8 sectors)
2017-10-25_09:30:22/sdi.log:Data LOST: 5.00 KB (10 sectors)
2017-10-25_09:30:22/sdj.log:Data LOST: 2.35 MB (4822 sectors)
2017-10-25_09:30:22/sdn.log:Data LOST: 5.00 KB (10 sectors)
2017-10-25_09:30:22/sdo.log:Data LOST: 4.00 KB (8 sectors)
2017-10-25_09:30:22/sdp.log:Data LOST: 4.00 KB (8 sectors)
2017-10-25_09:30:22/sdq.log:Data LOST: 2.00 KB (4 sectors)
2017-10-25_14:37:03/sdb.log:Data LOST: 5.00 KB (10 sectors)
2017-10-25_14:37:03/sdc.log:Data LOST: 3.00 KB (6 sectors)
2017-10-26_09:17:59/sdd.log:Data LOST: 3.00 KB (6 sectors)
2017-10-26_09:17:59/sde.log:Data LOST: 2.00 KB (4 sectors)
2017-10-26_09:17:59/sdf.log:Data LOST: 3.00 KB (6 sectors)
2017-10-26_09:17:59/sdg.log:Data LOST: 6.00 KB (12 sectors)
2017-10-26_09:17:59/sdh.log:Data LOST: 611.29 MB (1251918 sectors)
2017-10-26_09:17:59/sdi.log:Data LOST: 6.00 KB (12 sectors)
2017-10-26_09:17:59/sdl.log:Data LOST: 6.00 KB (12 sectors)
2017-10-26_09:17:59/sdo.log:Data LOST: 3.00 KB (6 sectors)
2017-10-26_09:17:59/sdp.log:Data LOST: 2.00 KB (4 sectors)
2017-10-26_09:17:59/sdq.log:Data LOST: 414.60 MB (849106 sectors)
2017-10-26_09:17:59/sdr.log:Data LOST: 65.29 MB (133712 sectors)
2017-10-26_09:17:59/sds.log:Data LOST: 5.00 KB (10 sectors)

I would like to sort the lines by the number of bad sectors noted at the end of the line. I tried using sort, but I don't know how to use it's --key option to make it do what I want.

I cannot cut the lines first, becasue I need the drive name (sda, sdb, etc) to be then extracted for a report.

8

Use sort -V if that option is available

-V, --version-sort

natural sort of (version) numbers within text

$ <cmd> | sort -k5,5V
2017-10-25_09:30:22/sdq.log:Data LOST: 2.00 KB (4 sectors)
2017-10-26_09:17:59/sde.log:Data LOST: 2.00 KB (4 sectors)
2017-10-26_09:17:59/sdp.log:Data LOST: 2.00 KB (4 sectors)
2017-10-25_14:37:03/sdc.log:Data LOST: 3.00 KB (6 sectors)
2017-10-26_09:17:59/sdd.log:Data LOST: 3.00 KB (6 sectors)
2017-10-26_09:17:59/sdf.log:Data LOST: 3.00 KB (6 sectors)
2017-10-26_09:17:59/sdo.log:Data LOST: 3.00 KB (6 sectors)
2017-10-25_09:30:22/sdf.log:Data LOST: 4.00 KB (8 sectors)
2017-10-25_09:30:22/sdo.log:Data LOST: 4.00 KB (8 sectors)
2017-10-25_09:30:22/sdp.log:Data LOST: 4.00 KB (8 sectors)
2017-10-25_09:30:22/sdi.log:Data LOST: 5.00 KB (10 sectors)
2017-10-25_09:30:22/sdn.log:Data LOST: 5.00 KB (10 sectors)
2017-10-25_14:37:03/sdb.log:Data LOST: 5.00 KB (10 sectors)
2017-10-26_09:17:59/sds.log:Data LOST: 5.00 KB (10 sectors)
2017-10-26_09:17:59/sdg.log:Data LOST: 6.00 KB (12 sectors)
2017-10-26_09:17:59/sdi.log:Data LOST: 6.00 KB (12 sectors)
2017-10-26_09:17:59/sdl.log:Data LOST: 6.00 KB (12 sectors)
2017-10-25_09:30:22/sdj.log:Data LOST: 2.35 MB (4822 sectors)
2017-10-26_09:17:59/sdr.log:Data LOST: 65.29 MB (133712 sectors)
2017-10-26_09:17:59/sdq.log:Data LOST: 414.60 MB (849106 sectors)
2017-10-26_09:17:59/sdh.log:Data LOST: 611.29 MB (1251918 sectors)
10

POSIXly, you can do:

sort -t '(' -k2n < file

That is set the field separator to ( and sort on the second field (or rather the portion of the line starting with the second field) numerically.

Alternatively, you could keep the default field separator (the transition from a non-blank to a blank) where the 5th field would be like " (12" and use:

sort -k5.3n < file

(that is, sort numerically on the portion of the line starting with the 3rd character of the 5th field).

For ties, the last-resort sorting order comes into play, and that's lexical comparison of the full line (which conveniently here will give you a chronological order).

If you wanted to sort ties on drive name instead, you could use:

sort -t '(' -k2n -k1.21

(second key being the portion of line starting with the 21st character, lexical comparison)

3

I've found it.

sort -n -t '(' -k2V

-n tells sort to read numeric values in the strings

-t '(' tells it to use ( character as field delimiter. Becasue the word sectors) is always the same later on it won't affect the sorting order.

-k2V defines a custom key, using the second column - text after first ( character - for sorting.

  • Note that you don't need the (GNU-specific) V flag here (which by the way makes -n redundant). That would be useful if the key didn't start with a digit like in @sundeep's approach where the key starts with " (". V is for version sort where v3.4 sorts before v3.20 (but after u5 or V5) which are not numbers and would be considered as 0 with the n flag. Same for " (2" vs " (10". – Stéphane Chazelas Nov 16 '17 at 8:52
3

If your standard sort version does not offer option -V, then you might be able to use an alternate sorting program msort. It allows the use of a hybrid option:
Hybrid comparison is like lexicographic comparison except for the fact that strings of digits are treated as numbers.

So a snippet like this:

# Utility functions: print-as-echo, print-line-with-visual-space.
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
pl " Input data file $FILE:"
tail -v $FILE

pl " Expected output:"
tail -v $E

pl " Results:"
msort -q -j -l -n 5,5 -c hybrid $FILE |
tail -v

produces this:

-----
 Input data file data1:
==> data1 <==
2017-10-26_09:17:59/sdf.log:Data LOST: 3.00 KB (6 sectors)
2017-10-26_09:17:59/sdg.log:Data LOST: 6.00 KB (12 sectors)
2017-10-26_09:17:59/sdh.log:Data LOST: 611.29 MB (1251918 sectors)
2017-10-26_09:17:59/sdi.log:Data LOST: 6.00 KB (12 sectors)
2017-10-26_09:17:59/sdl.log:Data LOST: 6.00 KB (12 sectors)
2017-10-26_09:17:59/sdo.log:Data LOST: 3.00 KB (6 sectors)
2017-10-26_09:17:59/sdp.log:Data LOST: 2.00 KB (4 sectors)
2017-10-26_09:17:59/sdq.log:Data LOST: 414.60 MB (849106 sectors)
2017-10-26_09:17:59/sdr.log:Data LOST: 65.29 MB (133712 sectors)
2017-10-26_09:17:59/sds.log:Data LOST: 5.00 KB (10 sectors)

-----
 Expected output:
==> expected-output1 <==
2017-10-25_09:30:22/sdn.log:Data LOST: 5.00 KB (10 sectors)
2017-10-25_14:37:03/sdb.log:Data LOST: 5.00 KB (10 sectors)
2017-10-26_09:17:59/sds.log:Data LOST: 5.00 KB (10 sectors)
2017-10-26_09:17:59/sdg.log:Data LOST: 6.00 KB (12 sectors)
2017-10-26_09:17:59/sdi.log:Data LOST: 6.00 KB (12 sectors)
2017-10-26_09:17:59/sdl.log:Data LOST: 6.00 KB (12 sectors)
2017-10-25_09:30:22/sdj.log:Data LOST: 2.35 MB (4822 sectors)
2017-10-26_09:17:59/sdr.log:Data LOST: 65.29 MB (133712 sectors)
2017-10-26_09:17:59/sdq.log:Data LOST: 414.60 MB (849106 sectors)
2017-10-26_09:17:59/sdh.log:Data LOST: 611.29 MB (1251918 sectors)

-----
 Results:
==> standard input <==
2017-10-25_09:30:22/sdi.log:Data LOST: 5.00 KB (10 sectors)
2017-10-25_09:30:22/sdn.log:Data LOST: 5.00 KB (10 sectors)
2017-10-25_14:37:03/sdb.log:Data LOST: 5.00 KB (10 sectors)
2017-10-26_09:17:59/sdi.log:Data LOST: 6.00 KB (12 sectors)
2017-10-26_09:17:59/sdl.log:Data LOST: 6.00 KB (12 sectors)
2017-10-26_09:17:59/sdg.log:Data LOST: 6.00 KB (12 sectors)
2017-10-25_09:30:22/sdj.log:Data LOST: 2.35 MB (4822 sectors)
2017-10-26_09:17:59/sdr.log:Data LOST: 65.29 MB (133712 sectors)
2017-10-26_09:17:59/sdq.log:Data LOST: 414.60 MB (849106 sectors)
2017-10-26_09:17:59/sdh.log:Data LOST: 611.29 MB (1251918 sectors)

Some details for msort:

msort   sort records in complex ways (man)
Path    : /usr/bin/msort
Version : 8.53
Type    : ELF 64-bit LSB executable, x86-64, version 1 (SYS ...)
Help    : probably available with -h,--help
Repo    : Debian 8.9 (jessie) 
Home    : http://www.billposer.org/Software/msort.html (pm)

On a system like:

OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution        : Debian 8.9 (jessie) 
bash GNU bash 4.3.30

Best wishes ... cheers, drl

  • I managed to do it with sort, even without the -V option, and msort is not by default installed on Linux Mint 18.2. Thanks anyway! – unfa Nov 15 '17 at 13:22
  • Hi. OK, good to see that you solved it, but keep msort in mind for tasks that sort might not be able to handle, and it's available at least on systems like Linux Mint 17.1 Rebecca ... cheers – drl Nov 15 '17 at 13:29
3

A slightly more flexible solution extracts the sector count, duplicates it at the beginning of the line, sorts, and finally removes the duplicated part again:

sed 's/^.*(\([0-9]\{1,\} \)sectors)$/\1&/' |
  sort -n |
  cut -d ' ' -f 2-

This solution is more flexible because it works for arbitrary patterns that identify the sort key in each record.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.