Tag Info

Hot answers tagged

41

It is not just GNU sort that has it. BSD sort has it too. And as to why? (I thought it was a good question too...) From the man page: "The argument given is the name of an output file to be used instead of the standard output. This file can be the same as one of the input files." You can't go to the same file with redirection, the output redirection ...


33

You can use du -sh * | sort -h. This tells sort that the input is the human-readable format. This feature was added recently to GNU Core Utilities 7.5 in Aug 2009, so many distributions do not yet have it.


24

That's the last resort comparison. When comparing two lines, if all the keys compare equal, then as a last resort, a basic string comparison of the whole lines is performed (-r still applies but not the other options). That behavior is specified by POSIX: Except when the -u option is specified, lines that otherwise compare equal shall be ordered as if ...


22

sort | uniq existed before sort -u, and is compatible with a wider range of systems, although almost all modern systems do support -u -- it's POSIX. It's mostly a throwback to the days when sort -u didn't exist (and people don't tend to change their methods if the way that they know continues to work, just look at ifconfig vs. ip adoption). The two were ...


16

sort has the -o, --output option that take a filename as argument. If it is the same as the input files, it write the result to a temporary file, then overwrite the original input file (somewhat as sed do). From GNU sort info page: `-o OUTPUT-FILE' `--output=OUTPUT-FILE' Write output to OUTPUT-FILE instead of standard output. Normally, `sort' ...


16

This is very similar to this question. The trouble is that you have an alphanumeric field that you are sorting on, and -n doesn't treat that sensibly, however version sort (-V) does. Thus use sort -V


14

You can keep the header at the top like this with bash: command | (read -r; printf "%s\n" "$REPLY"; sort) Or do it with perl: command | perl -e 'print scalar (<>); print sort { ... } <>'


14

You're looking for uniq -c If the output of that is not to your liking, it can be parsed and reformatted readily. For example: $ uniq -c logfile.txt | awk '{print $2": "$1}' 27.33.65.2: 2 58.161.137.7: 1 121.50.198.5: 1 184.173.187.1: 3


13

Stealing Andy's idea and making it a function so it's easier to use: # print the header (the first line of input) # and then run the specified command on the body (the rest of the input) # use it in a pipeline, e.g. ps | body grep somepattern body() { IFS= read -r header printf '%s\n' "$header" "$@" } Now I can do: $ ps -o pid,comm | body ...


12

Try using the -k flag to count 1K blocks intead of using human-readable. Then, you have a common unit and can easily do a numeric sort. du -ck | sort -n You don't explictly require human units, but if you did, then there are a bunch of ways to do it. Many seem to use the 1K block technique above, and then make a second call to du. ...


12

Some versions of sort have a -z option, which allows for null-terminated records. find folder1 folder2 -name "*.txt" -print0 | sort -z | xargs -r0 myCommand Additionally, you could also write a high-level script to do it: find folder1 folder2 -name "*.txt" -print0 | python -c 'import sys; sys.stdout.write("\0".join(sorted(sys.stdin.read().split("\0"))))' ...


12

One thing that hasn't been noted so far is that uniq compares whole line lexically, while sort's -u compares based on the sort specification given on the command line. $ printf '%s\n' 'a b' 'a c' | sort -uk1,1 a b $ printf '%s\n' 'a b' 'a c' | sort -k1,1 | uniq a b a c $ printf '%s\n' 0 -0 +0 00 '' | sort -n | uniq 0 -0 +0 00 $ printf '%s\n' 0 -0 +0 00 '' ...


11

Sorting the history This command works like sort|uniq, but keeps the lines in place nl|sort -k 2|uniq -f 1|sort -n|cut -f 2 Basically, prepends to each line its number. After sort|uniq-ing, all lines are sorted back according to their original order (using the line number field) and the line number field is removed from the lines. This solution has ...


10

I got the same error with Ubuntu 11.04, with sort and join both in version (GNU coreutils) 8.5. They are clearly incompatible. In fact the sort command seems bugged: there is no difference with or without the -f (--ignore-case) option. When sorting, aaB is always before aBa. Non alphanumeric characters seems also always ignored (abc is before ab-x) Join ...


10

If you have GNU utilities (or at least a set that can deal with zero-terminated lines) available, another answer has a great method: find . -maxdepth 1 -print0 | sort -z | uniq -diz Note: the output will have zero-terminated strings; the tool you use to further process it should be able to handle that. In the absence of tools that deal with ...


10

Try to pass the -n and -k2 command line options to sort. I.e., find . -maxdepth 1 -type f -iname "*.flac" | sort -n -k2 When I put your unsorted filenames into file 'data.txt' and run this command: sort -k2 -n data.txt I get this as output: ./Track 1.flac ./Track 2.flac ./Track 3.flac ./Track 9.flac ./Track 10.flac ./Track 11.flac explanation of ...


10

You can use a temporary sentinel character to delimit the number: $ sed 's/\([0-9]\+\)/;\1/' log | sort -n -t\; -k2,2 | tr -d ';' Here, the sentinel character is ';' - it must not be part of any filename you want to sort - but you can exchange the ';' with any character you like. You have to change the sed, sort and tr part then accordingly. The pipe ...


10

There's no need to use cat in this case: sort /home/emerg/Wedbackup.txt The problem with your example is that your file is being passed as the command line to sort, which is not what you want. For example, if this was your file: foo bar baz qux wibble wobble The arguments would look like this: sort foo bar baz qux wibble wobble This is not what you ...


10

You could use awk to add an initial serial number that changes every four lines and use that as the primary sorting column awk '{print int((NR-1)/4), $0}' file.txt | sort -n -k1,1 -k2,2 | cut -f2- -d' '


9

My shortest method uses zsh: print -rl **/*(.Om) If you have GNU find, make it print the file modification times and sort by that. I assume there are no newlines in file names. find . -type f -printf '%T@ %p\n' | sort -k 1 -n | sed 's/^[^ ]* //' If you have Perl (again, assuming no newlines in file names): find . -type f -print | perl -l -ne ' ...


9

Sorting depends on the locale; specifically, it depends on $LC_COLLATE (possibly overridden by $LC_ALL), falling back to $LANG if it doesn't exist. The command locale will show you what values you're effectively working with. See man 3 strcoll, man 3 setlocale, etc. LC_COLLATE=C (or POSIX or no locale at all) results in a strict byte-by-byte comparison. ...


9

The '-o' option was already in the the sort of the Sixth Edition of Unix However I agree with you that it is not within the Unix philosophy. uniq did not have that option (and sort did not have a -u then). On my PDP-11 I used a small program that would take one parameter: renac whatever If whatever already existed, it would write everything from stdin ...


8

You need to sort by the last field (considering / as a field separator). Unfortunately, I can't think of a tool that can do this when the number of fields varies (if only sort -k could take negative values). To get around this, you'll have to do a decorate-sort-undecorate. That is, take the filename and put it at the beginning followed by a field separator, ...


8

A bit hacky, but it ought to work: awk '{print substr($(NF-1), 6), $0}' file | sort -g | cut -d' ' -f2- It duplicates the second-to-last column at the front (dropping the days=), sorts numerically, and then removes the first column (the duplicated second-to-last one).


8

Assuming you're on Linux, the output of atq always has the date in the same format. Sort the fields in the appropriate order, taking care to declare which ones are numbers or month names. Make sure to use an English locale for the month names since that's what atq uses. atq | sort -k 6n -k 3M -k 4n -k 5 -k 7 -k 1 # year month day time queue id


8

you could use the join command to join the files on a specific column, and awk to parse the output. To join these files on column 1 pass the parameters -j 1 to the join command: usr@srv % join -j 1 test test2 12 1 13 2 14 2 15 6 Afterwards use awk to print only the second column: usr@srv % join -j 1 test test2 | awk '{print $2}' 1 2 2 6


8

If you tell GNU sort to split the fields by a different character, a dash - in your case it's pretty easy to sort this: $ sort -n -t"-" -k1 -k2M -k3 file.txt Example $ sort -n -t"-" -k1 -k2M -k3 file.txt 2013-May-20 21 2013-May-21 10 2013-May-30 2 2013-Jun-01 2 2013-Jun-09 17 2013-Jun-10 1 2013-Jun-18 9 2013-Jun-27 8 ...


8

GNU sort (which is the default on most Linux systems), has a --parallel option. From http://www.gnu.org/software/coreutils/manual/html_node/sort-invocation.html: ā€˜--parallel=nā€™ Set the number of sorts run in parallel to n. By default, n is set to the number of available processors, but limited to 8, as there are diminishing performance gains after ...


7

With GNU ls (i.e. on Linux, Cygwin, or other systems that have GNU ls specifically installed): ls -v In zsh: echo *(on) In other shells: echo log?.gz log??.gz log???.gz Replace echo by printf '%s\n' if you want each file name on a separate line. If you want file metadata as well (ls -l) and you don't have GNU ls, you'll need to call ls separately ...


7

for dir in */*; do # loop over the directories ( # run in a subshell ... cd "$dir" # ... so we don't have to cd back files=(*) # store the filenames in a zero-indexed array for index in "${!files[@]}"; do file=${files[$index]} ext=${file##*.}" ...



Only top voted, non community-wiki answers of a minimum length are eligible