Sorting text files or other data, for example with the sort utility.

learn more… | top users | synonyms

37
votes
9answers
12k views

how do you sort du output by size?

How do you sort du -sh /dir/* by size? I read one site that said use | sort -n but that's obviously not right. Here's an example that is wrong. [~]# du -sh /var/* | sort -n 0 /var/mail 1.2M ...
25
votes
2answers
719 views

Why is sort -o useful?

UNIX philosophy says: do one thing and do it well. Make programs that handle text, because that is a universal interface. The sort command, at least GNU sort, has an -o option to output to a file ...
22
votes
1answer
261 views

Why is sort changing the order of lines with identical sort keys?

Here is the data: D 2 B 2 A 2 When I run this command: sort -k2,2 file it outputs: A 2 B 2 D 2 My question is that when I specify only the second column -k2,2, why is it that it sorts by the ...
17
votes
3answers
4k views

What is the difference between “sort -u” and “sort | uniq”?

Everywhere I see someone needing to get a sorted, unique list, they always pipe to sort | uniq. I've never seen any examples where someone uses sort -u instead. Why not? What's the difference, and why ...
17
votes
5answers
6k views

sort but keep header line in the at the top?

I am getting output from a program that first produces one line that is a bunch of column headers, and then a bunch of lines of data. I want to cut various columns of this output and view it sorted ...
12
votes
6answers
5k views

List files sorted numerically

I have a bunch of files from log1 to log164. I'm trying to LIST the directory (sorted) in a UNIX terminal but the sort functions are only providing the format like this: home:logs Home$ ls -1 | sort ...
10
votes
4answers
7k views

Sorting the output of “find”?

I need to be able to alphabetically sort the output of find before piping it to a command. Entering | sort | between didn't work, so what could I do? find folder1 folder2 -name "*.txt" -print0 | ...
10
votes
4answers
10k views

How to list files sorted by modification date recursively (no stat command available!)

How can I get the list of all files under current directory along with their modification date and sorted by that date? Now I know how to achieve that with find, stat and sort, but for some weird ...
10
votes
5answers
3k views

case-insensitive search of duplicate file-names

I there a way to find all files in a directory with duplicate filenames, regardless of the casing (upper-case and/or lower-case)?
10
votes
4answers
427 views

Shuffle file randomly with some additional constraints

I have a huge music playlist and, while some artists have many albums, others have just one song. I wanted to sort the playlist so the same artist won't play twice in a row, or his songs won't end up ...
10
votes
2answers
2k views

Why does ls sorting ignore non-alphanumeric characters?

When sorting file names, ls ignores characters like -,_. I expected it to use those characters in sorting as well. An example: touch a1 a2 a-1 a-2 a_1 a_2 a.1 a.2 a,1 a,2 Now display these files ...
9
votes
3answers
2k views

How to sort the string which combined with string + numeric using bash script?

This is the data what I want to sort. But sort treats the numeric to string, the data it no sorted as I expected. /home/files/profile1 /home/files/profile10 /home/files/profile11 ...
9
votes
2answers
2k views

sort inplace - like sed --in-place - exists?

Am I blind or is there no option like --in-place for sort? In order to save results to the input file, sed uses -i (--in-place). Redirecting the output of sort to the input file sort < f > f ...
8
votes
3answers
2k views

How can I remove duplicates in my .bash_history, preserving order?

I really enjoying using control+r to recursively search my command history. I've found a few good options I like to use with it: # ignore duplicate commands, ignore commands starting with a space ...
8
votes
5answers
589 views

How intelligent are unix utilities?

Unix utilities like sort, find, grep, diff et al are very handy to perform quick tasks, sometimes without writing any code at all. I wanted to know what algorithms do they internally use and how do ...
8
votes
4answers
3k views

How do I make ls sort underscore characters first?

I like being able to name files and directories with an underscore prefix if it's something I want to keep separate from other files and directories at the same level. On Windows and Mac, for example, ...
6
votes
4answers
497 views

uniq a csv file ignoring a column, awk maybe?

Given this file (annotations are not part of file, but form part of explanation)... x,a,001,b,c,d,y x,a,002,b,c,e,yy x,bb,003,b,d,e,y x,c,004,b,d,e,y x,c,005,b,d,e,y # nb - dupe of row 4 ...
6
votes
2answers
349 views

Counting number of times each IP address appears in log file

I have a file in the format as follows: $ cat file.txt 27.33.65.2 27.33.65.2 58.161.137.7 121.50.198.5 184.173.187.1 184.173.187.1 184.173.187.1 What's the best way to parse the file file.txt into ...
6
votes
2answers
106 views

Sorting with Perl respecting Locale Settings

Following data needs to be sorted respecting the locale sort order: wird sin wär pêche war Über Uber péché peach There was nor problem by using sort $ sort < data peach pêche péché sin Uber ...
6
votes
3answers
527 views

sort at jobs chronologically

Possibly a simple solution I misplaced. How can I get the output of atq sorted chronologically, so I can easily see who's due to run next? The man page for sort doesn't have anything built-in to ...
6
votes
2answers
4k views

Trying to sort on two fields, second then first

I am trying to sort on multiple columns. The results are not as expected. Here's my data (people.txt): Simon Strange 62 Pete Brown 37 Mark Brown 46 Stefan Heinz 52 Tony Bedford 50 John Strange 51 ...
6
votes
1answer
89 views

How to specify additional parameters for GNU sort's --compress-program option?

sort --compress-program=/bin/gzip works fine, but how can I pass options down to the compression programm (e.g. --best or --fast)? sort --compress-program=/bin/gzip\ --best failes with the error ...
6
votes
1answer
66 views

biggest possible number for sort -n

How to determine the "biggest number" so that sort -n will always put it at the end? I'm thinking of something along Inf in some languages but I'm not sure if anything like this exists for sort. ...
6
votes
4answers
370 views

Fastest way to determine if two sorted lists contain unique elements

I have two sorted files A and B where the size of A is much greater than B, e.g. A is 100GB while B is 50MB. I want to quickly determine if there are any lines in B that are contained in A, stopping ...
5
votes
5answers
189 views

Sorting blocks of lines

I have a file that contains 4n lines. Here is an excerpt from it containing 8 lines 6115 8.88443 6116 6.61875 6118 16.5949 6117 19.4129 6116 6.619 6117 16.5979 6118 19.4111 6115 8.88433 What I ...
5
votes
5answers
8k views

join : “File 2 not in sorted order”

I've got two files _jeter3.txt and _jeter1.txt I've checked they are both sorted on the 20th column using sort -c sort -t ' ' -c -k20,20 _jeter3.txt sort -t ' ' -c -k20,20 _jeter1.txt #no ...
5
votes
2answers
1k views

sort output by column

I'd like to take this command find -maxdepth 1 -type d | while read -r dir; do printf "%s:\t" "$dir"; find "$dir" | wc -l; done ( from here ). which has an output of basically ./kennel: 11062 ...
5
votes
2answers
414 views

How would I sort these directory names numerically?

I'm aware that I can somehow sort this output numerically (so cpu1/ follows cpu0/) ... I could probably get something to work eventually by splitting up the string various ways with awk, but is there ...
5
votes
6answers
1k views

How to sort (by whatever key) BibTex entries in `.bib` file?

I work with LaTeX and do versioning with Git. For bibliography management I use Mendeley. The problem is that each time Mendeley synchronizes it's .bib exports, they are in different order, what ...
5
votes
2answers
130 views

Sort fields inline

I'm trying to sort within a line of input over an unknown number of fields: Input: ab bc bc ab cd ef bc bc cd ef cd bc ab ef ab bc cd gh Output: ab bc ab bc bc cd ef bc cd ef ab cb cd ab bc cd ...
5
votes
1answer
148 views

Alphabetizing names in `ls` alias with .files not intermingled

I am currently using the following ls alias: alias ls='ls -alhGkpsq --color=auto'. This results in following directory listing. [03:35] bryson@brys ~ :$ ls total 48K 4.0K drwx------ 4 bryson 4.0K ...
5
votes
2answers
335 views

sort every field numerically, varying field counts

I'm trying to sort some data using sort. I noticed it was sorting by digit rather than number, so I added the -n flag. It then seemingly only numerically sorts on the first field though. Breaking it ...
5
votes
2answers
44 views

Sort one column only, keeping remaining columns untouched

How to (numerically) sort a specific column in a text file, without affecting other columns (irrespective of whether they are sorted or unsorted)? On other threads I found -s argument, but it does ...
4
votes
3answers
239 views

Grouped sorting of continuous paragraphs (separated by blank line)?

I think I'm pretty experienced now in sorting by columns; however, I haven't found anything so far how to sort continuous rows. Supposing we have a text file that looks like this: (very simplified, ...
4
votes
2answers
217 views

sort part of a file

How can I sort the file by the second column from the terminal. The content of the file is: Nome Note ------------ Mehdi 0 Sunday 20 Others 10 Shnou 5 Using sort -t' ' -nk2 is not ...
4
votes
3answers
3k views

Get lines with maximum values in the column using awk, uniq and sort

I have a file with next format 2011-12-01 user1 access1 2011-12-01 user1 access2 2011-12-01 user2 access2 2011-12-01 user4 access2 2011-12-02 user1 access1 2012-01-01 user3 access1 2012-01-01 user4 ...
4
votes
3answers
3k views

How to find files in subdirs and sort them by filename in a single command?

Result of a normal find using find . ! -path "./build*" -name "*.txt": ./tool/001-sub.txt ./tool/000-main.txt ./zo/001-int.txt ./zo/id/002-and.txt ./as/002-mod.txt and when sorted with sort -n: ...
4
votes
3answers
135 views

Sort all directories based on their size

I'd like to sort all the directories/files in a specific directory based on their size (using du -sh "name"). I need to apply this command to all directories in my location, then sort them based on ...
4
votes
3answers
645 views

For each subfolder, sort files by name and rename them to sequential padded numbers (regardless of extension)

I have the following directory structure: . ├── event-a │   ├── album-a │   │   ├── a.jpg │   │   └── x.png │   └── album-b │   ├── a.jpg │   ├── x.png │   └── y.gif └── event-b ├── ...
4
votes
1answer
471 views

Is gnu coreutils sort broken?

Consider the following input to sort: cat > foo <<EOM D,,5014978 DD,,25 D,I,1972765530 D,Y,4223624 -,Y,71285059 YA,I,2 EOM Now try running sort foo The output is not sorted when trying ...
4
votes
1answer
1k views

How to find out common elements between two files?

For an example, I have 2 files having following info: File #1: 12 13 14 15 File #2: 12 1 13 2 14 2 15 6 16 7 17 8 Output File: 1 2 2 6 In the output file, I want only the second column ...
4
votes
1answer
605 views

Linux sort second last column

I would like know how to sort the second last column ? It has the word "days=" in front of the number. I'm able to get the column with awk '{print $(NF-1)}', but sorting is wrong. 457000 ...
4
votes
2answers
169 views

Sort files by modification time

I've seen many questions and answers here using a construction along the lines of list_dir=`ls -t /path/to/dir/` for i in $list_dir; do or ls -t | while read i; do Now, I know that you shouldn't ...
4
votes
1answer
568 views

Sorting a file based on one column using Unix and Awk

I need to sort the input file according the 6th column, which is the score. Input File: Sc2/80 20 . A T 86 Pass N=2 F=5;U=4 Sc2/80 20 . A C 80 Pass N=2 F=5;U=4 Sc2/60 55 . G T 90 Pass N=2 F=5;U=4 ...
4
votes
4answers
121 views

how to sort access log efficiently in blocks

Access logs are more or less sorted by time, but to aggregate connections by time (uniq -c), you need to sort them a bit more. For a huge access log sort is very inefficient, because it buffers and ...
4
votes
1answer
341 views

Renaming files from bash in linux

There is a bunch of files of the following format hh-mm-ss.png where hh, mm, ss are hour, minute and second. How can I rename all these files in the directory in a sequential way from 1.png til n.png ...
4
votes
4answers
343 views

How to shift filename numbers without collisions?

Consider this list of files: $ touch index-{1,2,3,4,5,6,7,8,9}.txt If I want to shift them down so they start at zero, it's relatively easy: $ rename --verbose 's/^index-([1-9])\.txt$/$1/; ...
4
votes
2answers
534 views

Sort behaves strangely on case sensitive sorting

Sort is sorting differently than I would expect. I have this file, call it text.txt: a 1 A 1 a 11 (the space is always one \t) I want to sort them alphabetically by the first column. ...
4
votes
1answer
62 views

Help with the sort command (numeric)

I have this output I'd like to sort numerically by port (starting at the 35th column): tcp 0 0 192.168.0.210:110 0.0.0.0:* LISTEN 3385/dovecot tcp 0 0 ...
4
votes
2answers
1k views

What's the default order of linux sort?

For a long period I think the default behavior of sort program is using ASCII. However, when I input the following lines into sort without any arguments: # @ I got: @ # But according to the ...