Sorting text files or other data, for example with the sort utility.

learn more… | top users | synonyms

45
votes
9answers
15k views

how do you sort du output by size?

How do you sort du -sh /dir/* by size? I read one site that said use | sort -n but that's obviously not right. Here's an example that is wrong. [~]# du -sh /var/* | sort -n 0 /var/mail 1.2M ...
25
votes
2answers
738 views

Why is sort -o useful?

UNIX philosophy says: do one thing and do it well. Make programs that handle text, because that is a universal interface. The sort command, at least GNU sort, has an -o option to output to a file ...
22
votes
1answer
310 views

Why is sort changing the order of lines with identical sort keys?

Here is the data: D 2 B 2 A 2 When I run this command: sort -k2,2 file it outputs: A 2 B 2 D 2 My question is that when I specify only the second column -k2,2, why is it that it sorts by the ...
19
votes
3answers
6k views

What is the difference between “sort -u” and “sort | uniq”?

Everywhere I see someone needing to get a sorted, unique list, they always pipe to sort | uniq. I've never seen any examples where someone uses sort -u instead. Why not? What's the difference, and why ...
19
votes
6answers
7k views

sort but keep header line at the top

I am getting output from a program that first produces one line that is a bunch of column headers, and then a bunch of lines of data. I want to cut various columns of this output and view it sorted ...
14
votes
7answers
7k views

List files sorted numerically

I have a bunch of files from log1 to log164. I'm trying to LIST the directory (sorted) in a UNIX terminal but the sort functions are only providing the format like this: home:logs Home$ ls -1 | sort ...
11
votes
3answers
682 views

Sorting data faster approach

I need to sort a bed file randomly 10000 times and take the top 1000 rows each time. Currently, I am using the following code: for i in {1..100}; do for j in {1..100}; do sort -R ...
11
votes
4answers
9k views

Sorting the output of “find”?

I need to be able to alphabetically sort the output of find before piping it to a command. Entering | sort | between didn't work, so what could I do? find folder1 folder2 -name "*.txt" -print0 | ...
11
votes
2answers
2k views

Why does ls sorting ignore non-alphanumeric characters?

When sorting file names, ls ignores characters like -,_. I expected it to use those characters in sorting as well. An example: touch a1 a2 a-1 a-2 a_1 a_2 a.1 a.2 a,1 a,2 Now display these files ...
10
votes
2answers
2k views

sort inplace - like sed --in-place - exists?

Am I blind or is there no option like --in-place for sort? In order to save results to the input file, sed uses -i (--in-place). Redirecting the output of sort to the input file sort < f > f ...
10
votes
4answers
12k views

How to list files sorted by modification date recursively (no stat command available!)

How can I get the list of all files under current directory along with their modification date and sorted by that date? Now I know how to achieve that with find, stat and sort, but for some weird ...
10
votes
5answers
3k views

case-insensitive search of duplicate file-names

I there a way to find all files in a directory with duplicate filenames, regardless of the casing (upper-case and/or lower-case)?
10
votes
4answers
509 views

Shuffle file randomly with some additional constraints

I have a huge music playlist and, while some artists have many albums, others have just one song. I wanted to sort the playlist so the same artist won't play twice in a row, or his songs won't end up ...
9
votes
3answers
3k views

How to sort the string which combined with string + numeric using bash script?

This is the data what I want to sort. But sort treats the numeric to string, the data it no sorted as I expected. /home/files/profile1 /home/files/profile10 /home/files/profile11 ...
8
votes
2answers
6k views

Trying to sort on two fields, second then first

I am trying to sort on multiple columns. The results are not as expected. Here's my data (people.txt): Simon Strange 62 Pete Brown 37 Mark Brown 46 Stefan Heinz 52 Tony Bedford 50 John Strange 51 ...
8
votes
3answers
2k views

How can I remove duplicates in my .bash_history, preserving order?

I really enjoying using control+r to recursively search my command history. I've found a few good options I like to use with it: # ignore duplicate commands, ignore commands starting with a space ...
8
votes
5answers
596 views

How intelligent are unix utilities?

Unix utilities like sort, find, grep, diff et al are very handy to perform quick tasks, sometimes without writing any code at all. I wanted to know what algorithms do they internally use and how do ...
8
votes
2answers
146 views

sort by hex value

Using coreutils sort, how can I sort numerically by a hexadecimal value (field)? I was expecting something along the lines of sort -k3,3x file_to_sort however, such an x does not exist. Edit: ...
8
votes
4answers
4k views

How do I make ls sort underscore characters first?

I like being able to name files and directories with an underscore prefix if it's something I want to keep separate from other files and directories at the same level. On Windows and Mac, for example, ...
7
votes
4answers
170 views

Sort fields inline

I'm trying to sort within a line of input over an unknown number of fields: Input: ab bc bc ab cd ef bc bc cd ef cd bc ab ef ab bc cd gh Output: ab bc ab bc bc cd ef bc cd ef ab cb cd ab bc cd ...
7
votes
1answer
532 views

Is gnu coreutils sort broken?

Consider the following input to sort: cat > foo <<EOM D,,5014978 DD,,25 D,I,1972765530 D,Y,4223624 -,Y,71285059 YA,I,2 EOM Now try running sort foo The output is not sorted when trying ...
6
votes
4answers
663 views

uniq a csv file ignoring a column, awk maybe?

Given this file (annotations are not part of file, but form part of explanation)... x,a,001,b,c,d,y x,a,002,b,c,e,yy x,bb,003,b,d,e,y x,c,004,b,d,e,y x,c,005,b,d,e,y # nb - dupe of row 4 ...
6
votes
2answers
612 views

Counting number of times each IP address appears in log file

I have a file in the format as follows: $ cat file.txt 27.33.65.2 27.33.65.2 58.161.137.7 121.50.198.5 184.173.187.1 184.173.187.1 184.173.187.1 What's the best way to parse the file file.txt into ...
6
votes
2answers
130 views

Sorting with Perl respecting Locale Settings

Following data needs to be sorted respecting the locale sort order: wird sin wär pêche war Über Uber péché peach There was nor problem by using sort $ sort < data peach pêche péché sin Uber ...
6
votes
2answers
501 views

How would I sort these directory names numerically?

I'm aware that I can somehow sort this output numerically (so cpu1/ follows cpu0/) ... I could probably get something to work eventually by splitting up the string various ways with awk, but is there ...
6
votes
3answers
626 views

sort at jobs chronologically

Possibly a simple solution I misplaced. How can I get the output of atq sorted chronologically, so I can easily see who's due to run next? The man page for sort doesn't have anything built-in to ...
6
votes
1answer
120 views

How to specify additional parameters for GNU sort's --compress-program option?

sort --compress-program=/bin/gzip works fine, but how can I pass options down to the compression programm (e.g. --best or --fast)? sort --compress-program=/bin/gzip\ --best failes with the error ...
6
votes
1answer
77 views

biggest possible number for sort -n

How to determine the "biggest number" so that sort -n will always put it at the end? I'm thinking of something along Inf in some languages but I'm not sure if anything like this exists for sort. ...
6
votes
4answers
389 views

Fastest way to determine if two sorted lists contain unique elements

I have two sorted files A and B where the size of A is much greater than B, e.g. A is 100GB while B is 50MB. I want to quickly determine if there are any lines in B that are contained in A, stopping ...
6
votes
0answers
33 views

What should be the buffer size for the sort command?

I have a machine with 2 TB of RAM and I am running a sort command on a file of size 150G where I have specified the buffer-size as 1000G, after doing my bit of reasearch on google, I got this piece of ...
5
votes
5answers
252 views

Sorting blocks of lines

I have a file that contains 4n lines. Here is an excerpt from it containing 8 lines 6115 8.88443 6116 6.61875 6118 16.5949 6117 19.4129 6116 6.619 6117 16.5979 6118 19.4111 6115 8.88433 What I ...
5
votes
5answers
9k views

join : “File 2 not in sorted order”

I've got two files _jeter3.txt and _jeter1.txt I've checked they are both sorted on the 20th column using sort -c sort -t ' ' -c -k20,20 _jeter3.txt sort -t ' ' -c -k20,20 _jeter1.txt #no ...
5
votes
2answers
1k views

sort output by column

I'd like to take this command find -maxdepth 1 -type d | while read -r dir; do printf "%s:\t" "$dir"; find "$dir" | wc -l; done ( from here ). which has an output of basically ./kennel: 11062 ...
5
votes
6answers
1k views

How to sort (by whatever key) BibTex entries in `.bib` file?

I work with LaTeX and do versioning with Git. For bibliography management I use Mendeley. The problem is that each time Mendeley synchronizes it's .bib exports, they are in different order, what ...
5
votes
1answer
1k views

Sorting a file based on one column using Unix and Awk

I need to sort the input file according the 6th column, which is the score. Input File: Sc2/80 20 . A T 86 Pass N=2 F=5;U=4 Sc2/80 20 . A C 80 Pass N=2 F=5;U=4 Sc2/60 55 . G T 90 Pass N=2 F=5;U=4 ...
5
votes
1answer
162 views

Alphabetizing names in `ls` alias with .files not intermingled

I am currently using the following ls alias: alias ls='ls -alhGkpsq --color=auto'. This results in following directory listing. [03:35] bryson@brys ~ :$ ls total 48K 4.0K drwx------ 4 bryson 4.0K ...
5
votes
2answers
2k views

What's the default order of linux sort?

For a long period I think the default behavior of sort program is using ASCII. However, when I input the following lines into sort without any arguments: # @ I got: @ # But according to the ...
5
votes
2answers
373 views

sort every field numerically, varying field counts

I'm trying to sort some data using sort. I noticed it was sorting by digit rather than number, so I added the -n flag. It then seemingly only numerically sorts on the first field though. Breaking it ...
5
votes
2answers
66 views

Sort one column only, keeping remaining columns untouched

How to (numerically) sort a specific column in a text file, without affecting other columns (irrespective of whether they are sorted or unsorted)? On other threads I found -s argument, but it does ...
4
votes
3answers
281 views

Grouped sorting of continuous paragraphs (separated by blank line)?

I think I'm pretty experienced now in sorting by columns; however, I haven't found anything so far how to sort continuous rows. Supposing we have a text file that looks like this: (very simplified, ...
4
votes
3answers
4k views

Get lines with maximum values in the column using awk, uniq and sort

I have a file with next format 2011-12-01 user1 access1 2011-12-01 user1 access2 2011-12-01 user2 access2 2011-12-01 user4 access2 2011-12-02 user1 access1 2012-01-01 user3 access1 2012-01-01 user4 ...
4
votes
3answers
4k views

How to find files in subdirs and sort them by filename in a single command?

Result of a normal find using find . ! -path "./build*" -name "*.txt": ./tool/001-sub.txt ./tool/000-main.txt ./zo/001-int.txt ./zo/id/002-and.txt ./as/002-mod.txt and when sorted with sort -n: ...
4
votes
2answers
258 views

sort part of a file

How can I sort the file by the second column from the terminal. The content of the file is: Nome Note ------------ Mehdi 0 Sunday 20 Others 10 Shnou 5 Using sort -t' ' -nk2 is not ...
4
votes
3answers
177 views

Sort all directories based on their size

I'd like to sort all the directories/files in a specific directory based on their size (using du -sh "name"). I need to apply this command to all directories in my location, then sort them based on ...
4
votes
3answers
709 views

For each subfolder, sort files by name and rename them to sequential padded numbers (regardless of extension)

I have the following directory structure: . ├── event-a │   ├── album-a │   │   ├── a.jpg │   │   └── x.png │   └── album-b │   ├── a.jpg │   ├── x.png │   └── y.gif └── event-b ├── ...
4
votes
1answer
2k views

How to find out common elements between two files?

For an example, I have 2 files having following info: File #1: 12 13 14 15 File #2: 12 1 13 2 14 2 15 6 16 7 17 8 Output File: 1 2 2 6 In the output file, I want only the second column ...
4
votes
1answer
687 views

Linux sort second last column

I would like know how to sort the second last column ? It has the word "days=" in front of the number. I'm able to get the column with awk '{print $(NF-1)}', but sorting is wrong. 457000 ...
4
votes
2answers
231 views

Sort files by modification time

I've seen many questions and answers here using a construction along the lines of list_dir=`ls -t /path/to/dir/` for i in $list_dir; do or ls -t | while read i; do Now, I know that you shouldn't ...
4
votes
4answers
144 views

how to sort access log efficiently in blocks

Access logs are more or less sorted by time, but to aggregate connections by time (uniq -c), you need to sort them a bit more. For a huge access log sort is very inefficient, because it buffers and ...
4
votes
1answer
370 views

Renaming files from bash in linux

There is a bunch of files of the following format hh-mm-ss.png where hh, mm, ss are hour, minute and second. How can I rename all these files in the directory in a sequential way from 1.png til n.png ...