Sorting text files or other data, for example with the sort utility.

learn more… | top users | synonyms

6
votes
3answers
2k views

sum columns based on matching fields

I have encountered a problem that exceeds my basic unix knowledge and would really appreciate some help. I have a large file in the following format: 2 1019 0 12 2 1019 3 0 2 1021 0 2 ...
6
votes
2answers
300 views

Sorting with Perl respecting Locale Settings

Following data needs to be sorted respecting the locale sort order: wird sin wär pêche war Über Uber péché peach There was nor problem by using sort $ sort < data peach pêche péché sin Uber ...
6
votes
4answers
169 views

Find newest file. Multiple Filetype restrictions

I want to write a command that gives me the newest file in a directory recursively. But that's not my only limitation. The files has to be an mp3 or a jpg file. (case insensitive prefered) I only need ...
6
votes
2answers
1k views

How would I sort these directory names numerically?

I'm aware that I can somehow sort this output numerically (so cpu1/ follows cpu0/) ... I could probably get something to work eventually by splitting up the string various ways with awk, but is there ...
6
votes
2answers
595 views

sort every field numerically, varying field counts

I'm trying to sort some data using sort. I noticed it was sorting by digit rather than number, so I added the -n flag. It then seemingly only numerically sorts on the first field though. Breaking it ...
6
votes
1answer
316 views

How to specify additional parameters for GNU sort's --compress-program option?

sort --compress-program=/bin/gzip works fine, but how can I pass options down to the compression programm (e.g. --best or --fast)? sort --compress-program=/bin/gzip\ --best failes with the error ...
6
votes
1answer
283 views

How to make a list generated by md5deep in alphabetical order of relative paths?

If I want make a md5sum list recursively, then I would use md5deep, but it starts to pop up some problems such as it won't generate the md5sum file in alphabetical order. For example, $ cd ...
6
votes
1answer
160 views

biggest possible number for sort -n

How to determine the "biggest number" so that sort -n will always put it at the end? I'm thinking of something along Inf in some languages but I'm not sure if anything like this exists for sort. ...
6
votes
4answers
456 views

Fastest way to determine if two sorted lists contain unique elements

I have two sorted files A and B where the size of A is much greater than B, e.g. A is 100GB while B is 50MB. I want to quickly determine if there are any lines in B that are contained in A, stopping ...
6
votes
2answers
1k views

What should be the buffer size for the sort command?

I have a machine with 2 TB of RAM and I am running a sort command on a file of size 150G where I have specified the buffer-size as 1000G, after doing my bit of reasearch on google, I got this piece of ...
5
votes
7answers
7k views

Sort the files in the directory recursively based on last modified date

Sort the files in the directory recursively based on last modified date I have modified a lot of files in my directory want to know what are those files by sorting them by the last modified date ...
5
votes
2answers
843 views

Fastest `uniq` tool in linux

I have large text file (1.5 G), I want to know what the fastest and more reliable tool in linux is. I usualy use: awk '!x[$0]++' file.txt But when I use htop command I see my memory usage is ...
5
votes
2answers
9k views

Sorting by date

Original: Jan 23 2011 10:42 SOMETHING 2007.12.20.avi Jun 26 2009 SOMETHING 2009.06.25.avi Feb 12 2010 SOMETHING 2010.02.11.avi Jan 29 2011 09:17 SOMETHING 2011.01.27.avi Feb 11 2011 20:06 SOMETHING ...
5
votes
5answers
2k views

Sort input file by the results of a regex

I'd like to sort a file based on the results of a regex expression. For example, if I have the following property declarations in Obj-C @property (nonatomic, strong) id <AlbumArtDelegate, ...
5
votes
2answers
2k views

Where has my `uniq` or `sort -u` line gone, with some unicode characters

What is going on in the following code snippet? I'm not getting my expected output. I'd think it was a bug, but it happens for 2 different programs (uniq and sort), so I suspect it is something ...
5
votes
4answers
1k views

For each subfolder, sort files by name and rename them to sequential padded numbers (regardless of extension)

I have the following directory structure: . ├── event-a │   ├── album-a │   │   ├── a.jpg │   │   └── x.png │   └── album-b │   ├── a.jpg │   ├── x.png │   └── y.gif └── event-b ├── ...
5
votes
1answer
4k views

Sorting a file based on one column using Unix and Awk

I need to sort the input file according the 6th column, which is the score. Input File: Sc2/80 20 . A T 86 Pass N=2 F=5;U=4 Sc2/80 20 . A C 80 Pass N=2 F=5;U=4 Sc2/60 55 . G T 90 Pass N=2 F=5;U=4 ...
5
votes
1answer
126 views

Sort output of awk except for first line?

This is a use case I am continually running into in parsing CSV files. When it is an inline awk script embedded in a shell script, I can use this workaround: myfile=$(mktemp) awk '(awk script here)' ...
5
votes
2answers
181 views

How to sort lines that contain “_” numerically?

Here's a subset of the file names in my file: profile_10_1_1 profile_10_1_2 profile_1_1_1 I'm trying to sort them numerically in ascending order, that is starting from 1 onwards. I used the ...
5
votes
1answer
32 views

Sort based on field1, field2 but preserving field 1 sort and ignoring nulls/blanks in 2nd sort

After doing a lot of home work and running multiple test scenarios, I decided to get help from experts Here is my scenario Input data after sorting by Column 1 -bash-3.2$ sort -t'|' -k1 -k2 ...
5
votes
2answers
1k views

Sort behaves strangely on case sensitive sorting

Sort is sorting differently than I would expect. I have this file, call it text.txt: a 1 A 1 a 11 (the space is always one \t) I want to sort them alphabetically by the first column. ...
5
votes
1answer
75 views

sort not working for similar entries

I am sorting a file prior to joining it with another file, using sort -k1 file1 > file1_sort When I try to join with the second file, I get an error saying file1 is not sorted. I think this is ...
5
votes
3answers
306 views

Sort one column only, keeping remaining columns untouched

How to (numerically) sort a specific column in a text file, without affecting other columns (irrespective of whether they are sorted or unsorted)? On other threads I found -s argument, but it does ...
5
votes
1answer
275 views

Alphabetizing names in `ls` alias with .files not intermingled

I am currently using the following ls alias: alias ls='ls -alhGkpsq --color=auto'. This results in following directory listing. [03:35] bryson@brys ~ :$ ls total 48K 4.0K drwx------ 4 bryson 4.0K ...
5
votes
4answers
757 views

How to shift filename numbers without collisions?

Consider this list of files: $ touch index-{1,2,3,4,5,6,7,8,9}.txt If I want to shift them down so they start at zero, it's relatively easy: $ rename --verbose 's/^index-([1-9])\.txt$/$1/; ...
5
votes
1answer
933 views

Is there a search utility which can binary split search a large sorted (sequential) text file?

Yes, I know it sounds odd. Sequential and Binary-splits don't mix.. That is unless the sequence is the byte offset within the file itself... I've scrambled together a binary split search in bash ...
5
votes
1answer
914 views

How can I find the sort order for ls with my locale?

I was using ls -l on a directory, and was surprised that spaces and underscores were ignored for the sort order. For example, $ echo $LANG en_AU.UTF-8 $ ls -l total 0 -rw-r--r-- 1 sparhawk sparhawk 0 ...
4
votes
7answers
2k views

Need some sed magic: Moving marked lines to the beginning of a file

I find myself in dire need of some sed magic (I really need to sit myself down and learn this). I have a file with lots of lines. After reviewing it, I've marked some lines by adding a star (*) to ...
4
votes
3answers
1k views

Sort using custom pattern

Is there a way to output file's contents using custom patterns? For instance, having a file myfile with following contents: a d b c ..how would one sort it using following pattern: print lines ...
4
votes
3answers
7k views

Get lines with maximum values in the column using awk, uniq and sort

I have a file with next format 2011-12-01 user1 access1 2011-12-01 user1 access2 2011-12-01 user2 access2 2011-12-01 user4 access2 2011-12-02 user1 access1 2012-01-01 user3 access1 2012-01-01 user4 ...
4
votes
2answers
277 views

Unix Multiple Column Sort Issue

I am trying to sort a file: TX001;A0004|Number|5|0|Y|1 TX001;A0110|VARCHAR|5|0|Y|4 TX001;A1332|VARCHAR|15|0|Y|3 TX001;B3321|VARCHAR|15|0|Y|5 ...
4
votes
5answers
329 views

sort order explanation

Giving a lot of files, sort -n gives me this result: f1.txt f10.txt f2.txt which doesn't seem right to me. What is the explanation? Can I get this by default: f1.txt f2.txt f10.txt
4
votes
3answers
570 views

Finding files on a Unix server and sorting by size

I have a web server where I have run out of space and it's causing issues with the WordPress sites I am running on them. I know that I have a lot of large .png files (the fact that it's PNG is a ...
4
votes
3answers
8k views

How to find files in subdirs and sort them by filename in a single command?

Result of a normal find using find . ! -path "./build*" -name "*.txt": ./tool/001-sub.txt ./tool/000-main.txt ./zo/001-int.txt ./zo/id/002-and.txt ./as/002-mod.txt and when sorted with sort -n: ...
4
votes
2answers
652 views

sort part of a file

How can I sort the file by the second column from the terminal. The content of the file is: Nome Note ------------ Mehdi 0 Sunday 20 Others 10 Shnou 5 Using sort -t' ' -nk2 is not ...
4
votes
4answers
144 views

Stable sort a file by presence of ordered substrings on each line

I have a list of sound sources that I'm processing with a script. An example would be: alsa_input.usb-AVEO_Technology_Corp._USB2.0_Camera-02-Camera.analog-mono ...
4
votes
1answer
5k views

How to numerical sort by last column?

I have this input: sdkxyosl 1 safkls 2 asdf--asdfasxy_asd 5 dkd8k jasd 29 sdi44sw 43 asasd afsdfs 10 rklyasd 4 I need this output: sdi44sw 43 dkd8k jasd 29 asasd afsdfs 10 asdf--asdfasxy_asd 5 ...
4
votes
3answers
599 views

Sort all directories based on their size

I'd like to sort all the directories/files in a specific directory based on their size (using du -sh "name"). I need to apply this command to all directories in my location, then sort them based on ...
4
votes
1answer
11k views

Sorting file by second column

How can I manipulate a two columns tab-separated text file by sorting by the first element of the second column (only if the first column element is the same)? Exemple: Input File 1 A 1-2 A ...
4
votes
1answer
3k views

How to find out common elements between two files?

For an example, I have 2 files having following info: File #1: 12 13 14 15 File #2: 12 1 13 2 14 2 15 6 16 7 17 8 Output File: 1 2 2 6 In the output file, I want only the second column ...
4
votes
1answer
1k views

Linux sort second last column

I would like know how to sort the second last column ? It has the word "days=" in front of the number. I'm able to get the column with awk '{print $(NF-1)}', but sorting is wrong. 457000 ...
4
votes
3answers
160 views

Find largest files from each date in a directory

I have a directory similar to the following: -rw-r--r-- 1 root root 223K Apr 28 14:25 2015.04.28_14.25 -rw-r--r-- 1 root root 253K Apr 28 14:55 2015.04.28_14.55 -rw-r--r-- 1 root root 276K Apr 28 ...
4
votes
4answers
619 views

Sorting directories by last modified date/time of the same-named contained file

Is it possible, using grep, find, etc, to sort a list of directories by the last modified date of the same-named file (e.g., file.php) within? For example: domain1/file.php (last modified 20-Jan-2014 ...
4
votes
2answers
275 views

GNU utility: sort

I have an issue sorting a file based on the first two columns. The layout of the file is: 1 998688068 PizzaFan Insurance 22.47 5 072821325 Plaisio Computers 26.35 4 998688068 PizzaFan Food 27.32 5 ...
4
votes
2answers
137 views

concatenate n lines with sed

Recently, I asked a question how to sort pairs of lines, and one of the answers suggested concatenating lines with sed, like this: cat file.txt | sed -n 'N;s/\n//;p' | sort -t";" -k43,43n | perl ...
4
votes
2answers
320 views

how to sort by the day of the week?

I have two files. file1: Dave 734.838.9800 Bob 313.123.4567 Carol 248.344.5576 Mary 313.449.1390 Ted 248.496.2204 Alice 616.556.4458 file2: Bob Tuesday Carol Monday Ted Sunday ...
4
votes
4answers
324 views

how to sort access log efficiently in blocks

Access logs are more or less sorted by time, but to aggregate connections by time (uniq -c), you need to sort them a bit more. For a huge access log sort is very inefficient, because it buffers and ...
4
votes
1answer
517 views

Renaming files from bash in linux

There is a bunch of files of the following format hh-mm-ss.png where hh, mm, ss are hour, minute and second. How can I rename all these files in the directory in a sequential way from 1.png til n.png ...
4
votes
1answer
8k views

Unix sort by multiple columns

I have trouble understanding unix sort. Consider the following file (tab separated) aa ~ a1 aa B b A b ~ e bb B bb ~ B When calling: cat tmp2 | sort -t $'\t' -k1,2 I get aa ~ a1 aa B ...
4
votes
2answers
138 views

Sorting strings with ANSI escape codes

I am trying to print a sorted list of all zsh options, with set options colored green and unset options colored red. I cannot get sort to work properly with on the colored lines though.The below ...