Sorting text files or other data, for example with the sort utility.

learn more… | top users | synonyms

6
votes
5answers
20k views

join : “File 2 not in sorted order”

I've got two files _jeter3.txt and _jeter1.txt I've checked they are both sorted on the 20th column using sort -c sort -t ' ' -c -k20,20 _jeter3.txt sort -t ' ' -c -k20,20 _jeter1.txt #no ...
6
votes
5answers
1k views

Sorting blocks of lines

I have a file that contains 4n lines. Here is an excerpt from it containing 8 lines 6115 8.88443 6116 6.61875 6118 16.5949 6117 19.4129 6116 6.619 6117 16.5979 6118 19.4111 6115 8.88433 What I ...
6
votes
3answers
3k views

sum columns based on matching fields

I have encountered a problem that exceeds my basic unix knowledge and would really appreciate some help. I have a large file in the following format: 2 1019 0 12 2 1019 3 0 2 1021 0 2 ...
6
votes
2answers
342 views

Sorting with Perl respecting Locale Settings

Following data needs to be sorted respecting the locale sort order: wird sin wär pêche war Über Uber péché peach There was nor problem by using sort $ sort < data peach pêche péché sin Uber ...
6
votes
4answers
191 views

Find newest file. Multiple Filetype restrictions

I want to write a command that gives me the newest file in a directory recursively. But that's not my only limitation. The files has to be an mp3 or a jpg file. (case insensitive prefered) I only need ...
6
votes
2answers
2k views

Where has my `uniq` or `sort -u` line gone, with some unicode characters

What is going on in the following code snippet? I'm not getting my expected output. I'd think it was a bug, but it happens for 2 different programs (uniq and sort), so I suspect it is something ...
6
votes
2answers
1k views

How would I sort these directory names numerically?

I'm aware that I can somehow sort this output numerically (so cpu1/ follows cpu0/) ... I could probably get something to work eventually by splitting up the string various ways with awk, but is there ...
6
votes
2answers
632 views

sort every field numerically, varying field counts

I'm trying to sort some data using sort. I noticed it was sorting by digit rather than number, so I added the -n flag. It then seemingly only numerically sorts on the first field though. Breaking it ...
6
votes
1answer
385 views

How to specify additional parameters for GNU sort's --compress-program option?

sort --compress-program=/bin/gzip works fine, but how can I pass options down to the compression programm (e.g. --best or --fast)? sort --compress-program=/bin/gzip\ --best failes with the error ...
6
votes
1answer
322 views

Alphabetizing names in `ls` alias with .files not intermingled

I am currently using the following ls alias: alias ls='ls -alhGkpsq --color=auto'. This results in following directory listing. [03:35] bryson@brys ~ :$ ls total 48K 4.0K drwx------ 4 bryson 4.0K ...
6
votes
4answers
832 views

How to shift filename numbers without collisions?

Consider this list of files: $ touch index-{1,2,3,4,5,6,7,8,9}.txt If I want to shift them down so they start at zero, it's relatively easy: $ rename --verbose 's/^index-([1-9])\.txt$/$1/; $_="...
6
votes
1answer
321 views

How to make a list generated by md5deep in alphabetical order of relative paths?

If I want make a md5sum list recursively, then I would use md5deep, but it starts to pop up some problems such as it won't generate the md5sum file in alphabetical order. For example, $ cd /media/...
6
votes
1answer
177 views

biggest possible number for sort -n

How to determine the "biggest number" so that sort -n will always put it at the end? I'm thinking of something along Inf in some languages but I'm not sure if anything like this exists for sort. ...
6
votes
4answers
457 views

Fastest way to determine if two sorted lists contain unique elements

I have two sorted files A and B where the size of A is much greater than B, e.g. A is 100GB while B is 50MB. I want to quickly determine if there are any lines in B that are contained in A, stopping ...
6
votes
2answers
2k views

What should be the buffer size for the sort command?

I have a machine with 2 TB of RAM and I am running a sort command on a file of size 150G where I have specified the buffer-size as 1000G, after doing my bit of reasearch on google, I got this piece of ...
5
votes
7answers
10k views

Sort the files in the directory recursively based on last modified date

Sort the files in the directory recursively based on last modified date I have modified a lot of files in my directory want to know what are those files by sorting them by the last modified date ...
5
votes
2answers
1k views

Fastest `uniq` tool in linux

I have large text file (1.5 G), I want to know what the fastest and more reliable tool in linux is. I usualy use: awk '!x[$0]++' file.txt But when I use htop command I see my memory usage is ...
5
votes
2answers
11k views

Sorting by date

Original: Jan 23 2011 10:42 SOMETHING 2007.12.20.avi Jun 26 2009 SOMETHING 2009.06.25.avi Feb 12 2010 SOMETHING 2010.02.11.avi Jan 29 2011 09:17 SOMETHING 2011.01.27.avi Feb 11 2011 20:06 SOMETHING ...
5
votes
1answer
6k views

How to numerical sort by last column?

I have this input: sdkxyosl 1 safkls 2 asdf--asdfasxy_asd 5 dkd8k jasd 29 sdi44sw 43 asasd afsdfs 10 rklyasd 4 I need this output: sdi44sw 43 dkd8k jasd 29 asasd afsdfs 10 asdf--asdfasxy_asd 5 ...
5
votes
5answers
2k views

Sort input file by the results of a regex

I'd like to sort a file based on the results of a regex expression. For example, if I have the following property declarations in Obj-C @property (nonatomic, strong) id <AlbumArtDelegate, ...
5
votes
4answers
1k views

For each subfolder, sort files by name and rename them to sequential padded numbers (regardless of extension)

I have the following directory structure: . ├── event-a │   ├── album-a │   │   ├── a.jpg │   │   └── x.png │   └── album-b │   ├── a.jpg │   ├── x.png │   └── y.gif └── event-b ├── ...
5
votes
1answer
4k views

Sorting a file based on one column using Unix and Awk

I need to sort the input file according the 6th column, which is the score. Input File: Sc2/80 20 . A T 86 Pass N=2 F=5;U=4 Sc2/80 20 . A C 80 Pass N=2 F=5;U=4 Sc2/60 55 . G T 90 Pass N=2 F=5;U=4 ...
5
votes
1answer
242 views

Sort output of awk except for first line?

This is a use case I am continually running into in parsing CSV files. When it is an inline awk script embedded in a shell script, I can use this workaround: myfile=$(mktemp) awk '(awk script here)' ...
5
votes
2answers
192 views

How to sort lines that contain “_” numerically?

Here's a subset of the file names in my file: profile_10_1_1 profile_10_1_2 profile_1_1_1 I'm trying to sort them numerically in ascending order, that is starting from 1 onwards. I used the ...
5
votes
1answer
44 views

Sort based on field1, field2 but preserving field 1 sort and ignoring nulls/blanks in 2nd sort

After doing a lot of home work and running multiple test scenarios, I decided to get help from experts Here is my scenario Input data after sorting by Column 1 -bash-3.2$ sort -t'|' -k1 -k2 clm....
5
votes
2answers
1k views

Sort behaves strangely on case sensitive sorting

Sort is sorting differently than I would expect. I have this file, call it text.txt: a 1 A 1 a 11 (the space is always one \t) I want to sort them alphabetically by the first column. However,...
5
votes
1answer
81 views

sort not working for similar entries

I am sorting a file prior to joining it with another file, using sort -k1 file1 > file1_sort When I try to join with the second file, I get an error saying file1 is not sorted. I think this is ...
5
votes
3answers
401 views

Sort one column only, keeping remaining columns untouched

How to (numerically) sort a specific column in a text file, without affecting other columns (irrespective of whether they are sorted or unsorted)? On other threads I found -s argument, but it does ...
5
votes
1answer
992 views

Is there a search utility which can binary split search a large sorted (sequential) text file?

Yes, I know it sounds odd. Sequential and Binary-splits don't mix.. That is unless the sequence is the byte offset within the file itself... I've scrambled together a binary split search in bash ...
5
votes
1answer
1k views

How can I find the sort order for ls with my locale?

I was using ls -l on a directory, and was surprised that spaces and underscores were ignored for the sort order. For example, $ echo $LANG en_AU.UTF-8 $ ls -l total 0 -rw-r--r-- 1 sparhawk sparhawk 0 ...
5
votes
2answers
99 views

What is a good way to sort two log files as they are created?

I have two scripts running, both of which output a log file. I'd like to make a third script that can sort these logs by time stamp and merge them into one file as they are created. What is a good ...
5
votes
1answer
135 views

Non-standard sorting of the ls output

I usually include a date in the filename so that files and directories are listed in chronological order when I use the ls command, e.g. 2015-08-29_letter_to_santa.txt. A drawback is that tab-...
4
votes
7answers
2k views

Need some sed magic: Moving marked lines to the beginning of a file

I find myself in dire need of some sed magic (I really need to sit myself down and learn this). I have a file with lots of lines. After reviewing it, I've marked some lines by adding a star (*) to ...
4
votes
3answers
2k views

Sort using custom pattern

Is there a way to output file's contents using custom patterns? For instance, having a file myfile with following contents: a d b c ..how would one sort it using following pattern: print lines ...
4
votes
3answers
7k views

Get lines with maximum values in the column using awk, uniq and sort

I have a file with next format 2011-12-01 user1 access1 2011-12-01 user1 access2 2011-12-01 user2 access2 2011-12-01 user4 access2 2011-12-02 user1 access1 2012-01-01 user3 access1 2012-01-01 user4 ...
4
votes
5answers
348 views

sort order explanation

Giving a lot of files, sort -n gives me this result: f1.txt f10.txt f2.txt which doesn't seem right to me. What is the explanation? Can I get this by default: f1.txt f2.txt f10.txt
4
votes
3answers
9k views

How to find files in subdirs and sort them by filename in a single command?

Result of a normal find using find . ! -path "./build*" -name "*.txt": ./tool/001-sub.txt ./tool/000-main.txt ./zo/001-int.txt ./zo/id/002-and.txt ./as/002-mod.txt and when sorted with sort -n: ./...
4
votes
3answers
615 views

Finding files on a Unix server and sorting by size

I have a web server where I have run out of space and it's causing issues with the WordPress sites I am running on them. I know that I have a lot of large .png files (the fact that it's PNG is a ...
4
votes
2answers
806 views

sort part of a file

How can I sort the file by the second column from the terminal. The content of the file is: Nome Note ------------ Mehdi 0 Sunday 20 Others 10 Shnou 5 Using sort -t' ' -nk2 is not ...
4
votes
3answers
4k views

Sort comma-separated fields on each line by numeric value

I tried this: $ echo "2,3435,1" | sort -n 2,3435,1 $ sort -t',' -n test_sort.txt kill,gill,burger 110,20,30,13 $ cat test_sort.txt 110,20,30,13 kill,gill,burger Why doesn't my sort command work? ...
4
votes
4answers
157 views

Stable sort a file by presence of ordered substrings on each line

I have a list of sound sources that I'm processing with a script. An example would be: alsa_input.usb-AVEO_Technology_Corp._USB2.0_Camera-02-Camera.analog-mono alsa_input.pci-0000_00_14.2.analog-...
4
votes
3answers
675 views

Sort all directories based on their size

I'd like to sort all the directories/files in a specific directory based on their size (using du -sh "name"). I need to apply this command to all directories in my location, then sort them based on ...
4
votes
1answer
3k views

How to find out common elements between two files?

For an example, I have 2 files having following info: File #1: 12 13 14 15 File #2: 12 1 13 2 14 2 15 6 16 7 17 8 Output File: 1 2 2 6 In the output file, I want only the second column ...
4
votes
5answers
105 views

How do I sort on the first column every 4 lines in a file with a key in the second column

Example File : first column can have fixed set of 4 unordered values world1.com,world2.com,world3.com or world4.com second column is a key belongs to each line such that each of the four sets has ...
4
votes
1answer
1k views

Linux sort second last column

I would like know how to sort the second last column ? It has the word "days=" in front of the number. I'm able to get the column with awk '{print $(NF-1)}', but sorting is wrong. 457000 (...
4
votes
3answers
218 views

Find largest files from each date in a directory

I have a directory similar to the following: -rw-r--r-- 1 root root 223K Apr 28 14:25 2015.04.28_14.25 -rw-r--r-- 1 root root 253K Apr 28 14:55 2015.04.28_14.55 -rw-r--r-- 1 root root 276K Apr 28 15:...
4
votes
4answers
811 views

Sorting directories by last modified date/time of the same-named contained file

Is it possible, using grep, find, etc, to sort a list of directories by the last modified date of the same-named file (e.g., file.php) within? For example: domain1/file.php (last modified 20-Jan-2014 ...
4
votes
1answer
6k views

The simplest method to count lines matching specific patterns, including '0' if line is not found?

I have very big logs (several gigabytes per day), that can (but do not need to) contain specific lines. I have to count the number of occurences of every one of these lines on a daily basis. I have a ...
4
votes
2answers
285 views

GNU utility: sort

I have an issue sorting a file based on the first two columns. The layout of the file is: 1 998688068 PizzaFan Insurance 22.47 5 072821325 Plaisio Computers 26.35 4 998688068 PizzaFan Food 27.32 5 ...
4
votes
2answers
184 views

concatenate n lines with sed

Recently, I asked a question how to sort pairs of lines, and one of the answers suggested concatenating lines with sed, like this: cat file.txt | sed -n 'N;s/\n//;p' | sort -t";" -k43,43n | perl -F';'...