Sorting text files or other data, for example with the sort utility.

learn more… | top users | synonyms

6
votes
4answers
435 views

Fastest way to determine if two sorted lists contain unique elements

I have two sorted files A and B where the size of A is much greater than B, e.g. A is 100GB while B is 50MB. I want to quickly determine if there are any lines in B that are contained in A, stopping ...
6
votes
1answer
2k views

comm: file is not in sorted order

I used comm to compare two sorted files. Each line in these files are positive integer numbers. But the results show comm: file 1 is not in sorted order comm: file 2 is not in sorted order How ...
6
votes
2answers
861 views

What should be the buffer size for the sort command?

I have a machine with 2 TB of RAM and I am running a sort command on a file of size 150G where I have specified the buffer-size as 1000G, after doing my bit of reasearch on google, I got this piece of ...
5
votes
5answers
14k views

join : “File 2 not in sorted order”

I've got two files _jeter3.txt and _jeter1.txt I've checked they are both sorted on the 20th column using sort -c sort -t ' ' -c -k20,20 _jeter3.txt sort -t ' ' -c -k20,20 _jeter1.txt #no ...
5
votes
5answers
721 views

Sorting blocks of lines

I have a file that contains 4n lines. Here is an excerpt from it containing 8 lines 6115 8.88443 6116 6.61875 6118 16.5949 6117 19.4129 6116 6.619 6117 16.5979 6118 19.4111 6115 8.88433 What I ...
5
votes
3answers
6k views

How is it possible to sort ps command's CPU field?

I'm trying to make a small command that will find the processes that use the most CPU power. Firstly, I use ps aux > file.txt and then cut -c 16-20 file.txt | sort -n | tail -5. The result I get ...
5
votes
2answers
1k views

Where has my `uniq` or `sort -u` line gone, with some unicode characters

What is going on in the following code snippet? I'm not getting my expected output. I'd think it was a bug, but it happens for 2 different programs (uniq and sort), so I suspect it is something ...
5
votes
4answers
1k views

For each subfolder, sort files by name and rename them to sequential padded numbers (regardless of extension)

I have the following directory structure: . ├── event-a │   ├── album-a │   │   ├── a.jpg │   │   └── x.png │   └── album-b │   ├── a.jpg │   ├── x.png │   └── y.gif └── event-b ├── ...
5
votes
1answer
3k views

Sorting a file based on one column using Unix and Awk

I need to sort the input file according the 6th column, which is the score. Input File: Sc2/80 20 . A T 86 Pass N=2 F=5;U=4 Sc2/80 20 . A C 80 Pass N=2 F=5;U=4 Sc2/60 55 . G T 90 Pass N=2 F=5;U=4 ...
5
votes
2answers
166 views

How to sort lines that contain “_” numerically?

Here's a subset of the file names in my file: profile_10_1_1 profile_10_1_2 profile_1_1_1 I'm trying to sort them numerically in ascending order, that is starting from 1 onwards. I used the ...
5
votes
1answer
29 views

Sort based on field1, field2 but preserving field 1 sort and ignoring nulls/blanks in 2nd sort

After doing a lot of home work and running multiple test scenarios, I decided to get help from experts Here is my scenario Input data after sorting by Column 1 -bash-3.2$ sort -t'|' -k1 -k2 ...
5
votes
1answer
65 views

sort not working for similar entries

I am sorting a file prior to joining it with another file, using sort -k1 file1 > file1_sort When I try to join with the second file, I get an error saying file1 is not sorted. I think this is ...
5
votes
3answers
212 views

Sort one column only, keeping remaining columns untouched

How to (numerically) sort a specific column in a text file, without affecting other columns (irrespective of whether they are sorted or unsorted)? On other threads I found -s argument, but it does ...
5
votes
1answer
243 views

Alphabetizing names in `ls` alias with .files not intermingled

I am currently using the following ls alias: alias ls='ls -alhGkpsq --color=auto'. This results in following directory listing. [03:35] bryson@brys ~ :$ ls total 48K 4.0K drwx------ 4 bryson 4.0K ...
5
votes
4answers
679 views

How to shift filename numbers without collisions?

Consider this list of files: $ touch index-{1,2,3,4,5,6,7,8,9}.txt If I want to shift them down so they start at zero, it's relatively easy: $ rename --verbose 's/^index-([1-9])\.txt$/$1/; ...
5
votes
1answer
849 views

Is there a search utility which can binary split search a large sorted (sequential) text file?

Yes, I know it sounds odd. Sequential and Binary-splits don't mix.. That is unless the sequence is the byte offset within the file itself... I've scrambled together a binary split search in bash ...
4
votes
7answers
1k views

Need some sed magic: Moving marked lines to the beginning of a file

I find myself in dire need of some sed magic (I really need to sit myself down and learn this). I have a file with lots of lines. After reviewing it, I've marked some lines by adding a star (*) to ...
4
votes
5answers
304 views

sort order explanation

Giving a lot of files, sort -n gives me this result: f1.txt f10.txt f2.txt which doesn't seem right to me. What is the explanation? Can I get this by default: f1.txt f2.txt f10.txt
4
votes
3answers
1k views

Sort using custom pattern

Is there a way to output file's contents using custom patterns? For instance, having a file myfile with following contents: a d b c ..how would one sort it using following pattern: print lines ...
4
votes
3answers
510 views

Finding files on a Unix server and sorting by size

I have a web server where I have run out of space and it's causing issues with the WordPress sites I am running on them. I know that I have a lot of large .png files (the fact that it's PNG is a ...
4
votes
2answers
604 views

Fastest `uniq` tool in linux

I have large text file (1.5 G), I want to know what the fastest and more reliable tool in linux is. I usualy use: awk '!x[$0]++' file.txt But when I use htop command I see my memory usage is ...
4
votes
3answers
6k views

Get lines with maximum values in the column using awk, uniq and sort

I have a file with next format 2011-12-01 user1 access1 2011-12-01 user1 access2 2011-12-01 user2 access2 2011-12-01 user4 access2 2011-12-02 user1 access1 2012-01-01 user3 access1 2012-01-01 user4 ...
4
votes
3answers
7k views

How to find files in subdirs and sort them by filename in a single command?

Result of a normal find using find . ! -path "./build*" -name "*.txt": ./tool/001-sub.txt ./tool/000-main.txt ./zo/001-int.txt ./zo/id/002-and.txt ./as/002-mod.txt and when sorted with sort -n: ...
4
votes
2answers
528 views

sort part of a file

How can I sort the file by the second column from the terminal. The content of the file is: Nome Note ------------ Mehdi 0 Sunday 20 Others 10 Shnou 5 Using sort -t' ' -nk2 is not ...
4
votes
4answers
125 views

Stable sort a file by presence of ordered substrings on each line

I have a list of sound sources that I'm processing with a script. An example would be: alsa_input.usb-AVEO_Technology_Corp._USB2.0_Camera-02-Camera.analog-mono ...
4
votes
3answers
531 views

Sort all directories based on their size

I'd like to sort all the directories/files in a specific directory based on their size (using du -sh "name"). I need to apply this command to all directories in my location, then sort them based on ...
4
votes
1answer
4k views

How to numerical sort by last column?

I have this input: sdkxyosl 1 safkls 2 asdf--asdfasxy_asd 5 dkd8k jasd 29 sdi44sw 43 asasd afsdfs 10 rklyasd 4 I need this output: sdi44sw 43 dkd8k jasd 29 asasd afsdfs 10 asdf--asdfasxy_asd 5 ...
4
votes
1answer
22k views

Sort data in descending order of first column, for equal values, use second column in ascending order

Allow me to clarify: Assume I have some keywords with frequency of their usage: 12 Hi 7 Hash 7 C++ 9 Superuser 17 Stackoverflow 9 LaTeX 42 Life 9 Ubuntu What I want, is to sort this data ...
4
votes
1answer
3k views

How to find out common elements between two files?

For an example, I have 2 files having following info: File #1: 12 13 14 15 File #2: 12 1 13 2 14 2 15 6 16 7 17 8 Output File: 1 2 2 6 In the output file, I want only the second column ...
4
votes
4answers
1k views

Sort input file by the results of a regex

I'd like to sort a file based on the results of a regex expression. For example, if I have the following property declarations in Obj-C @property (nonatomic, strong) id <AlbumArtDelegate, ...
4
votes
1answer
1k views

Linux sort second last column

I would like know how to sort the second last column ? It has the word "days=" in front of the number. I'm able to get the column with awk '{print $(NF-1)}', but sorting is wrong. 457000 ...
4
votes
3answers
105 views

Find largest files from each date in a directory

I have a directory similar to the following: -rw-r--r-- 1 root root 223K Apr 28 14:25 2015.04.28_14.25 -rw-r--r-- 1 root root 253K Apr 28 14:55 2015.04.28_14.55 -rw-r--r-- 1 root root 276K Apr 28 ...
4
votes
4answers
300 views

Sorting directories by last modified date/time of the same-named contained file

Is it possible, using grep, find, etc, to sort a list of directories by the last modified date of the same-named file (e.g., file.php) within? For example: domain1/file.php (last modified 20-Jan-2014 ...
4
votes
4answers
265 views

how to sort access log efficiently in blocks

Access logs are more or less sorted by time, but to aggregate connections by time (uniq -c), you need to sort them a bit more. For a huge access log sort is very inefficient, because it buffers and ...
4
votes
1answer
486 views

Renaming files from bash in linux

There is a bunch of files of the following format hh-mm-ss.png where hh, mm, ss are hour, minute and second. How can I rename all these files in the directory in a sequential way from 1.png til n.png ...
4
votes
2answers
150 views

how to sort by the day of the week?

I have two files. file1: Dave 734.838.9800 Bob 313.123.4567 Carol 248.344.5576 Mary 313.449.1390 Ted 248.496.2204 Alice 616.556.4458 file2: Bob Tuesday Carol Monday Ted Sunday ...
4
votes
2answers
5k views

Sort and count number of occurrence of lines

I have Apache logfile, access.log, how to count number of line occurrence in that file? for example the result of cut -f 7 -d ' ' | cut -d '?' -f 1 | tr '[:upper:]' '[:lower:]' is a.php b.php a.php ...
4
votes
2answers
116 views

Sorting strings with ANSI escape codes

I am trying to print a sorted list of all zsh options, with set options colored green and unset options colored red. I cannot get sort to work properly with on the colored lines though.The below ...
4
votes
1answer
1k views

How do get unix sort to sort in same order as Java (by unicode value)

I shell out sorting to the unix sort command in a Java program I've written. However I am having problems arising from Java's string comparison behaving differently than the comparisons done by sort. ...
4
votes
2answers
1k views

Sort behaves strangely on case sensitive sorting

Sort is sorting differently than I would expect. I have this file, call it text.txt: a 1 A 1 a 11 (the space is always one \t) I want to sort them alphabetically by the first column. ...
4
votes
2answers
48 views

How to sort a collection of lines from different files?

I want to sort second line in each file by second column, and also print name of the corresponding file. I am doing it like this - rm tmp; for filename in file*; do num=`head -2 $filename | tail ...
4
votes
1answer
95 views

Help with the sort command (numeric)

I have this output I'd like to sort numerically by port (starting at the 35th column): tcp 0 0 192.168.0.210:110 0.0.0.0:* LISTEN 3385/dovecot tcp 0 0 ...
4
votes
1answer
1k views

How can I sort a list of file names by some substring of the name?

I have a number of files with names like: beginning_num1_734_num2_1.363e+12_num3_800.pdf beginning_num1_735_num2_7.453e+13_num3_800.pdf beginning_num1_1007_num2_9.453e+12_num3_1200.pdf I'd like to ...
4
votes
1answer
496 views

Why is uniq ignoring Unicode and lines with a single letter?

I'm trying to combine both the American and British dictionaries into one large dictionaries, and I'm trying to remove all the duplicates from the superset, but it seems that uniq is not outputting ...
4
votes
2answers
870 views

Sort lines by unicode value

I'm trying to sort a text file linewise by their Unicode values. As far as I can tell, this means numerals first, then letters, then CJK-Ideographs. However, sort (with LC_ALL=C) fails horribly at ...
4
votes
1answer
849 views

Sort command not working as expected

I have got a dataset like this tack2@domain.com,2009-11-27 overflow@domain2.com,2009-11-27 overflow@domain2.com,2009-11-27 When I am running command to delete all of the same entries of column2 ...
4
votes
1answer
237 views

How to sort an associative array and retain the keys?

I have an array with filenames as keys and a numerical value as values. MYARRAY[00001.jpg] = 31 MYARRAY[00002.jpg] = 200 MYARRAY[00003.jpg] = 98 I need to sort them so they are ordered by value. ...
4
votes
1answer
637 views

How can I find the sort order for ls with my locale?

I was using ls -l on a directory, and was surprised that spaces and underscores were ignored for the sort order. For example, $ echo $LANG en_AU.UTF-8 $ ls -l total 0 -rw-r--r-- 1 sparhawk sparhawk 0 ...
3
votes
3answers
332 views

sort lines according to day

I have a text file in following format, and I need to sort the lines according to date. Is there some simple way to do it (preferably in bash) ? 2013-May-30 2 2013-May-21 10 2013-Jun-27 8 ...
3
votes
2answers
410 views

How to use output of one command as input in another command

I made a copy of all the backups made on Wed of every week The time stamp of each file is not sorted but the day is Wednesday of every file Now, I need to sort each file based on time Stamp e.g if ...