Sorting text files or other data, for example with the sort utility.

learn more… | top users | synonyms

2
votes
1answer
71 views

Sort doesn't use natural order

I have used uniq -c and sort extensively on other machines, and it always worked very well. Now, on my new Ubuntu (13.04) machine, I end up with a weird ordering: % cat queries.log | grep INSERT | ...
5
votes
5answers
276 views

Sorting blocks of lines

I have a file that contains 4n lines. Here is an excerpt from it containing 8 lines 6115 8.88443 6116 6.61875 6118 16.5949 6117 19.4129 6116 6.619 6117 16.5979 6118 19.4111 6115 8.88433 What I ...
10
votes
4answers
527 views

Shuffle file randomly with some additional constraints

I have a huge music playlist and, while some artists have many albums, others have just one song. I wanted to sort the playlist so the same artist won't play twice in a row, or his songs won't end up ...
0
votes
1answer
72 views

Why this sort command gives me an empty file? [duplicate]

prova1 is a file with some text. When I run this code prova1 become empty: sort prova1 > prova1 I know that to achieve the effect I can do: sort prova1 -o prova1 I would just know the reason ...
2
votes
2answers
97 views

When do `sort -k 2,3b` and `sort -k 2,3` differ?

The POSIX description of the -b flag for the sort command is Ignore leading characters when determining the starting and ending positions of a restricted sort key. I can understand the use for ...
-1
votes
1answer
118 views

Select a range of files from directory structure /year/month/date.db

I have databases stored in the directory structure year/month/date.db an example would be db/2013/10/15.db I need to be able to select a range of these with a script ("dbSelect ...
25
votes
2answers
746 views

Why is sort -o useful?

UNIX philosophy says: do one thing and do it well. Make programs that handle text, because that is a universal interface. The sort command, at least GNU sort, has an -o option to output to a file ...
1
vote
4answers
140 views

Why are sort options -c (check if sorted) and -o (write to output file) not compatiable

I am sorting a large number of files one after another. Using sort -o file file. The for loop halted midway and before restarting I want to use sort's -c option to speedup sorting. But it says that ...
0
votes
1answer
65 views

What option(s) would you use to sort a file named foo in place, that is replacing the original file with the sorted version? [duplicate]

I have this homework question. I looked at the man page for sort and I believe the answer is -o, --output=FILE Is that correct? It is a Fedora server. We're supposed to look on man pages, if there ...
6
votes
4answers
714 views

uniq a csv file ignoring a column, awk maybe?

Given this file (annotations are not part of file, but form part of explanation)... x,a,001,b,c,d,y x,a,002,b,c,e,yy x,bb,003,b,d,e,y x,c,004,b,d,e,y x,c,005,b,d,e,y # nb - dupe of row 4 ...
4
votes
1answer
73 views

Help with the sort command (numeric)

I have this output I'd like to sort numerically by port (starting at the 35th column): tcp 0 0 192.168.0.210:110 0.0.0.0:* LISTEN 3385/dovecot tcp 0 0 ...
2
votes
2answers
909 views

Any tricks to speed up sort?

For example for bzip there is pbzip, a parallel version of bzip. Is there any such parallelization tools for sort to improve performance?
2
votes
3answers
1k views

How is it possible to sort ps command's CPU field?

I'm trying to make a small command that will find the processes that use the most CPU power. Firstly, I use ps aux > file.txt and then cut -c 16-20 file.txt | sort -n | tail -5. The result I get ...
0
votes
3answers
105 views

selecting rows that have the same value [closed]

I have a problem in selecting rows that have the same value. My data is too huge to go row by row to do that. I want you guys to let me know scripts that can perform this. My data looks like as ...
3
votes
1answer
342 views

How can I sort a list of file names by some substring of the name?

I have a number of files with names like: beginning_num1_734_num2_1.363e+12_num3_800.pdf beginning_num1_735_num2_7.453e+13_num3_800.pdf beginning_num1_1007_num2_9.453e+12_num3_1200.pdf I'd like to ...
4
votes
3answers
297 views

Grouped sorting of continuous paragraphs (separated by blank line)?

I think I'm pretty experienced now in sorting by columns; however, I haven't found anything so far how to sort continuous rows. Supposing we have a text file that looks like this: (very simplified, ...
0
votes
1answer
59 views

Get the most recent backup files [duplicate]

I'm googling for this, but the terms are so generic I'm getting way too many false positives, so what the hey, I'll ask here. I'm sure there's a simple answer but I don't generally script in *nix. ...
3
votes
3answers
175 views

sort lines according to day

I have a text file in following format, and I need to sort the lines according to date. Is there some simple way to do it (preferably in bash) ? 2013-May-30 2 2013-May-21 10 2013-Jun-27 8 ...
1
vote
2answers
4k views

How to sort results from ls command by modification date (directories first)?

I can fire ls -lrt to get files and folders sorted by modification date, but this does not separate directories from files. I want ls to show me first directories by modification date and then ...
5
votes
1answer
1k views

Sorting a file based on one column using Unix and Awk

I need to sort the input file according the 6th column, which is the score. Input File: Sc2/80 20 . A T 86 Pass N=2 F=5;U=4 Sc2/80 20 . A C 80 Pass N=2 F=5;U=4 Sc2/60 55 . G T 90 Pass N=2 F=5;U=4 ...
3
votes
2answers
94 views

Ignore blank keys in coreutils sort

How do you ignore or is there a way to change the sort order of empty keys when using sort -k? # cat afile a b 11 20 93 45 55 26 09 31 How can I sort the empty column b values at the bottom so it ...
1
vote
2answers
227 views

How batch mv files to folders created and named by modification date?

I am quite new to the terminal stuff and I just couldn't find any answer on the aforementioned subject. On my OS X, I have a folder with about 4000 pics that I'd like to move into folders that are ...
1
vote
2answers
402 views

Sort lines based on a variable-width number at a fixed position

I'm trying to sort a file using the Sort command but its not working: The file contains lines like: photo1 photo11 photo12 photo13 photo17 photo18 photo19 photo2 photo22 photo23 photo25 photo26 ...
1
vote
5answers
445 views

Need some sed magic: Moving marked lines to the beginning of a file

I find myself in dire need of some sed magic (I really need to sit myself down and learn this). I have a file with lots of lines. After reviewing it, I've marked some lines by adding a star (*) to ...
2
votes
1answer
208 views

Incorrect sorting according en_US.UTF-8

I'm seeing a weird behavior of the sort command when I sort by en_US.UTF-8. I'm trying to sort this file (LC_COLLATE=en_US.UTF-8 sort testdata): 㐀 㐃㐄 ▼▽◢ ㎏㎜㎝ bqz ㄠㄨㄩ─┃┆ ...
1
vote
0answers
32 views

Characters not defined into iso14651_t1_pinyin are not sorted properly

when I try to sort characters not defined in iso14651_t1_pinyin, no sort happens, they are simply left in place. I would expect to see them sorted according to their codepoint. Is there a way I can ...
3
votes
1answer
467 views

Can I unzip and merge sorted text files in a single operation?

I have tens of thousands of gzipped text files, each of which has already been sorted internally. I've written a script to unzip them (creating a copy) and then using sort -m to merge them. It all ...
0
votes
1answer
2k views

How to sort by multiple columns? [duplicate]

I have a tab-separated file, and I want to sort it by its columns 9, 14, and 16. By this I mean that all rows that agree at column 9 should be sorted by column 14, and among these, those that also ...
0
votes
1answer
140 views

How to sort date: YYYY.MM.DD. HH:MM?

INPUT: b|2013.06.03. 18:47| a|2013.06.03. 13:15| c|2013.06.03. 13:53| needed OUTPUT: a|2013.06.03. 13:15| c|2013.06.03. 13:53| b|2013.06.03. 18:47| How can I sort the dates correctly in ...
3
votes
1answer
106 views

Sort only on the second column [duplicate]

I'm trying to sort a file where typically (not always) the lines are whatever:[A-Ba-b0-9_]: values where fields are separated with :. Some lines do not follow this pattern, I'm not interested in ...
4
votes
2answers
269 views

Sort files by modification time

I've seen many questions and answers here using a construction along the lines of list_dir=`ls -t /path/to/dir/` for i in $list_dir; do or ls -t | while read i; do Now, I know that you shouldn't ...
4
votes
4answers
150 views

how to sort access log efficiently in blocks

Access logs are more or less sorted by time, but to aggregate connections by time (uniq -c), you need to sort them a bit more. For a huge access log sort is very inefficient, because it buffers and ...
20
votes
3answers
7k views

What is the difference between “sort -u” and “sort | uniq”?

Everywhere I see someone needing to get a sorted, unique list, they always pipe to sort | uniq. I've never seen any examples where someone uses sort -u instead. Why not? What's the difference, and why ...
3
votes
2answers
202 views

How to use output of one command as input in another command

I made a copy of all the backups made on Wed of every week The time stamp of each file is not sorted but the day is Wednesday of every file Now, I need to sort each file based on time Stamp e.g if ...
4
votes
3answers
424 views

Sort using custom pattern

Is there a way to output file's contents using custom patterns? For instance, having a file myfile with following contents: a d b c ..how would one sort it using following pattern: print lines ...
4
votes
2answers
267 views

sort part of a file

How can I sort the file by the second column from the terminal. The content of the file is: Nome Note ------------ Mehdi 0 Sunday 20 Others 10 Shnou 5 Using sort -t' ' -nk2 is not ...
3
votes
2answers
876 views

Specify the sort order with LC_COLLATE so lowercase is before uppercase

Given the file: $ cat file 1 a C B 2 c 3 A b By default sort will: $ sort file 1 2 3 a A b B c C With LC_COLLATE=C so will sort in uppercase letter before lowercase: $ LC_COLLATE=C sort file 1 ...
2
votes
6answers
663 views

sort ls output by users

On Linux, is there a way to sort ls output by users? What I try to achieve is something like this: user_a   file1 user_a   file2 user_b   another_file user_c   this_file user_c ...
1
vote
2answers
192 views

How to remove duplicate entries from a file?

I have a comma-separated list of domain-names in a file; unfortunately there are duplicate entries for some domain-names. Is there a way to find and remove the duplicate entries from the file? The ...
0
votes
3answers
384 views

Sorting IP address according to the second field in file

in my ksh script I need to add the following task ( OS - linux/solaris ) I have the following file more test.txt /etc/backup/app 172.1.120 172.110.120 /etc/backup/app 172.1.120.12 ...
1
vote
3answers
192 views

another question with sort in bash [duplicate]

How can I ignore the space in "Kuala Lumpur" when sorting this list? I cheated it by tabbing the columns and sorted on the tab which gives me the right results but I would like to know how to deal ...
1
vote
3answers
785 views

Sort with unequal whitespace in first column

I need to sort this list by name, high temp and low temp: Kuala Lumpur 78 56 Seoul 85 66 Karachi 95 75 Tokyo 85 60 Lahore 85 75 Manila 90 85 I figured since whitespace is the delimiter for a column ...
4
votes
2answers
429 views

Unix sort out of memory

I'm sorting a 25 GB file containing 200M lines on Fedora 18 using: sort -S 10G -T /bigdisk bigfile and I am getting process killed due to no memory. The process isn't maxing out my RAM before death ...
2
votes
2answers
307 views

Sort lines by unicode value

I'm trying to sort a text file linewise by their Unicode values. As far as I can tell, this means numerals first, then letters, then CJK-Ideographs. However, sort (with LC_ALL=C) fails horribly at ...
2
votes
0answers
35 views

Unexpected ordering of strings in `sort` command [duplicate]

I'm trying to sort a text file using certain columns in the file as a sort key. Since the file doesn't have separators and fields sometimes stick to each other, I have to convert spaces to some ...
1
vote
1answer
81 views

Sort aligned columns

I have a program which output looks like this # Status VAL1 VAL2 Description 1 some text 2.5 4.4 Some long other text 5 some text 2 14.5 2.0 Some long other ...
2
votes
2answers
158 views

Select greatest numbered filename

Simple requirement but can't find anything online which can achieve it. I have a list of dated files as below... filename_20120101.dat filename_20120102.dat filename_20120103.dat I ...
1
vote
3answers
386 views

Reading a directory in shell script

I want to read a numbered directory which is under a path, into a variable in shell script. The paths are something like this: .../releases/R1/... .../releases/R2/... .../releases/R3/... Each time ...
3
votes
2answers
180 views

Delete duplicate images. Need Software for computing average color of an image

I want to delete duplicates in a massive bunch of images. Well as I have dups of the same picture in a different resolution I will make the deletion myself. BUT I want to do this in linear time. So I ...
1
vote
1answer
1k views

How to match 2 large file and print the difference in shell script

I have 2 files. File1 has 400k numerical records. e.g: 1 2 3 4 5 6 .. and so on File 2 also has 420k numerical records. e.g: 1 2 3 4 6 .. and so on Both these file are in unsorted manner. I want ...