Sorting text files or other data, for example with the sort utility.

learn more… | top users | synonyms

4
votes
1answer
70 views

How to get this collation in sort?

I need to set up command sort, so it is sorting lines into this order: Akira Kurosawa Al Ernest Garcia Alain Sarde Alan Ladd, Jr. Albert S. Ruddy Alberto Grimaldi Alfred Hitchcock Amanda Plummer ...
8
votes
2answers
235 views

Why non-number record shows after “0” in sorting?

I want to sort files according to the number in the filename. Here are the files: $ ls *.f 0.f 13.f 1.f 22.f 4.f abc.f The sorting result: $ ls *.f | sort -t. -k1n 0.f abc.f # note this file! ...
0
votes
1answer
51 views

ls default sorting inconsistent

Why does adding identical characters to the end of filenames change the sort order as shown below? In any sane string comparison method I would expect the comparison of 2 strings to not differ if the ...
1
vote
2answers
34 views

Find out most frequent strings in section of file

I have a file that looks like this: o99 ksad<tab>abcde $%!H$!(<tab>00 !sdaf as<tab>12 sarwa &!§! &% (where <tab> represents a real TAB character). For clarification: It ...
4
votes
2answers
171 views

concatenate n lines with sed

Recently, I asked a question how to sort pairs of lines, and one of the answers suggested concatenating lines with sed, like this: cat file.txt | sed -n 'N;s/\n//;p' | sort -t";" -k43,43n | perl ...
3
votes
2answers
294 views

Unix Multiple Column Sort Issue

I am trying to sort a file: TX001;A0004|Number|5|0|Y|1 TX001;A0110|VARCHAR|5|0|Y|4 TX001;A1332|VARCHAR|15|0|Y|3 TX001;B3321|VARCHAR|15|0|Y|5 ...
3
votes
1answer
168 views

Sorting the output of ps does not work

I want to sort the output of the following command based on pcpu: ps -p 29492 -L -o pid,tid,psr,pcpu I tried the following: ps -p 29492 -L -o pid,tid,psr,pcpu --sort=pcpu But it doesn't seem to ...
1
vote
0answers
55 views

Removing duplicates in a large text list

I've searched around the internet and stackexchange for this. Even though there are lots of similar topics, I haven't found a solution yet. So, I have a quite large list (approx. 20GB), which ...
1
vote
4answers
286 views

How to sort files by part of the filename?

Given the files below: ABC38388.SC01.StatueGrade_MKP ABC38388.SC02.Statue_GKP DEF38389.SC03.Statue_HKP XYZ38390.SC00.Statue_WKP How can I list them all based on the SC value, like this: ...
2
votes
2answers
80 views

sort file but keep pairs of lines together

I have a file, with entries of the structure a1;b1;c1; d1;e1;f1;g1 a2;b2;c2; d2;e2;f2;g2 etc. I want to sort this file via the g's, but keep the abc's with the corresponding g. Is there any nice ...
1
vote
3answers
117 views

How to sort blocks of lines with a specific line from the block?

I have a file with the data like this BEGIN hello2 5 world1 END BEGIN hello4 2 world5 END BEGIN hello6 4 END I want to sort the lines in the following manner based on the number inside the block. ...
1
vote
1answer
43 views

sort ignores mode for second key

I am trying to sort a file that contains entries like this. This is the output of head file.txt | sort -n 1 1 360030.650313113 23.6496749206273 1 2 184005.303709890 ...
-1
votes
2answers
55 views

Sorting multiple line columns

In the text: 35 EAST 23rd Street SOUTH AFRICA 5 600 5000000 6 83 NORTH YELLOWLIGHT AVENUE SOUTH AFRICA 4 700 7000000 5 777 NEW AVENUE SAUDIA ...
2
votes
1answer
60 views

sort behaves weirdly with scientific notation

I have a file with semicolon separated fields, which I want to sort according to general numeric value of the 26th column. I tried this: cat file.txt | grep -v setch | sort -t";" -k26 -g The grep ...
0
votes
1answer
75 views

sort and uniq commands not running as expected when run though cron

Below is a part of script which gives proper output when run manually but gives incorrect output when run using cron: sort < file1.out | uniq -ic |sort -nr> file2.out When run on the command ...
0
votes
1answer
58 views

how to get latest file in each dir

Is there a Unix command to get the latest file in logsarchive of each dir of /app/instance? In the example below, I have 5 folders in /app/instance: as_a, as_b, as_c, aunz, naeu and I need to get the ...
0
votes
1answer
66 views

bash: How to determine the largest number (newest version) from: ls *.tar.gz | cut -c 2-4

Hi I'm trying to get the largest number (newest version) of a file, I got a bunch of tar.gz files in a directory, versions of some software in the format v${version_number}_software.tar.gz, where ...
1
vote
0answers
21 views

sort behavior when using numerical option: “sort -k2,2 -nk6,6 foo” vs “sort -k2,2 -k6,6n foo”

The two examples in the title produce very different results. The first (sort -k2,2 -nk6,6 foo) brings back the following results: 153759 townhome 900 2 1 79000 876543 townhome 745 2 1 79000 222999 ...
3
votes
1answer
137 views

Unix - count unique IP addresses, sort them by most frequent and also sort them by IP when number of repetitions is same

I have a list of IP addresses in file: 72.204.55.250 72.204.55.250 72.204.55.250 72.204.55.250 72.204.55.250 96.41.51.202 208.115.113.91 178.137.94.166 178.137.94.166 ...
2
votes
1answer
87 views

How to sort text file by column starting at certain character position? [duplicate]

Considering the following input file (example)... afaefkl aaa al kmm ddd aajwoe bbb bnm aefp ccc ... what would be the most straight-forward way to sort it by column 2 (starting at ...
-1
votes
4answers
24 views

sort and du together to list file sizes but the output unit is wrong

So I have a large list of files to sort by sizes and this is what I got so far: du -ha /local | sort -n -r | head -n 10 1020K /local/dir/more/dir/filesnames.blabla 1020K ...
2
votes
2answers
50 views

Combine columns using awk? (Or other suggestions)

I'm trying to be a bit effective and need a script or command solution. Say I make a file with 2 columns, or two files with one column, whichever is easier: AA1 B2 ZZ1 YYY XX1 AA2 B2 ...
3
votes
3answers
56 views

Sorting more than one column

I have a file which has content that looks like this: 19.58 1925 Alpha 20.40 1924 Otter 13.66 1920 Gold I am trying to sort it "column by column" to output the following: 13.40 1920 Alpha 19.58 ...
5
votes
1answer
219 views

Sort output of awk except for first line?

This is a use case I am continually running into in parsing CSV files. When it is an inline awk script embedded in a shell script, I can use this workaround: myfile=$(mktemp) awk '(awk script here)' ...
0
votes
0answers
53 views

How to find 100 GB of most recently modified files and send to rsync for local synchronization

I need to select 100gig of most recently changed files from a local directory and do an rsync to a local directory (skipping files that are already rsynced). If I try something like find . -mtime ...
0
votes
1answer
36 views

Using `tree`, how do I output files before subdirectories?

I tried to combine --dirsfirst and -r, but directories still show up on top, only in reverse alphabetical order. It seems like -r is applied first, which is also indicated by the man pages. Any other ...
0
votes
3answers
94 views

How to list all files *.html bigger than {size} excluding some {*parts*}

I need a command to list files. But I have three problems: I want files ending with .HTML and .HTM If any file has parts like: google docs or form in the name, I want to exclude it. I want just ...
0
votes
0answers
85 views

Join, file is not sorted

I have two files whose first lines are File1 Locus_1174 [six columns of numbers] locus_6396 [six columns of numbers] Locus_12135 [six columns of numbers] locus_13689 [six columns of ...
1
vote
2answers
68 views

Reordering a file, based on a second file

I have a file that is out of order.I would like to reorder it by using a second file. File 1 1 ...
2
votes
2answers
328 views

How does the utility “sort” work? (why does it not work the way I think it should?) [duplicate]

I have 2 text files: $ cat /tmp/test1 "AAP" bar "AEM" bar "AA" bar "AEO" bar "A" bar $ cat /tmp/test2 "AEM" foo "AAP" foo "A" foo "AEO" foo "AA" foo I want to sort them $ sort /tmp/test1 "AA" bar ...
1
vote
1answer
53 views

Why does this command not sort based on the uniq count?

I have lines in a log similar to: 2015/11/02-07:55:39.735 INFO failed with ERR_AUTHORIZATION_REQUIRED. (10.10.10.11:61618) is not a trusted source. 2015/11/02-07:55:40.515 INFO failed with ...
3
votes
1answer
51 views

How to list files named like XXXmmyy.dat in natural chronological order?

I use a Mac OS X (V10.9.5) and work in the X11 window, and I often need to deal with thousands of files named like XXXmmyy.dat, where XXX is a 3-letter acronym of a place, mm is the 2-digit month, and ...
0
votes
1answer
59 views

How do I sort a file and output the latest record in case duplicates exist?

I save all the requests, received via API, in a file. I wish to process only the latest request if duplicate requests arrive. Sort command is not helping, I am using : sort -r -k 1.69,1.88 ...
2
votes
1answer
68 views

“Count how many different starting letters are in languages names”

So I have this text document with 89 lines. In this document the first column of every line is another language. My task is to count how many different starting letters are used in the document. I ...
1
vote
2answers
36 views

Position of r before and after k in Sort

I have tried the following two scripts to sort a CSV file $ sort -t"," -k1,1 -k3,3 -k4,4 -k6,6r myFile.csv and $ sort -t"," -k1,1 -k3,3 -k4,4 -rk6,6 myFile.csv I found having the -r before ...
3
votes
1answer
80 views

Strange default sort behaviour

I have some trouble to understand what is happening here: [guido@localhost 9]$ ls -1 Star\ Wars\ Episode\ * Star Wars Episode II Attack of the Clones.avi Star Wars Episode III Revenge of the Sith.avi ...
0
votes
1answer
36 views

Keeping first instance of duplicates when column values are interchangeable

I have previously asked a question about keeping the first instance of duplicates and the solution provided has been working perfectly until now (see Keeping first instance of duplicates). However, I ...
0
votes
3answers
58 views

Understanding options to sort

part 1 ps -e | sort -k 1 -nr What is the interpretation of the above command? I want to understand the role of -nr. It is not clearly given the man pages of sort. part 2 When you open the man ...
0
votes
0answers
29 views

grep names from big list and sort it according to pattern [duplicate]

I have 2 big files as shown below. I want to read the pattern file and grep the line from full-info file and sort the channel name order according to pattern. pattern file HBO COMEDY; DTV CABLE ...
0
votes
1answer
41 views

How to find maximum number of different strings?

I have a file: 2.0-00-04 2.0-00-03 2.0-00-02 2.0-00-01 2.0-00-00 1.0-00-02 1.0-00-01 1.0-00-00 How can I find maximum number of each group? In the string above, output should be: 2.0-00-04 ...
0
votes
0answers
25 views

Properly sorting a csv file [duplicate]

The sort function looked promising, but as far as I know, it can't handle escaped line feeds, and the field deliminator option -t doesn't have an option that will take into account escaped commas. ...
4
votes
1answer
122 views

Non-standard sorting of the ls output

I usually include a date in the filename so that files and directories are listed in chronological order when I use the ls command, e.g. 2015-08-29_letter_to_santa.txt. A drawback is that ...
2
votes
1answer
119 views

Wrong order with sort command

How to get sort command working with data separated by "|" (pipe character) ? Consider this sample file called "x" which have two fields: axis-AXIS|OK axis-AXISR|OK axis-AXISD2X|OK axis-AXISA|OK ...
0
votes
0answers
28 views

Sorting different groups (separated by a line) based on 16th column

I have a CSV file which is in this format cluster 0 aaa bbb ccc ....... xxx yyy zzz bhc adf ghy ....... hjn dft cbn cluster 1 adg bhg klo ....... jio jik mkl asf ghb nbv ....... cvb xcz hga ... ...
1
vote
3answers
456 views

Select and sort IP address keeping the whole line

So I need to sort the IP Addresses and then sort my line by them. I can sort IP addres in a file by using : sort -n -t . -k1,1 -k2,2 -k 3,3 -k4,4 If my file looks like : add method1 a.b.c.d ...
13
votes
2answers
367 views

Fastest way to delete duplicates in large wordlist?

I need to deduplicate a large wordlist. I tried several commands and did some research here and here where they explain that the fastest way to deduplicate a wordlist seems to be using awk. awk ...
12
votes
4answers
747 views

Sort lines by number of words per line

Given input: hello: world foo bar baz bar: baz: bin boop bop fiz bang beep bap: bim bam bop boatkeeper: poughkeepsie I would like to sort it into most words at the top, to least at the end, like ...
7
votes
2answers
190 views

Sort command inconsistent behavior

I want to sort two files but I cannot get consistent results. It seems there are problems with collation but I cannot understand the reason. In sample files separator is a single space: file1: a b B ...
1
vote
1answer
83 views

“sort -h” not working in Redhat 5.9. Any way to upgrade it?

I'm trying to sort the folders disk usage in human readable format in RHEL 5.9. I mounted the drive as NFS in Cent OS 6.4, where sort -h option works fine. But in RHEL 5.9, it throws error as ...
6
votes
3answers
425 views

Print random lines respecting the order of the source file

I have a large text file (~500K lines) with short sentences (couple of words long). Additionally, there is some XML markup in most of the lines. Finally, the text file has been sorted before the ...