Sorting text files or other data, for example with the sort utility.

learn more… | top users | synonyms

3
votes
2answers
292 views

Unix Multiple Column Sort Issue

I am trying to sort a file: TX001;A0004|Number|5|0|Y|1 TX001;A0110|VARCHAR|5|0|Y|4 TX001;A1332|VARCHAR|15|0|Y|3 TX001;B3321|VARCHAR|15|0|Y|5 ...
3
votes
1answer
162 views

Sorting the output of ps does not work

I want to sort the output of the following command based on pcpu: ps -p 29492 -L -o pid,tid,psr,pcpu I tried the following: ps -p 29492 -L -o pid,tid,psr,pcpu --sort=pcpu But it doesn't seem to ...
1
vote
0answers
49 views

Removing duplicates in a large text list

I've searched around the internet and stackexchange for this. Even though there are lots of similar topics, I haven't found a solution yet. So, I have a quite large list (approx. 20GB), which ...
1
vote
4answers
237 views

How to sort files by part of the filename?

Given the files below: ABC38388.SC01.StatueGrade_MKP ABC38388.SC02.Statue_GKP DEF38389.SC03.Statue_HKP XYZ38390.SC00.Statue_WKP How can I list them all based on the SC value, like this: ...
2
votes
2answers
78 views

sort file but keep pairs of lines together

I have a file, with entries of the structure a1;b1;c1; d1;e1;f1;g1 a2;b2;c2; d2;e2;f2;g2 etc. I want to sort this file via the g's, but keep the abc's with the corresponding g. Is there any nice ...
1
vote
3answers
95 views

How to sort blocks of lines with a specific line from the block?

I have a file with the data like this BEGIN hello2 5 world1 END BEGIN hello4 2 world5 END BEGIN hello6 4 END I want to sort the lines in the following manner based on the number inside the block. ...
1
vote
1answer
43 views

sort ignores mode for second key

I am trying to sort a file that contains entries like this. This is the output of head file.txt | sort -n 1 1 360030.650313113 23.6496749206273 1 2 184005.303709890 ...
-1
votes
2answers
48 views

Sorting multiple line columns

In the text: 35 EAST 23rd Street SOUTH AFRICA 5 600 5000000 6 83 NORTH YELLOWLIGHT AVENUE SOUTH AFRICA 4 700 7000000 5 777 NEW AVENUE SAUDIA ...
2
votes
1answer
59 views

sort behaves weirdly with scientific notation

I have a file with semicolon separated fields, which I want to sort according to general numeric value of the 26th column. I tried this: cat file.txt | grep -v setch | sort -t";" -k26 -g The grep ...
0
votes
1answer
65 views

sort and uniq commands not running as expected when run though cron

Below is a part of script which gives proper output when run manually but gives incorrect output when run using cron: sort < file1.out | uniq -ic |sort -nr> file2.out When run on the command ...
0
votes
1answer
51 views

how to get latest file in each dir

Is there a Unix command to get the latest file in logsarchive of each dir of /app/instance? In the example below, I have 5 folders in /app/instance: as_a, as_b, as_c, aunz, naeu and I need to get the ...
0
votes
1answer
62 views

bash: How to determine the largest number (newest version) from: ls *.tar.gz | cut -c 2-4

Hi I'm trying to get the largest number (newest version) of a file, I got a bunch of tar.gz files in a directory, versions of some software in the format v${version_number}_software.tar.gz, where ...
1
vote
0answers
21 views

sort behavior when using numerical option: “sort -k2,2 -nk6,6 foo” vs “sort -k2,2 -k6,6n foo”

The two examples in the title produce very different results. The first (sort -k2,2 -nk6,6 foo) brings back the following results: 153759 townhome 900 2 1 79000 876543 townhome 745 2 1 79000 222999 ...
3
votes
1answer
129 views

Unix - count unique IP addresses, sort them by most frequent and also sort them by IP when number of repetitions is same

I have a list of IP addresses in file: 72.204.55.250 72.204.55.250 72.204.55.250 72.204.55.250 72.204.55.250 96.41.51.202 208.115.113.91 178.137.94.166 178.137.94.166 ...
2
votes
1answer
81 views

How to sort text file by column starting at certain character position? [duplicate]

Considering the following input file (example)... afaefkl aaa al kmm ddd aajwoe bbb bnm aefp ccc ... what would be the most straight-forward way to sort it by column 2 (starting at ...
-1
votes
4answers
20 views

sort and du together to list file sizes but the output unit is wrong

So I have a large list of files to sort by sizes and this is what I got so far: du -ha /local | sort -n -r | head -n 10 1020K /local/dir/more/dir/filesnames.blabla 1020K ...
2
votes
2answers
50 views

Combine columns using awk? (Or other suggestions)

I'm trying to be a bit effective and need a script or command solution. Say I make a file with 2 columns, or two files with one column, whichever is easier: AA1 B2 ZZ1 YYY XX1 AA2 B2 ...
3
votes
3answers
56 views

Sorting more than one column

I have a file which has content that looks like this: 19.58 1925 Alpha 20.40 1924 Otter 13.66 1920 Gold I am trying to sort it "column by column" to output the following: 13.40 1920 Alpha 19.58 ...
5
votes
1answer
206 views

Sort output of awk except for first line?

This is a use case I am continually running into in parsing CSV files. When it is an inline awk script embedded in a shell script, I can use this workaround: myfile=$(mktemp) awk '(awk script here)' ...
0
votes
0answers
50 views

How to find 100 GB of most recently modified files and send to rsync for local synchronization

I need to select 100gig of most recently changed files from a local directory and do an rsync to a local directory (skipping files that are already rsynced). If I try something like find . -mtime ...
0
votes
1answer
35 views

Using `tree`, how do I output files before subdirectories?

I tried to combine --dirsfirst and -r, but directories still show up on top, only in reverse alphabetical order. It seems like -r is applied first, which is also indicated by the man pages. Any other ...
0
votes
3answers
81 views

How to list all files *.html bigger than {size} excluding some {*parts*}

I need a command to list files. But I have three problems: I want files ending with .HTML and .HTM If any file has parts like: google docs or form in the name, I want to exclude it. I want just ...
0
votes
0answers
81 views

Join, file is not sorted

I have two files whose first lines are File1 Locus_1174 [six columns of numbers] locus_6396 [six columns of numbers] Locus_12135 [six columns of numbers] locus_13689 [six columns of ...
1
vote
2answers
65 views

Reordering a file, based on a second file

I have a file that is out of order.I would like to reorder it by using a second file. File 1 1 ...
2
votes
2answers
324 views

How does the utility “sort” work? (why does it not work the way I think it should?) [duplicate]

I have 2 text files: $ cat /tmp/test1 "AAP" bar "AEM" bar "AA" bar "AEO" bar "A" bar $ cat /tmp/test2 "AEM" foo "AAP" foo "A" foo "AEO" foo "AA" foo I want to sort them $ sort /tmp/test1 "AA" bar ...
1
vote
1answer
51 views

Why does this command not sort based on the uniq count?

I have lines in a log similar to: 2015/11/02-07:55:39.735 INFO failed with ERR_AUTHORIZATION_REQUIRED. (10.10.10.11:61618) is not a trusted source. 2015/11/02-07:55:40.515 INFO failed with ...
3
votes
1answer
51 views

How to list files named like XXXmmyy.dat in natural chronological order?

I use a Mac OS X (V10.9.5) and work in the X11 window, and I often need to deal with thousands of files named like XXXmmyy.dat, where XXX is a 3-letter acronym of a place, mm is the 2-digit month, and ...
0
votes
1answer
56 views

How do I sort a file and output the latest record in case duplicates exist?

I save all the requests, received via API, in a file. I wish to process only the latest request if duplicate requests arrive. Sort command is not helping, I am using : sort -r -k 1.69,1.88 ...
2
votes
1answer
68 views

“Count how many different starting letters are in languages names”

So I have this text document with 89 lines. In this document the first column of every line is another language. My task is to count how many different starting letters are used in the document. I ...
1
vote
2answers
35 views

Position of r before and after k in Sort

I have tried the following two scripts to sort a CSV file $ sort -t"," -k1,1 -k3,3 -k4,4 -k6,6r myFile.csv and $ sort -t"," -k1,1 -k3,3 -k4,4 -rk6,6 myFile.csv I found having the -r before ...
3
votes
1answer
79 views

Strange default sort behaviour

I have some trouble to understand what is happening here: [guido@localhost 9]$ ls -1 Star\ Wars\ Episode\ * Star Wars Episode II Attack of the Clones.avi Star Wars Episode III Revenge of the Sith.avi ...
0
votes
1answer
36 views

Keeping first instance of duplicates when column values are interchangeable

I have previously asked a question about keeping the first instance of duplicates and the solution provided has been working perfectly until now (see Keeping first instance of duplicates). However, I ...
0
votes
3answers
57 views

Understanding options to sort

part 1 ps -e | sort -k 1 -nr What is the interpretation of the above command? I want to understand the role of -nr. It is not clearly given the man pages of sort. part 2 When you open the man ...
0
votes
0answers
29 views

grep names from big list and sort it according to pattern [duplicate]

I have 2 big files as shown below. I want to read the pattern file and grep the line from full-info file and sort the channel name order according to pattern. pattern file HBO COMEDY; DTV CABLE ...
0
votes
1answer
40 views

How to find maximum number of different strings?

I have a file: 2.0-00-04 2.0-00-03 2.0-00-02 2.0-00-01 2.0-00-00 1.0-00-02 1.0-00-01 1.0-00-00 How can I find maximum number of each group? In the string above, output should be: 2.0-00-04 ...
0
votes
0answers
24 views

Properly sorting a csv file [duplicate]

The sort function looked promising, but as far as I know, it can't handle escaped line feeds, and the field deliminator option -t doesn't have an option that will take into account escaped commas. ...
4
votes
1answer
121 views

Non-standard sorting of the ls output

I usually include a date in the filename so that files and directories are listed in chronological order when I use the ls command, e.g. 2015-08-29_letter_to_santa.txt. A drawback is that ...
2
votes
1answer
110 views

Wrong order with sort command

How to get sort command working with data separated by "|" (pipe character) ? Consider this sample file called "x" which have two fields: axis-AXIS|OK axis-AXISR|OK axis-AXISD2X|OK axis-AXISA|OK ...
0
votes
0answers
28 views

Sorting different groups (separated by a line) based on 16th column

I have a CSV file which is in this format cluster 0 aaa bbb ccc ....... xxx yyy zzz bhc adf ghy ....... hjn dft cbn cluster 1 adg bhg klo ....... jio jik mkl asf ghb nbv ....... cvb xcz hga ... ...
1
vote
3answers
386 views

Select and sort IP address keeping the whole line

So I need to sort the IP Addresses and then sort my line by them. I can sort IP addres in a file by using : sort -n -t . -k1,1 -k2,2 -k 3,3 -k4,4 If my file looks like : add method1 a.b.c.d ...
13
votes
2answers
354 views

Fastest way to delete duplicates in large wordlist?

I need to deduplicate a large wordlist. I tried several commands and did some research here and here where they explain that the fastest way to deduplicate a wordlist seems to be using awk. awk ...
12
votes
4answers
722 views

Sort lines by number of words per line

Given input: hello: world foo bar baz bar: baz: bin boop bop fiz bang beep bap: bim bam bop boatkeeper: poughkeepsie I would like to sort it into most words at the top, to least at the end, like ...
7
votes
2answers
183 views

Sort command inconsistent behavior

I want to sort two files but I cannot get consistent results. It seems there are problems with collation but I cannot understand the reason. In sample files separator is a single space: file1: a b B ...
1
vote
1answer
78 views

“sort -h” not working in Redhat 5.9. Any way to upgrade it?

I'm trying to sort the folders disk usage in human readable format in RHEL 5.9. I mounted the drive as NFS in Cent OS 6.4, where sort -h option works fine. But in RHEL 5.9, it throws error as ...
6
votes
3answers
418 views

Print random lines respecting the order of the source file

I have a large text file (~500K lines) with short sentences (couple of words long). Additionally, there is some XML markup in most of the lines. Finally, the text file has been sorted before the ...
2
votes
2answers
101 views

How can I list files by class/indicator (as in ls --classify) then by name?

ls --clasify append indicator (one of */=>@|) to entries. I would like to group the entries of the same class/indicator together. I can only find ls --group-directories-first for grouping the / ...
2
votes
5answers
282 views

How to sort file by character occurrences per line?

I'm quite new to Linux, and I've found quite a bit of useful information on how to do character counts in a file, but is there a way in Linux/terminal to sort a text file by the number of times a ...
1
vote
1answer
40 views

Group by Functionality on Mac Address

I'm looking for a solution that should do grouping of identical mac address(device ids). I tried using sort -k 2, but I'm not confident of the result. Data Set looks like: 1616.93,ac:22:0b:a6:22:c3 ...
7
votes
2answers
259 views

Sorting a file with integer in parentheses with no space

I am trying to sort a file like this (which is a list of subroutine names) cfn=(1370) __lib_file_MOD_file_open fn=(1368) __universe_MOD_general_boot fn=(916) __universe_MOD_general_main fn=(6142) ...
0
votes
2answers
212 views

Display filenames with grep

I'm looking for a way to display the filename when using this command: cat *.tcp | grep "tcp" | grep "open" | sort | uniq Is there a way to do it?