Sorting text files or other data, for example with the sort utility.

learn more… | top users | synonyms

55
votes
5answers
51k views

What is the difference between “sort -u” and “sort | uniq”?

Everywhere I see someone needing to get a sorted, unique list, they always pipe to sort | uniq. I've never seen any examples where someone uses sort -u instead. Why not? What's the difference, and why ...
1
vote
3answers
124 views
2
votes
1answer
146 views

sort based on column

10 awhdk; 14 hjoeow; 2 kdkld; 4 jkjdksl How to sort this based on column one (i.e the count) ? I'd want to get this output : 14 hjoeow;10 awhdk;4 jkjdksl;2 kdkld I need both the columns to be ...
29
votes
3answers
77k views

Sort based on the third column

I'm facing a huge 4-columns file. I'd like to display the sorted file in stdout based on its 3rd column: cat myFile | sort -u -k3 Is that enough to perform the trick?
0
votes
3answers
57 views

Understanding options to sort

part 1 ps -e | sort -k 1 -nr What is the interpretation of the above command? I want to understand the role of -nr. It is not clearly given the man pages of sort. part 2 When you open the man ...
0
votes
1answer
36 views

Keeping first instance of duplicates when column values are interchangeable

I have previously asked a question about keeping the first instance of duplicates and the solution provided has been working perfectly until now (see Keeping first instance of duplicates). However, I ...
6
votes
5answers
18k views

join : “File 2 not in sorted order”

I've got two files _jeter3.txt and _jeter1.txt I've checked they are both sorted on the 20th column using sort -c sort -t ' ' -c -k20,20 _jeter3.txt sort -t ' ' -c -k20,20 _jeter1.txt #no ...
2
votes
4answers
2k views

sorting filenames numerically when they have non-numeric prefix

I have files in the following format: x1 x2 ... x10 x11 ... x101 ... I want to rename these files, so they all have the same name length: x001 x002 ... x010 ... I can do this with files, that don't ...
0
votes
0answers
29 views

grep names from big list and sort it according to pattern [duplicate]

I have 2 big files as shown below. I want to read the pattern file and grep the line from full-info file and sort the channel name order according to pattern. pattern file HBO COMEDY; DTV CABLE ...
4
votes
1answer
3k views

How to find out common elements between two files?

For an example, I have 2 files having following info: File #1: 12 13 14 15 File #2: 12 1 13 2 14 2 15 6 16 7 17 8 Output File: 1 2 2 6 In the output file, I want only the second column ...
6
votes
5answers
1k views

Grouped sorting of continuous paragraphs (separated by blank line)?

I think I'm pretty experienced now in sorting by columns; however, I haven't found anything so far how to sort continuous rows. Supposing we have a text file that looks like this: (very simplified, ...
0
votes
1answer
40 views

How to find maximum number of different strings?

I have a file: 2.0-00-04 2.0-00-03 2.0-00-02 2.0-00-01 2.0-00-00 1.0-00-02 1.0-00-01 1.0-00-00 How can I find maximum number of each group? In the string above, output should be: 2.0-00-04 ...
0
votes
0answers
24 views

Properly sorting a csv file [duplicate]

The sort function looked promising, but as far as I know, it can't handle escaped line feeds, and the field deliminator option -t doesn't have an option that will take into account escaped commas. ...
2
votes
1answer
110 views

Wrong order with sort command

How to get sort command working with data separated by "|" (pipe character) ? Consider this sample file called "x" which have two fields: axis-AXIS|OK axis-AXISR|OK axis-AXISD2X|OK axis-AXISA|OK ...
4
votes
1answer
121 views

Non-standard sorting of the ls output

I usually include a date in the filename so that files and directories are listed in chronological order when I use the ls command, e.g. 2015-08-29_letter_to_santa.txt. A drawback is that ...
0
votes
0answers
28 views

Sorting different groups (separated by a line) based on 16th column

I have a CSV file which is in this format cluster 0 aaa bbb ccc ....... xxx yyy zzz bhc adf ghy ....... hjn dft cbn cluster 1 adg bhg klo ....... jio jik mkl asf ghb nbv ....... cvb xcz hga ... ...
13
votes
2answers
354 views

Fastest way to delete duplicates in large wordlist?

I need to deduplicate a large wordlist. I tried several commands and did some research here and here where they explain that the fastest way to deduplicate a wordlist seems to be using awk. awk ...
1
vote
3answers
388 views

Select and sort IP address keeping the whole line

So I need to sort the IP Addresses and then sort my line by them. I can sort IP addres in a file by using : sort -n -t . -k1,1 -k2,2 -k 3,3 -k4,4 If my file looks like : add method1 a.b.c.d ...
5
votes
2answers
941 views

Fastest `uniq` tool in linux

I have large text file (1.5 G), I want to know what the fastest and more reliable tool in linux is. I usualy use: awk '!x[$0]++' file.txt But when I use htop command I see my memory usage is ...
12
votes
4answers
723 views

Sort lines by number of words per line

Given input: hello: world foo bar baz bar: baz: bin boop bop fiz bang beep bap: bim bam bop boatkeeper: poughkeepsie I would like to sort it into most words at the top, to least at the end, like ...
7
votes
2answers
183 views

Sort command inconsistent behavior

I want to sort two files but I cannot get consistent results. It seems there are problems with collation but I cannot understand the reason. In sample files separator is a single space: file1: a b B ...
1
vote
1answer
79 views

“sort -h” not working in Redhat 5.9. Any way to upgrade it?

I'm trying to sort the folders disk usage in human readable format in RHEL 5.9. I mounted the drive as NFS in Cent OS 6.4, where sort -h option works fine. But in RHEL 5.9, it throws error as ...
6
votes
3answers
418 views

Print random lines respecting the order of the source file

I have a large text file (~500K lines) with short sentences (couple of words long). Additionally, there is some XML markup in most of the lines. Finally, the text file has been sorted before the ...
4
votes
4answers
752 views

Sorting directories by last modified date/time of the same-named contained file

Is it possible, using grep, find, etc, to sort a list of directories by the last modified date of the same-named file (e.g., file.php) within? For example: domain1/file.php (last modified 20-Jan-2014 ...
30
votes
5answers
36k views

Sorting the output of “find”?

I need to be able to alphabetically sort the output of find before piping it to a command. Entering | sort | between didn't work, so what could I do? find folder1 folder2 -name "*.txt" -print0 | ...
8
votes
7answers
3k views

How to sort (by whatever key) BibTex entries in `.bib` file?

I work with LaTeX and do versioning with Git. For bibliography management I use Mendeley. The problem is that each time Mendeley synchronizes it's .bib exports, they are in different order, what ...
2
votes
2answers
101 views

How can I list files by class/indicator (as in ls --classify) then by name?

ls --clasify append indicator (one of */=>@|) to entries. I would like to group the entries of the same class/indicator together. I can only find ls --group-directories-first for grouping the / ...
2
votes
5answers
283 views

How to sort file by character occurrences per line?

I'm quite new to Linux, and I've found quite a bit of useful information on how to do character counts in a file, but is there a way in Linux/terminal to sort a text file by the number of times a ...
8
votes
5answers
623 views

How intelligent are unix utilities?

Unix utilities like sort, find, grep, diff et al are very handy to perform quick tasks, sometimes without writing any code at all. I wanted to know what algorithms do they internally use and how do ...
5
votes
4answers
1k views

For each subfolder, sort files by name and rename them to sequential padded numbers (regardless of extension)

I have the following directory structure: . ├── event-a │   ├── album-a │   │   ├── a.jpg │   │   └── x.png │   └── album-b │   ├── a.jpg │   ├── x.png │   └── y.gif └── event-b ├── ...
1
vote
1answer
40 views

Group by Functionality on Mac Address

I'm looking for a solution that should do grouping of identical mac address(device ids). I tried using sort -k 2, but I'm not confident of the result. Data Set looks like: 1616.93,ac:22:0b:a6:22:c3 ...
7
votes
2answers
260 views

Sorting a file with integer in parentheses with no space

I am trying to sort a file like this (which is a list of subroutine names) cfn=(1370) __lib_file_MOD_file_open fn=(1368) __universe_MOD_general_boot fn=(916) __universe_MOD_general_main fn=(6142) ...
3
votes
2answers
302 views

sort and uniq columns individually in a text file

I have a file as below: D F T E A R T E K A O E E T P J I would like to sort each column and unique the columns individually as below: A A O E D F P J E R T K T I wonder if there is anyone who ...
0
votes
2answers
214 views

Display filenames with grep

I'm looking for a way to display the filename when using this command: cat *.tcp | grep "tcp" | grep "open" | sort | uniq Is there a way to do it?
2
votes
2answers
248 views

Sort files with a specific extension by modified time and store them into an array

I have found many answers to sort the files of directory and even to save them in an array. Sorting Example ls -t Directory listing in array array=(*) My particular problem needs to sort files ...
3
votes
2answers
78 views

sort in unix on a big file [duplicate]

I have a file like this chr1 101466461 101466462 1 chr6 160143888 160143889 1 chr19 19231081 19231082 1 chr18 47008735 47008736 1 chr1 161407811 161407812 1 chr4 ...
0
votes
3answers
86 views

How to sort/reformat text information in a file?

I have a file (called customers) with information that needs to be sorted; example: qweasd,asdqwd@asd.org,256.62394383 qweasdczasd,asdghjghqwd@assssd.org,256.62394383 ...
12
votes
2answers
12k views

How to sort big files?

I have a PC with Intel(R) Pentium(R) CPU G640 @ 2.80GHz and 8 GByte RAM. Installed Scientific Linux 6.5 on it with EXT3 filesystem. Q: What is the fastest way I can do a "sort -u list.txt -o ...
0
votes
1answer
162 views

Sort -u without sorting but with better uniqueness? [duplicate]

I don't want to sort my file, just filter out duplicate lines, maintaining the original ordering. Is there a way to use sort's unique function without it's sort function (something like cat -u would ...
5
votes
3answers
363 views

Sort one column only, keeping remaining columns untouched

How to (numerically) sort a specific column in a text file, without affecting other columns (irrespective of whether they are sorted or unsorted)? On other threads I found -s argument, but it does ...
1
vote
1answer
62 views

Sorting in reverse numeric order on subfield in a multi-field sort

I'm trying to use GNU sort on a directory/file listing, eg (subset of much longer listing): exams2008/Zoology/Summer/ZY4019Sum08.pdf exams2009/Zoology/Summer/ZY4019.pdf ...
3
votes
3answers
3k views

Sort comma-separated fields on each line by numeric value

I tried this: $ echo "2,3435,1" | sort -n 2,3435,1 $ sort -t',' -n test_sort.txt kill,gill,burger 110,20,30,13 $ cat test_sort.txt 110,20,30,13 kill,gill,burger Why doesn't my sort command work? ...
0
votes
0answers
126 views

Archlinux: Gnome 3.16 - ls & nautilus incorrect sorting

Recently switched from Xfce 4.12 DE to Gnome 3.16 and I have a problem with dirs & files sorting in terminal and nautilus. Test directories (how it should be): [ Pqr ] Abc Klm My current order ...
16
votes
2answers
3k views

How does awk '!a[$0]++' work?

This one-liner removes duplicate lines from text input without pre-sorting. For example: $ cat >f q w e w r $ awk '!a[$0]++' <f q w e r $ The original code I have found on the internets ...
3
votes
1answer
180 views

Find, count and sort all audio files. ALAC (M4A) files

I'm putting a script together to sort a very large music collection. It's approx 22000 albums, a mixture of FLAC, WAV, AIFF, M4A (AAC & ALAC). So far, I can sort by file type and get a total size ...
4
votes
3answers
202 views

Find largest files from each date in a directory

I have a directory similar to the following: -rw-r--r-- 1 root root 223K Apr 28 14:25 2015.04.28_14.25 -rw-r--r-- 1 root root 253K Apr 28 14:55 2015.04.28_14.55 -rw-r--r-- 1 root root 276K Apr 28 ...
2
votes
3answers
767 views

How to sort data in the file using script?

I have a file logs.txt with numbers which repeat: 1 QWE 1 ASD 1 QWE 1 QWE 1 QWE 1 qwe 12 22 qwe 2 aaa 2 vcc 2 xxx 4 asa 44 qwe 4 gfd 4 bcx 5 6 kuy 7 76 lol 5 4 ggg 23 ...
0
votes
2answers
113 views

How to turn Diff command into bash script with prompt

I have a list that i've copy and i want to paste it to the shell to give me just the repeated lines. 1 1 3 2 As i read about bash commands i've maded this: cat > /tmp/sortme ...
0
votes
1answer
16 views

see newest release last (YML release files with the same timestamp from Git)

I pull from Git *yml files with "$packagename-$release.yml". The highest number is the most recent release. I wish to have the listening with latest release last. Correct sort order ...
2
votes
1answer
65 views

sort -u (unique) works for multi-keys but not for a single key

Using GNU sort. I want to print only the first of a group of lines, based on duplicate key(s). -k1,2 works as expected – only the 1st sorted line is output printf %s\\n 'a a c' 'a a b' 'a a a' | ...