Sorting text files or other data, for example with the sort utility.

learn more… | top users | synonyms

1
vote
0answers
7 views

Fastest way to delete duplicates in large wordlist

I need to deduplicate a large wordlist. I tried several commands and did some research here and here where they explain that the fastest way to deduplicate a wordlist seems to be using awk. awk ...
10
votes
4answers
312 views

Sort lines by number of words per line

Given input: hello: world foo bar baz bar: baz: bin boop bop fiz bang beep bap: bim bam bop boatkeeper: poughkeepsie I would like to sort it into most words at the top, to least at the end, like ...
7
votes
2answers
84 views

Sort command inconsistent behavior

I want to sort two files but I cannot get consistent results. It seems there are problems with collation but I cannot understand the reason. In sample files separator is a single space: file1: a b B ...
1
vote
1answer
23 views

“sort -h” not working in Redhat 5.9. Any way to upgrade it?

I'm trying to sort the folders disk usage in human readable format in RHEL 5.9. I mounted the drive as NFS in Cent OS 6.4, where sort -h option works fine. But in RHEL 5.9, it throws error as ...
71
votes
12answers
38k views

how do you sort du output by size?

How do you sort du -sh /dir/* by size? I read one site that said use | sort -n but that's obviously not right. Here's an example that is wrong. [~]# du -sh /var/* | sort -n 0 /var/mail 1.2M ...
1
vote
1answer
63 views

Sorting Numbers & Alphabetic Text

I have a pipe delimited file to sort. The column on which I have to sort the file can have numbers as well as alphabetic text. Please suggest which sort option should I use to sort numbers and text ...
6
votes
3answers
282 views

Print random lines respecting the order of the source file

I have a large text file (~500K lines) with short sentences (couple of words long). Additionally, there is some XML markup in most of the lines. Finally, the text file has been sorted before the ...
4
votes
4answers
292 views

Sorting directories by last modified date/time of the same-named contained file

Is it possible, using grep, find, etc, to sort a list of directories by the last modified date of the same-named file (e.g., file.php) within? For example: domain1/file.php (last modified 20-Jan-2014 ...
23
votes
5answers
24k views

Sorting the output of “find”?

I need to be able to alphabetically sort the output of find before piping it to a command. Entering | sort | between didn't work, so what could I do? find folder1 folder2 -name "*.txt" -print0 | ...
8
votes
7answers
2k views

How to sort (by whatever key) BibTex entries in `.bib` file?

I work with LaTeX and do versioning with Git. For bibliography management I use Mendeley. The problem is that each time Mendeley synchronizes it's .bib exports, they are in different order, what ...
1
vote
2answers
24 views

How can I list files by class/indicator (as in ls --classify) then by name?

ls --clasify append indicator (one of */=>@|) to entries. I would like to group the entries of the same class/indicator together. I can only find ls --group-directories-first for grouping the / ...
2
votes
5answers
148 views

How to sort file by character occurrences per line?

I'm quite new to Linux, and I've found quite a bit of useful information on how to do character counts in a file, but is there a way in Linux/terminal to sort a text file by the number of times a ...
8
votes
5answers
619 views

How intelligent are unix utilities?

Unix utilities like sort, find, grep, diff et al are very handy to perform quick tasks, sometimes without writing any code at all. I wanted to know what algorithms do they internally use and how do ...
5
votes
4answers
1k views

For each subfolder, sort files by name and rename them to sequential padded numbers (regardless of extension)

I have the following directory structure: . ├── event-a │   ├── album-a │   │   ├── a.jpg │   │   └── x.png │   └── album-b │   ├── a.jpg │   ├── x.png │   └── y.gif └── event-b ├── ...
2
votes
2answers
529 views

Calculate median value for a column and say if this value is “OK” or “ERROR” if falls within acceptable range

I would like to calculate the median value of a particular column in a .txt file, output that value and also say if that output value is ok. I can do the initial steps and output the median value, ...
1
vote
1answer
34 views

Group by Functionality on Mac Address

I'm looking for a solution that should do grouping of identical mac address(device ids). I tried using sort -k 2, but I'm not confident of the result. Data Set looks like: 1616.93,ac:22:0b:a6:22:c3 ...
7
votes
2answers
183 views

Sorting a file with integer in parentheses with no space

I am trying to sort a file like this (which is a list of subroutine names) cfn=(1370) __lib_file_MOD_file_open fn=(1368) __universe_MOD_general_boot fn=(916) __universe_MOD_general_main fn=(6142) ...
2
votes
2answers
118 views

sort and uniq columns individually in a text file

I have a file as below: D F T E A R T E K A O E E T P J I would like to sort each column and unique the columns individually as below: A A O E D F P J E R T K T I wonder if there is anyone who ...
0
votes
2answers
64 views

Display filenames with grep

I'm looking for a way to display the filename when using this command: cat *.tcp | grep "tcp" | grep "open" | sort | uniq Is there a way to do it?
2
votes
2answers
84 views

Sort files with a specific extension by modified time and store them into an array

I have found many answers to sort the files of directory and even to save them in an array. Sorting Example ls -t Directory listing in array array=(*) My particular problem needs to sort files ...
3
votes
2answers
56 views

sort in unix on a big file [duplicate]

I have a file like this chr1 101466461 101466462 1 chr6 160143888 160143889 1 chr19 19231081 19231082 1 chr18 47008735 47008736 1 chr1 161407811 161407812 1 chr4 ...
6
votes
5answers
3k views

Any tricks to speed up sort?

For example for bzip there is pbzip, a parallel version of bzip. Is there any such parallelization tools for sort to improve performance?
0
votes
3answers
58 views

How to sort/reformat text information in a file?

I have a file (called customers) with information that needs to be sorted; example: qweasd,asdqwd@asd.org,256.62394383 qweasdczasd,asdghjghqwd@assssd.org,256.62394383 ...
5
votes
5answers
539 views

Grouped sorting of continuous paragraphs (separated by blank line)?

I think I'm pretty experienced now in sorting by columns; however, I haven't found anything so far how to sort continuous rows. Supposing we have a text file that looks like this: (very simplified, ...
8
votes
2answers
6k views

How to sort big files?

I have a PC with Intel(R) Pentium(R) CPU G640 @ 2.80GHz and 8 GByte RAM. Installed Scientific Linux 6.5 on it with EXT3 filesystem. Q: What is the fastest way I can do a "sort -u list.txt -o ...
0
votes
1answer
46 views

Sort -u without sorting but with better uniqueness? [duplicate]

I don't want to sort my file, just filter out duplicate lines, maintaining the original ordering. Is there a way to use sort's unique function without it's sort function (something like cat -u would ...
5
votes
3answers
210 views

Sort one column only, keeping remaining columns untouched

How to (numerically) sort a specific column in a text file, without affecting other columns (irrespective of whether they are sorted or unsorted)? On other threads I found -s argument, but it does ...
1
vote
1answer
37 views

Sorting in reverse numeric order on subfield in a multi-field sort

I'm trying to use GNU sort on a directory/file listing, eg (subset of much longer listing): exams2008/Zoology/Summer/ZY4019Sum08.pdf exams2009/Zoology/Summer/ZY4019.pdf ...
3
votes
3answers
2k views

Sort comma-separated fields on each line by numeric value

I tried this: $ echo "2,3435,1" | sort -n 2,3435,1 $ sort -t',' -n test_sort.txt kill,gill,burger 110,20,30,13 $ cat test_sort.txt 110,20,30,13 kill,gill,burger Why doesn't my sort command work? ...
0
votes
0answers
58 views

Archlinux: Gnome 3.16 - ls & nautilus incorrect sorting

Recently switched from Xfce 4.12 DE to Gnome 3.16 and I have a problem with dirs & files sorting in terminal and nautilus. Test directories (how it should be): [ Pqr ] Abc Klm My current order ...
15
votes
2answers
2k views

How does awk '!a[$0]++' work?

This one-liner removes duplicate lines from text input without pre-sorting. For example: $ cat >f q w e w r $ awk '!a[$0]++' <f q w e r $ The original code I have found on the internets ...
3
votes
1answer
115 views

Find, count and sort all audio files. ALAC (M4A) files

I'm putting a script together to sort a very large music collection. It's approx 22000 albums, a mixture of FLAC, WAV, AIFF, M4A (AAC & ALAC). So far, I can sort by file type and get a total size ...
37
votes
5answers
29k views

What is the difference between “sort -u” and “sort | uniq”?

Everywhere I see someone needing to get a sorted, unique list, they always pipe to sort | uniq. I've never seen any examples where someone uses sort -u instead. Why not? What's the difference, and why ...
4
votes
3answers
105 views

Find largest files from each date in a directory

I have a directory similar to the following: -rw-r--r-- 1 root root 223K Apr 28 14:25 2015.04.28_14.25 -rw-r--r-- 1 root root 253K Apr 28 14:55 2015.04.28_14.55 -rw-r--r-- 1 root root 276K Apr 28 ...
2
votes
3answers
289 views

How to sort data in the file using script?

I have a file logs.txt with numbers which repeat: 1 QWE 1 ASD 1 QWE 1 QWE 1 QWE 1 qwe 12 22 qwe 2 aaa 2 vcc 2 xxx 4 asa 44 qwe 4 gfd 4 bcx 5 6 kuy 7 76 lol 5 4 ggg 23 ...
0
votes
2answers
73 views

How to turn Diff command into bash script with prompt

I have a list that i've copy and i want to paste it to the shell to give me just the repeated lines. 1 1 3 2 As i read about bash commands i've maded this: cat > /tmp/sortme ...
0
votes
1answer
12 views

see newest release last (YML release files with the same timestamp from Git)

I pull from Git *yml files with "$packagename-$release.yml". The highest number is the most recent release. I wish to have the listening with latest release last. Correct sort order ...
2
votes
1answer
51 views

sort -u (unique) works for multi-keys but not for a single key

Using GNU sort. I want to print only the first of a group of lines, based on duplicate key(s). -k1,2 works as expected – only the 1st sorted line is output printf %s\\n 'a a c' 'a a b' 'a a a' | ...
4
votes
2answers
5k views

Sort and count number of occurrence of lines

I have Apache logfile, access.log, how to count number of line occurrence in that file? for example the result of cut -f 7 -d ' ' | cut -d '?' -f 1 | tr '[:upper:]' '[:lower:]' is a.php b.php a.php ...
1
vote
1answer
58 views

what is the default delimiter for sort -k option

What is the default delimiter for sort's -k option? I searched some man page and samples and still feel lost.
25
votes
8answers
21k views

List files sorted numerically

I have a bunch of files from log1 to log164. I'm trying to LIST the directory (sorted) in a UNIX terminal but the sort functions are only providing the format like this: home:logs Home$ ls -1 | sort ...
4
votes
4answers
124 views

Stable sort a file by presence of ordered substrings on each line

I have a list of sound sources that I'm processing with a script. An example would be: alsa_input.usb-AVEO_Technology_Corp._USB2.0_Camera-02-Camera.analog-mono ...
1
vote
1answer
105 views

sort file based one colum and group by another column

I have a file of three columns: 1 A 0.5 2 B 0.7 3 A 10 4 C 4 5 B 4 I want to sort the file by increasing order of column 3 and group by column 2 1 A 0.5 3 A 10 ...
5
votes
2answers
165 views

How to sort lines that contain “_” numerically?

Here's a subset of the file names in my file: profile_10_1_1 profile_10_1_2 profile_1_1_1 I'm trying to sort them numerically in ascending order, that is starting from 1 onwards. I used the ...
5
votes
3answers
6k views

How is it possible to sort ps command's CPU field?

I'm trying to make a small command that will find the processes that use the most CPU power. Firstly, I use ps aux > file.txt and then cut -c 16-20 file.txt | sort -n | tail -5. The result I get ...
1
vote
1answer
62 views

Search and sorts error logs based on a particular string

How to search through all the errors.zip as well as from errors.log files filter by "Error" string and sort by date? What about totals of errors? requirement : here are 3 components are installed ...
2
votes
2answers
809 views

Creating recursively sorted RAR archive

For various reasons I decided to use RAR archive to store some data, instead compressed tar. Happy with decision, except for one thing: I can't find a way to store files sorted. Order of files is ...
7
votes
3answers
851 views

Unix sort out of memory

I'm sorting a 25 GB file containing 200M lines on Fedora 18 using: sort -S 10G -T /bigdisk bigfile and I am getting process killed due to no memory. The process isn't maxing out my RAM before death ...
0
votes
1answer
55 views

direct GNU sort to use much memory and many cores

I have a fairly large text file (19Gb, half a billion lines) and a fairly large server on which to sort it (16 cores, 64Gb memory) running Ubuntu 12.04. What is the best way to use the command line ...
1
vote
2answers
66 views

Append words to wordlist with sort -u avoinding duplicata

I have 2 txt file called one.txt with duplicata: yesterday yesterday today today tomorrow tomorrow and the second txt called two.txt with duplicata: mike mike paul paul tomorrow tomorrow Using ...