Sorting text files or other data, for example with the sort utility.

learn more… | top users | synonyms

1
vote
2answers
48 views

sort and uniq columns individually in a text file

I have a file as below: D F T E A R T E K A O E E T P J I would like to sort each column and unique the columns individually as below: A A O E D F P J E R T K T I wonder if there is anyone who ...
3
votes
2answers
45 views

sort in unix on a big file [duplicate]

I have a file like this chr1 101466461 101466462 1 chr6 160143888 160143889 1 chr19 19231081 19231082 1 chr18 47008735 47008736 1 chr1 161407811 161407812 1 chr4 ...
5
votes
5answers
2k views

Any tricks to speed up sort?

For example for bzip there is pbzip, a parallel version of bzip. Is there any such parallelization tools for sort to improve performance?
0
votes
3answers
54 views

How to sort/reformat text information in a file?

I have a file (called customers) with information that needs to be sorted; example: qweasd,asdqwd@asd.org,256.62394383 qweasdczasd,asdghjghqwd@assssd.org,256.62394383 ...
5
votes
5answers
488 views

Grouped sorting of continuous paragraphs (separated by blank line)?

I think I'm pretty experienced now in sorting by columns; however, I haven't found anything so far how to sort continuous rows. Supposing we have a text file that looks like this: (very simplified, ...
8
votes
2answers
5k views

How to sort big files?

I have a PC with Intel(R) Pentium(R) CPU G640 @ 2.80GHz and 8 GByte RAM. Installed Scientific Linux 6.5 on it with EXT3 filesystem. Q: What is the fastest way I can do a "sort -u list.txt -o ...
0
votes
1answer
37 views

Sort -u without sorting but with better uniqueness? [duplicate]

I don't want to sort my file, just filter out duplicate lines, maintaining the original ordering. Is there a way to use sort's unique function without it's sort function (something like cat -u would ...
5
votes
3answers
184 views

Sort one column only, keeping remaining columns untouched

How to (numerically) sort a specific column in a text file, without affecting other columns (irrespective of whether they are sorted or unsorted)? On other threads I found -s argument, but it does ...
1
vote
1answer
30 views

Sorting in reverse numeric order on subfield in a multi-field sort

I'm trying to use GNU sort on a directory/file listing, eg (subset of much longer listing): exams2008/Zoology/Summer/ZY4019Sum08.pdf exams2009/Zoology/Summer/ZY4019.pdf ...
3
votes
3answers
1k views

Sort comma-separated fields on each line by numeric value

I tried this: $ echo "2,3435,1" | sort -n 2,3435,1 $ sort -t',' -n test_sort.txt kill,gill,burger 110,20,30,13 $ cat test_sort.txt 110,20,30,13 kill,gill,burger Why doesn't my sort command work? ...
0
votes
0answers
16 views

Archlinux: Gnome 3.16 - ls & nautilus incorrect sorting

Recently switched from Xfce 4.12 DE to Gnome 3.16 and I have a problem with dirs & files sorting in terminal and nautilus. Test directories (how it should be): [ Pqr ] Abc Klm My current order ...
15
votes
2answers
1k views

How does awk '!a[$0]++' work?

This one-liner removes duplicate lines from text input without pre-sorting. For example: $ cat >f q w e w r $ awk '!a[$0]++' <f q w e r $ The original code I have found on the internets ...
3
votes
1answer
92 views

Find, count and sort all audio files. ALAC (M4A) files

I'm putting a script together to sort a very large music collection. It's approx 22000 albums, a mixture of FLAC, WAV, AIFF, M4A (AAC & ALAC). So far, I can sort by file type and get a total size ...
34
votes
5answers
25k views

What is the difference between “sort -u” and “sort | uniq”?

Everywhere I see someone needing to get a sorted, unique list, they always pipe to sort | uniq. I've never seen any examples where someone uses sort -u instead. Why not? What's the difference, and why ...
4
votes
3answers
79 views

Find largest files from each date in a directory

I have a directory similar to the following: -rw-r--r-- 1 root root 223K Apr 28 14:25 2015.04.28_14.25 -rw-r--r-- 1 root root 253K Apr 28 14:55 2015.04.28_14.55 -rw-r--r-- 1 root root 276K Apr 28 ...
2
votes
3answers
227 views

How to sort data in the file using script?

I have a file logs.txt with numbers which repeat: 1 QWE 1 ASD 1 QWE 1 QWE 1 QWE 1 qwe 12 22 qwe 2 aaa 2 vcc 2 xxx 4 asa 44 qwe 4 gfd 4 bcx 5 6 kuy 7 76 lol 5 4 ggg 23 ...
0
votes
2answers
68 views

How to turn Diff command into bash script with prompt

I have a list that i've copy and i want to paste it to the shell to give me just the repeated lines. 1 1 3 2 As i read about bash commands i've maded this: cat > /tmp/sortme ...
0
votes
1answer
11 views

see newest release last (YML release files with the same timestamp from Git)

I pull from Git *yml files with "$packagename-$release.yml". The highest number is the most recent release. I wish to have the listening with latest release last. Correct sort order ...
2
votes
1answer
46 views

sort -u (unique) works for multi-keys but not for a single key

Using GNU sort. I want to print only the first of a group of lines, based on duplicate key(s). -k1,2 works as expected – only the 1st sorted line is output printf %s\\n 'a a c' 'a a b' 'a a a' | ...
3
votes
2answers
2k views

Sort and count number of occurrence of lines

I have Apache logfile, access.log, how to count number of line occurrence in that file? for example the result of cut -f 7 -d ' ' | cut -d '?' -f 1 | tr '[:upper:]' '[:lower:]' is a.php b.php a.php ...
1
vote
1answer
46 views

what is the default delimiter for sort -k option

What is the default delimiter for sort's -k option? I searched some man page and samples and still feel lost.
21
votes
8answers
18k views

List files sorted numerically

I have a bunch of files from log1 to log164. I'm trying to LIST the directory (sorted) in a UNIX terminal but the sort functions are only providing the format like this: home:logs Home$ ls -1 | sort ...
4
votes
4answers
115 views

Stable sort a file by presence of ordered substrings on each line

I have a list of sound sources that I'm processing with a script. An example would be: alsa_input.usb-AVEO_Technology_Corp._USB2.0_Camera-02-Camera.analog-mono ...
1
vote
1answer
66 views

sort file based one colum and group by another column

I have a file of three columns: 1 A 0.5 2 B 0.7 3 A 10 4 C 4 5 B 4 I want to sort the file by increasing order of column 3 and group by column 2 1 A 0.5 3 A 10 ...
5
votes
2answers
162 views

How to sort lines that contain “_” numerically?

Here's a subset of the file names in my file: profile_10_1_1 profile_10_1_2 profile_1_1_1 I'm trying to sort them numerically in ascending order, that is starting from 1 onwards. I used the ...
4
votes
3answers
5k views

How is it possible to sort ps command's CPU field?

I'm trying to make a small command that will find the processes that use the most CPU power. Firstly, I use ps aux > file.txt and then cut -c 16-20 file.txt | sort -n | tail -5. The result I get ...
1
vote
1answer
54 views

Search and sorts error logs based on a particular string

How to search through all the errors.zip as well as from errors.log files filter by "Error" string and sort by date? What about totals of errors? requirement : here are 3 components are installed ...
2
votes
2answers
788 views

Creating recursively sorted RAR archive

For various reasons I decided to use RAR archive to store some data, instead compressed tar. Happy with decision, except for one thing: I can't find a way to store files sorted. Order of files is ...
7
votes
3answers
786 views

Unix sort out of memory

I'm sorting a 25 GB file containing 200M lines on Fedora 18 using: sort -S 10G -T /bigdisk bigfile and I am getting process killed due to no memory. The process isn't maxing out my RAM before death ...
0
votes
1answer
47 views

direct GNU sort to use much memory and many cores

I have a fairly large text file (19Gb, half a billion lines) and a fairly large server on which to sort it (16 cores, 64Gb memory) running Ubuntu 12.04. What is the best way to use the command line ...
1
vote
2answers
54 views

Append words to wordlist with sort -u avoinding duplicata

I have 2 txt file called one.txt with duplicata: yesterday yesterday today today tomorrow tomorrow and the second txt called two.txt with duplicata: mike mike paul paul tomorrow tomorrow Using ...
1
vote
1answer
122 views

How to tell the sizes of files and folders, sorted, and in MB, GB, instead of a long integer? [duplicate]

On a Mac, using Bash, I tend to do du -sh * to tell the sizes of files and folders in the current directory, in MB and GB. But if I want to sort them by the sizes, I might do du -sh * | sort -g ...
-1
votes
1answer
85 views

sort | uniq not working on different files [closed]

I have no idea why the command sort filename.txt | uniq is not working on this file: http://packetstormsecurity.com/files/32089/koran.gz.html I have also used: awk '!seen[$0]++' koran.txt perl ...
4
votes
2answers
132 views

how to sort by the day of the week?

I have two files. file1: Dave 734.838.9800 Bob 313.123.4567 Carol 248.344.5576 Mary 313.449.1390 Ted 248.496.2204 Alice 616.556.4458 file2: Bob Tuesday Carol Monday Ted Sunday ...
65
votes
12answers
34k views

how do you sort du output by size?

How do you sort du -sh /dir/* by size? I read one site that said use | sort -n but that's obviously not right. Here's an example that is wrong. [~]# du -sh /var/* | sort -n 0 /var/mail 1.2M ...
21
votes
4answers
6k views

sort inplace - like sed --in-place - exists?

Am I blind or is there no option like --in-place for sort? In order to save results to the input file, sed uses -i (--in-place). Redirecting the output of sort to the input file sort < f > f ...
1
vote
1answer
149 views

sort and uniq in awk

I know there are "sort" and "uniq" out there, however, today's question is about how to utilise AWK to do that kind of a job. Say if I have a list of anything really (ips, names, or numbers) and I ...
1
vote
2answers
95 views

sort only the first column and uniq

I have a list as such below: 1,cat 1,dog 2,apple 3,human I'd like an output like this: 1,cat,dog 2,apple 3,human So value 1 from column 1 contains the value of cat and dog from ...
2
votes
3answers
136 views

Sorting numbers and respective units in columns

I have a list like this: 1,kg,cat 1000,g,dog 20,g,apple And I want to sort it by weight. 1 kg and 1000 g would be the same, so I'd like to have my rows sorted by weight. My data set is ...
0
votes
1answer
61 views

can we get sorted grep output

I have same: 1.txt with apple written in it, 2.txt with mango written in it, 3.txt with apple written in it, 4.txt with mango written in it. grep -e apple -e mango *.txt will give result as: 1.txt: ...
4
votes
1answer
25 views

Sort based on field1, field2 but preserving field 1 sort and ignoring nulls/blanks in 2nd sort

After doing a lot of home work and running multiple test scenarios, I decided to get help from experts Here is my scenario Input data after sorting by Column 1 -bash-3.2$ sort -t'|' -k1 -k2 ...
10
votes
3answers
6k views

How to sort the string which combined with string + numeric using bash script?

This is the data what I want to sort. But sort treats the numeric to string, the data it no sorted as I expected. /home/files/profile1 /home/files/profile10 /home/files/profile11 ...
6
votes
3answers
364 views

sum columns based on matching fields

I have encountered a problem that exceeds my basic unix knowledge and would really appreciate some help. I have a large file in the following format: 2 1019 0 12 2 1019 3 0 2 1021 0 2 ...
1
vote
3answers
142 views

lines with unique checksum

I have a file A of lines of equal length n. I have a program that for each line in A can output a line of length n which its key. The program can either write the sequence of keys for the lines in A ...
3
votes
5answers
147 views

Sorting bigger files gives segmentation fault

mkdir tmp sort -u *.txt -T tmp/ -o output.txt there are just a few few ten GByte sized files that needs to be sorted.. The problem is that after a few minutes sort gives a segmentation fault. The ...
1
vote
1answer
69 views

Combining, sorting and deleting duplicates in numerous gzip files

I am going to be combining 15 different gzip files. Ranging in size from 2 gigs to 15 gigs each so the files are relatively large. I have done research on the best way to do it but still have some ...
3
votes
1answer
106 views

How to sort an associative array and retain the keys?

I have an array with filenames as keys and a numerical value as values. MYARRAY[00001.jpg] = 31 MYARRAY[00002.jpg] = 200 MYARRAY[00003.jpg] = 98 I need to sort them so they are ordered by value. ...
1
vote
2answers
307 views

How do I move a line in a text file up or down by one line?

I have some text files, and I'd like to be able to move an arbitrary line in any of the files up or down one line (lines at the beginning or end of the file would stay where they are). I have some ...
-1
votes
1answer
45 views

How to sort by column and break ties randomly

I have a tab-delimited file with three columns like this: joe W 4 bob A 1 ana F 1 roy J 3 sam S 0 don R 2 tim L 0 cyb M 0 I want to sort this file by decreasing values in the third column, but to ...
0
votes
3answers
68 views

sort command in unix with numeric sort

I have file1 : "$lvl=57" "$lvl=20" "$lvl=48" "$lvl=17" "$lvl=58" file2 i want : "$lvl=17" "$lvl=20" "$lvl=48" "$lvl=57" "$lvl=58" Basically numerically sort of file1