Sorting text files or other data, for example with the sort utility.

learn more… | top users | synonyms

4
votes
4answers
103 views

Stable sort a file by presence of ordered substrings on each line

I have a list of sound sources that I'm processing with a script. An example would be: alsa_input.usb-AVEO_Technology_Corp._USB2.0_Camera-02-Camera.analog-mono ...
1
vote
1answer
43 views

sort file based one colum and group by another column

I have a file of three columns: 1 A 0.5 2 B 0.7 3 A 10 4 C 4 5 B 4 I want to sort the file by increasing order of column 3 and group by column 2 1 A 0.5 3 A 10 ...
5
votes
2answers
154 views

How to sort lines that contain “_” numerically?

Here's a subset of the file names in my file: profile_10_1_1 profile_10_1_2 profile_1_1_1 I'm trying to sort them numerically in ascending order, that is starting from 1 onwards. I used the ...
1
vote
1answer
47 views

Search and sorts error logs based on a particular string

How to search through all the errors.zip as well as from errors.log files filter by "Error" string and sort by date? What about totals of errors? requirement : here are 3 components are installed ...
1
vote
2answers
47 views

Append words to wordlist with sort -u avoinding duplicata

I have 2 txt file called one.txt with duplicata: yesterday yesterday today today tomorrow tomorrow and the second txt called two.txt with duplicata: mike mike paul paul tomorrow tomorrow Using ...
-1
votes
1answer
81 views

sort | uniq not working on different files [closed]

I have no idea why the command sort filename.txt | uniq is not working on this file: http://packetstormsecurity.com/files/32089/koran.gz.html I have also used: awk '!seen[$0]++' koran.txt perl ...
0
votes
1answer
40 views

direct GNU sort to use much memory and many cores

I have a fairly large text file (19Gb, half a billion lines) and a fairly large server on which to sort it (16 cores, 64Gb memory) running Ubuntu 12.04. What is the best way to use the command line ...
1
vote
1answer
107 views

How to tell the sizes of files and folders, sorted, and in MB, GB, instead of a long integer? [duplicate]

On a Mac, using Bash, I tend to do du -sh * to tell the sizes of files and folders in the current directory, in MB and GB. But if I want to sort them by the sizes, I might do du -sh * | sort -g ...
4
votes
2answers
116 views

how to sort by the day of the week?

I have two files. file1: Dave 734.838.9800 Bob 313.123.4567 Carol 248.344.5576 Mary 313.449.1390 Ted 248.496.2204 Alice 616.556.4458 file2: Bob Tuesday Carol Monday Ted Sunday ...
1
vote
1answer
71 views

sort and uniq in awk

I know there are "sort" and "uniq" out there, however, today's question is about how to utilise AWK to do that kind of a job. Say if I have a list of anything really (ips, names, or numbers) and I ...
1
vote
2answers
70 views

sort only the first column and uniq

I have a list as such below: 1,cat 1,dog 2,apple 3,human I'd like an output like this: 1,cat,dog 2,apple 3,human So value 1 from column 1 contains the value of cat and dog from ...
2
votes
3answers
120 views

Sorting numbers and respective units in columns

I have a list like this: 1,kg,cat 1000,g,dog 20,g,apple And I want to sort it by weight. 1 kg and 1000 g would be the same, so I'd like to have my rows sorted by weight. My data set is ...
0
votes
1answer
55 views

can we get sorted grep output

I have same: 1.txt with apple written in it, 2.txt with mango written in it, 3.txt with apple written in it, 4.txt with mango written in it. grep -e apple -e mango *.txt will give result as: 1.txt: ...
4
votes
1answer
23 views

Sort based on field1, field2 but preserving field 1 sort and ignoring nulls/blanks in 2nd sort

After doing a lot of home work and running multiple test scenarios, I decided to get help from experts Here is my scenario Input data after sorting by Column 1 -bash-3.2$ sort -t'|' -k1 -k2 ...
4
votes
3answers
212 views

sum columns based on matching fields

I have encountered a problem that exceeds my basic unix knowledge and would really appreciate some help. I have a large file in the following format: 2 1019 0 12 2 1019 3 0 2 1021 0 2 ...
1
vote
3answers
138 views

lines with unique checksum

I have a file A of lines of equal length n. I have a program that for each line in A can output a line of length n which its key. The program can either write the sequence of keys for the lines in A ...
1
vote
1answer
50 views

Combining, sorting and deleting duplicates in numerous gzip files

I am going to be combining 15 different gzip files. Ranging in size from 2 gigs to 15 gigs each so the files are relatively large. I have done research on the best way to do it but still have some ...
1
vote
2answers
186 views

How do I move a line in a text file up or down by one line?

I have some text files, and I'd like to be able to move an arbitrary line in any of the files up or down one line (lines at the beginning or end of the file would stay where they are). I have some ...
3
votes
5answers
130 views

Sorting bigger files gives segmentation fault

mkdir tmp sort -u *.txt -T tmp/ -o output.txt there are just a few few ten GByte sized files that needs to be sorted.. The problem is that after a few minutes sort gives a segmentation fault. The ...
3
votes
1answer
81 views

How to sort an associative array and retain the keys?

I have an array with filenames as keys and a numerical value as values. MYARRAY[00001.jpg] = 31 MYARRAY[00002.jpg] = 200 MYARRAY[00003.jpg] = 98 I need to sort them so they are ordered by value. ...
-1
votes
1answer
36 views

How to sort by column and break ties randomly

I have a tab-delimited file with three columns like this: joe W 4 bob A 1 ana F 1 roy J 3 sam S 0 don R 2 tim L 0 cyb M 0 I want to sort this file by decreasing values in the third column, but to ...
1
vote
2answers
47 views

How do I transform array output sorted in lexicographic order to numeric order

I have a text file that consists of data in this form: int array[0] int array[1001] int array[1002] int array[1003] int array[2] int array[2001] int array[2002] ... In other words, the numbers ...
1
vote
4answers
82 views

Sort command on more than one field

I have a file with contents 192.168.1.3 192.168.1.4 192.168.1.2 10.1.1.1 10.1.1.2 10.1.1.3 192.168.1.56 192.168.1.43 10.1.1.23 When I gave $ sort -h -t. -k3,4 sort_test.txt 10.1.1.1 10.1.1.2 ...
1
vote
1answer
57 views

How sort command column option is handled

I have a file like 1.2.3.4 1.1.1.1 4.5.6.7 1.1.3.5 When I gave sort -t. -k3 inp3 1.1.1.1 1.2.3.4 1.1.3.5 4.5.6.7 and when I gave sort -t. -k3,3 inp3 1.1.1.1 1.1.3.5 1.2.3.4 4.5.6.7 second ...
2
votes
1answer
133 views

Unix sort by multiple columns

I have trouble understanding unix sort. Consider the following file (tab separated) aa ~ a1 aa B b A b ~ e bb B bb ~ B When calling: cat tmp2 | sort -t $'\t' -k1,2 I get aa ~ a1 aa B ...
4
votes
5answers
292 views

sort order explanation

Giving a lot of files, sort -n gives me this result: f1.txt f10.txt f2.txt which doesn't seem right to me. What is the explanation? Can I get this by default: f1.txt f2.txt f10.txt
1
vote
3answers
60 views

Display distinct values of a list and number of occurences

I have a program that writes on the standard output a list of string values, one per line, and I would like to display in real time the list of distinct values along with the number of occurences for ...
2
votes
1answer
89 views

counting occurrences of a string

I have a large database of this format: 0500506 gi|561108321|ref|NC_018143.2| 71446 0 0500506 gi|561108321|ref|NC_018143.2| 71447 0 0500508 gi|561108321|ref|NC_018143.2| 71449 0 0500509 ...
1
vote
3answers
282 views

How to tar files with a sorted order?

If you tar a directory recursively, it just uses the order from the os's readdir. But in some cases its nice to tar the files sorted. Whats a good way to tar a directory sorted alphabetically? ...
1
vote
1answer
107 views

How can I list hidden directories, then directories, then hidden files and finally files with ls?

I'm trying to figure out how to list the contents of a directory in the following group order: Hidden directories Non-hidden directories All other hidden files (including regular files, symlinks, ...
2
votes
0answers
85 views

Installed coreutils on CentOS for sort --human-readable flag but it still doesn't work

I'm trying to do du -h | sort -h, but my CentOS 5.7 doesn't have -h flag for sort. So I installed coreutils Updated: coreutils.x86_64 0:5.97-34.el5_8.1 Complete! yet sort still doesn't have ...
3
votes
2answers
50 views

Trying to sort on two fields first is second criteria

I'm trying to sort on multiple columns a file with GNU sort. The format of the source file tmp/file.txt is : 03A12345679 2014-09-30-00.00.01.00 "03" => first column "A" => field separator ...
3
votes
2answers
514 views

Sorting the output of `ps`

I am trying to understand why ps does not behave the way I expect it to. From the man pages, the following command should display ppid and lstart, with items sorted by lstart order. However when I run ...
2
votes
1answer
1k views

Cut and Paste Commands

So I have: $ cat fruits 2 bananas 3 cherries 4 figs 5 dates 6 elderberries 7 apples 8 grapes and 1 $ cat prices 2 2.18 3 4.11 4 1.69 5 4.52 6 1.73 7 1.01 8 1.09 Every line from 'fruits' ...
1
vote
5answers
151 views

How to get the unique count of a particular part of a string

I have a set of data in a file. psf7433-nlhrms unit7433-nobody unit7333-opera bpx7333-operations app7333-osm unit7330-partners psf7331-pdesmond unit7333-projm mnp7330-redirect unit7333-retailbanking ...
2
votes
2answers
83 views

order for listing files

If I have a list of files under a directory ullman-000.pbm ullman-001.pbm ... ullman-098.pbm ullman-099.pbm ullman-1000.pbm ullman-1001.pbm ullman-1002.pbm ullman-1003.pbm ullman-1004.pbm ...
1
vote
1answer
78 views

Sort and backup log4j files

I apologize for length of this question. It's difficult to explain, even though I was fluent in English, which I'm not ;) Not really sure where post this question, because it has java, log4j library ...
5
votes
1answer
171 views

How to make a list generated by md5deep in alphabetical order of relative paths?

If I want make a md5sum list recursively, then I would use md5deep, but it starts to pop up some problems such as it won't generate the md5sum file in alphabetical order. For example, $ cd ...
0
votes
1answer
194 views

Merging CSV (single line per row) files without losing the header line

I need to merge two CSV files: datasorted1.csv and datasorted2.csv, but my headers keep getting sorted. head -n 1 datasorted1.csv datasorted2.csv > resultado.csv && tail -n +2 ...
2
votes
2answers
803 views

Sorting a csv file [duplicate]

I need to sort a CSV file, but the the header row (1st row) keeps getting sorted. This is what I'm using: cat data1.csv | sort -t"|" -k 1 -o data1.csv Here's a sample line: ...
2
votes
1answer
1k views

Sort and count number of occurrence of lines

I have Apache logfile, access.log, how to count number of line occurrence in that file? for example the result of cut -f 7 -d ' ' | cut -d '?' -f 1 | tr '[:upper:]' '[:lower:]' is a.php b.php a.php ...
1
vote
2answers
86 views

Compare and print data from different text files to one in Shell

How can I compare and print data from different text files to one in Shell. I have captures NAS details of three different boxes using SSH, now I need to combine all the three text files to one file ...
17
votes
3answers
2k views

Why is coreutils sort slower than Python?

I wrote the following script to test the speed of Python's sort functionality: from sys import stdin, stdout lines = list(stdin) lines.sort() stdout.writelines(lines) I then compared this to the ...
3
votes
1answer
165 views

Sorting lists when there are no leading zeros

I would like to sort this numerically and the sort command keys off the first character. CPL_V11.01.00_1 CPL_V11.01.00_10 CPL_V11.01.00_2 CPL_V11.01.00_3 CPL_V11.01.00_35 CPL_V11.01.00_36 ...
0
votes
3answers
57 views

sort command in unix with numeric sort

I have file1 : "$lvl=57" "$lvl=20" "$lvl=48" "$lvl=17" "$lvl=58" file2 i want : "$lvl=17" "$lvl=20" "$lvl=48" "$lvl=57" "$lvl=58" Basically numerically sort of file1
1
vote
1answer
114 views

Removing duplicate lines after comparing first pattern and special pattern

I'm looking for code to removing duplicate lines after comparing the first tab-delimited pattern: Example : (↦ = tab) car ↦ ab ↦ 38/43 car ↦ ca ↦ 2/34 house ↦ ca ↦ 34/34 house ↦ d ↦ 2/3 house ↦ ht ↦ ...
2
votes
1answer
59 views

Sorting image files based on whether the content is picture or graphics: updating an older solution?

Someone wanted to sort files in a directory according to whether the files are pictures(camera) vs. design graphics. On Stackoverflow, there is an older solution/proof-of-concept involving the ...
1
vote
4answers
100 views

Sort datetime string in 12-hour format

I have a log file with contents like this: 11-12-2014 - 03:03:59AM lat = 41.990516; lon = -93.430704<br> 11-12-2014 - 03:05:15AM lat = 41.001546; lon = -93.443352<br> 11-12-2014 - ...
1
vote
5answers
102 views

How to sort on two fields of numbers

I have a list of filename like so: epdb_0_0.pdb epdb_0_10.pdb epdb_0_11.pdb epdb_0_12.pdb epdb_0_13.pdb epdb_0_14.pdb epdb_0_15.pdb epdb_0_16.pdb epdb_0_17.pdb epdb_0_18.pdb epdb_0_19.pdb ...
1
vote
1answer
44 views

Different results OS X/Linux with “find PATH -mount \( -type f -o -type d \) -print0 | LC_ALL=C sort --zero-terminated > OUTPUT.txt”

this command is part of a script I use for comparing files: find test/ -mount \( -type f -o -type d \) -print0 \ | LC_ALL=C sort --zero-terminated > OUTPUT.txt // test/ is the PATH ...