Sorting text files or other data, for example with the sort utility.

learn more… | top users | synonyms

1
vote
1answer
98 views

How to tell the sizes of files and folders, sorted, and in MB, GB, instead of a long integer? [duplicate]

On a Mac, using Bash, I tend to do du -sh * to tell the sizes of files and folders in the current directory, in MB and GB. But if I want to sort them by the sizes, I might do du -sh * | sort -g ...
-1
votes
1answer
61 views

sort | uniq not working on different files [on hold]

I have no idea why the command sort filename.txt | uniq is not working on this file: http://packetstormsecurity.com/files/32089/koran.gz.html I have also used: awk '!seen[$0]++' koran.txt perl ...
4
votes
2answers
90 views

how to sort by the day of the week?

I have two files. file1: Dave 734.838.9800 Bob 313.123.4567 Carol 248.344.5576 Mary 313.449.1390 Ted 248.496.2204 Alice 616.556.4458 file2: Bob Tuesday Carol Monday Ted Sunday ...
61
votes
12answers
29k views

how do you sort du output by size?

How do you sort du -sh /dir/* by size? I read one site that said use | sort -n but that's obviously not right. Here's an example that is wrong. [~]# du -sh /var/* | sort -n 0 /var/mail 1.2M ...
0
votes
0answers
29 views

direct GNU sort to use much memory and many cores

I have a fairly large text file (19Gb, half a billion lines) and a fairly large server on which to sort it (16 cores, 64Gb memory) running Ubuntu 12.04. What is the best way to use the command line ...
17
votes
4answers
4k views

sort inplace - like sed --in-place - exists?

Am I blind or is there no option like --in-place for sort? In order to save results to the input file, sed uses -i (--in-place). Redirecting the output of sort to the input file sort < f > f ...
26
votes
4answers
20k views

What is the difference between “sort -u” and “sort | uniq”?

Everywhere I see someone needing to get a sorted, unique list, they always pipe to sort | uniq. I've never seen any examples where someone uses sort -u instead. Why not? What's the difference, and why ...
1
vote
1answer
51 views

sort and uniq in awk

I know there are "sort" and "uniq" out there, however, today's question is about how to utilise AWK to do that kind of a job. Say if I have a list of anything really (ips, names, or numbers) and I ...
1
vote
2answers
65 views

sort only the first column and uniq

I have a list as such below: 1,cat 1,dog 2,apple 3,human I'd like an output like this: 1,cat,dog 2,apple 3,human So value 1 from column 1 contains the value of cat and dog from ...
2
votes
3answers
113 views

Sorting numbers and respective units in columns

I have a list like this: 1,kg,cat 1000,g,dog 20,g,apple And I want to sort it by weight. 1 kg and 1000 g would be the same, so I'd like to have my rows sorted by weight. My data set is ...
0
votes
1answer
52 views

can we get sorted grep output

I have same: 1.txt with apple written in it, 2.txt with mango written in it, 3.txt with apple written in it, 4.txt with mango written in it. grep -e apple -e mango *.txt will give result as: 1.txt: ...
4
votes
1answer
22 views

Sort based on field1, field2 but preserving field 1 sort and ignoring nulls/blanks in 2nd sort

After doing a lot of home work and running multiple test scenarios, I decided to get help from experts Here is my scenario Input data after sorting by Column 1 -bash-3.2$ sort -t'|' -k1 -k2 ...
18
votes
8answers
16k views

List files sorted numerically

I have a bunch of files from log1 to log164. I'm trying to LIST the directory (sorted) in a UNIX terminal but the sort functions are only providing the format like this: home:logs Home$ ls -1 | sort ...
10
votes
3answers
6k views

How to sort the string which combined with string + numeric using bash script?

This is the data what I want to sort. But sort treats the numeric to string, the data it no sorted as I expected. /home/files/profile1 /home/files/profile10 /home/files/profile11 ...
4
votes
3answers
146 views

sum columns based on matching fields

I have encountered a problem that exceeds my basic unix knowledge and would really appreciate some help. I have a large file in the following format: 2 1019 0 12 2 1019 3 0 2 1021 0 2 ...
1
vote
3answers
136 views

lines with unique checksum

I have a file A of lines of equal length n. I have a program that for each line in A can output a line of length n which its key. The program can either write the sequence of keys for the lines in A ...
3
votes
5answers
121 views

Sorting bigger files gives segmentation fault

mkdir tmp sort -u *.txt -T tmp/ -o output.txt there are just a few few ten GByte sized files that needs to be sorted.. The problem is that after a few minutes sort gives a segmentation fault. The ...
1
vote
1answer
36 views

Combining, sorting and deleting duplicates in numerous gzip files

I am going to be combining 15 different gzip files. Ranging in size from 2 gigs to 15 gigs each so the files are relatively large. I have done research on the best way to do it but still have some ...
3
votes
1answer
63 views

How to sort an associative array and retain the keys?

I have an array with filenames as keys and a numerical value as values. MYARRAY[00001.jpg] = 31 MYARRAY[00002.jpg] = 200 MYARRAY[00003.jpg] = 98 I need to sort them so they are ordered by value. ...
1
vote
2answers
131 views

How do I move a line in a text file up or down by one line?

I have some text files, and I'd like to be able to move an arbitrary line in any of the files up or down one line (lines at the beginning or end of the file would stay where they are). I have some ...
3
votes
3answers
2k views

Any tricks to speed up sort?

For example for bzip there is pbzip, a parallel version of bzip. Is there any such parallelization tools for sort to improve performance?
-1
votes
1answer
32 views

How to sort by column and break ties randomly

I have a tab-delimited file with three columns like this: joe W 4 bob A 1 ana F 1 roy J 3 sam S 0 don R 2 tim L 0 cyb M 0 I want to sort this file by decreasing values in the third column, but to ...
0
votes
3answers
55 views

sort command in unix with numeric sort

I have file1 : "$lvl=57" "$lvl=20" "$lvl=48" "$lvl=17" "$lvl=58" file2 i want : "$lvl=17" "$lvl=20" "$lvl=48" "$lvl=57" "$lvl=58" Basically numerically sort of file1
1
vote
4answers
81 views

Sort command on more than one field

I have a file with contents 192.168.1.3 192.168.1.4 192.168.1.2 10.1.1.1 10.1.1.2 10.1.1.3 192.168.1.56 192.168.1.43 10.1.1.23 When I gave $ sort -h -t. -k3,4 sort_test.txt 10.1.1.1 10.1.1.2 ...
1
vote
2answers
38 views

How do I transform array output sorted in lexicographic order to numeric order

I have a text file that consists of data in this form: int array[0] int array[1001] int array[1002] int array[1003] int array[2] int array[2001] int array[2002] ... In other words, the numbers ...
0
votes
1answer
181 views

Merging CSV (single line per row) files without losing the header line

I need to merge two CSV files: datasorted1.csv and datasorted2.csv, but my headers keep getting sorted. head -n 1 datasorted1.csv datasorted2.csv > resultado.csv && tail -n +2 ...
1
vote
1answer
54 views

How sort command column option is handled

I have a file like 1.2.3.4 1.1.1.1 4.5.6.7 1.1.3.5 When I gave sort -t. -k3 inp3 1.1.1.1 1.2.3.4 1.1.3.5 4.5.6.7 and when I gave sort -t. -k3,3 inp3 1.1.1.1 1.1.3.5 1.2.3.4 4.5.6.7 second ...
2
votes
1answer
66 views

Unix sort by multiple columns

I have trouble understanding unix sort. Consider the following file (tab separated) aa ~ a1 aa B b A b ~ e bb B bb ~ B When calling: cat tmp2 | sort -t $'\t' -k1,2 I get aa ~ a1 aa B ...
4
votes
5answers
281 views

sort order explanation

Giving a lot of files, sort -n gives me this result: f1.txt f10.txt f2.txt which doesn't seem right to me. What is the explanation? Can I get this by default: f1.txt f2.txt f10.txt
1
vote
3answers
56 views

Display distinct values of a list and number of occurences

I have a program that writes on the standard output a list of string values, one per line, and I would like to display in real time the list of distinct values along with the number of occurences for ...
12
votes
4answers
4k views

How can I remove duplicates in my .bash_history, preserving order?

I really enjoying using control+r to recursively search my command history. I've found a few good options I like to use with it: # ignore duplicate commands, ignore commands starting with a space ...
2
votes
1answer
77 views

counting occurrences of a string

I have a large database of this format: 0500506 gi|561108321|ref|NC_018143.2| 71446 0 0500506 gi|561108321|ref|NC_018143.2| 71447 0 0500508 gi|561108321|ref|NC_018143.2| 71449 0 0500509 ...
1
vote
3answers
233 views

How to tar files with a sorted order?

If you tar a directory recursively, it just uses the order from the os's readdir. But in some cases its nice to tar the files sorted. Whats a good way to tar a directory sorted alphabetically? ...
3
votes
3answers
266 views

sort lines according to day

I have a text file in following format, and I need to sort the lines according to date. Is there some simple way to do it (preferably in bash) ? 2013-May-30 2 2013-May-21 10 2013-Jun-27 8 ...
1
vote
1answer
90 views

How can I list hidden directories, then directories, then hidden files and finally files with ls?

I'm trying to figure out how to list the contents of a directory in the following group order: Hidden directories Non-hidden directories All other hidden files (including regular files, symlinks, ...
2
votes
0answers
68 views

Installed coreutils on CentOS for sort --human-readable flag but it still doesn't work

I'm trying to do du -h | sort -h, but my CentOS 5.7 doesn't have -h flag for sort. So I installed coreutils Updated: coreutils.x86_64 0:5.97-34.el5_8.1 Complete! yet sort still doesn't have ...
3
votes
2answers
47 views

Trying to sort on two fields first is second criteria

I'm trying to sort on multiple columns a file with GNU sort. The format of the source file tmp/file.txt is : 03A12345679 2014-09-30-00.00.01.00 "03" => first column "A" => field separator ...
3
votes
2answers
339 views

Sorting the output of `ps`

I am trying to understand why ps does not behave the way I expect it to. From the man pages, the following command should display ppid and lstart, with items sorted by lstart order. However when I run ...
2
votes
1answer
795 views

Cut and Paste Commands

So I have: $ cat fruits 2 bananas 3 cherries 4 figs 5 dates 6 elderberries 7 apples 8 grapes and 1 $ cat prices 2 2.18 3 4.11 4 1.69 5 4.52 6 1.73 7 1.01 8 1.09 Every line from 'fruits' ...
4
votes
7answers
992 views

Need some sed magic: Moving marked lines to the beginning of a file

I find myself in dire need of some sed magic (I really need to sit myself down and learn this). I have a file with lots of lines. After reviewing it, I've marked some lines by adding a star (*) to ...
1
vote
5answers
141 views

How to get the unique count of a particular part of a string

I have a set of data in a file. psf7433-nlhrms unit7433-nobody unit7333-opera bpx7333-operations app7333-osm unit7330-partners psf7331-pdesmond unit7333-projm mnp7330-redirect unit7333-retailbanking ...
2
votes
2answers
82 views

order for listing files

If I have a list of files under a directory ullman-000.pbm ullman-001.pbm ... ullman-098.pbm ullman-099.pbm ullman-1000.pbm ullman-1001.pbm ullman-1002.pbm ullman-1003.pbm ullman-1004.pbm ...
1
vote
1answer
73 views

Sort and backup log4j files

I apologize for length of this question. It's difficult to explain, even though I was fluent in English, which I'm not ;) Not really sure where post this question, because it has java, log4j library ...
5
votes
1answer
162 views

How to make a list generated by md5deep in alphabetical order of relative paths?

If I want make a md5sum list recursively, then I would use md5deep, but it starts to pop up some problems such as it won't generate the md5sum file in alphabetical order. For example, $ cd ...
2
votes
2answers
625 views

Sorting a csv file [duplicate]

I need to sort a CSV file, but the the header row (1st row) keeps getting sorted. This is what I'm using: cat data1.csv | sort -t"|" -k 1 -o data1.csv Here's a sample line: ...
2
votes
1answer
521 views

Sort and count number of occurrence of lines

I have Apache logfile, access.log, how to count number of line occurrence in that file? for example the result of cut -f 7 -d ' ' | cut -d '?' -f 1 | tr '[:upper:]' '[:lower:]' is a.php b.php a.php ...
1
vote
2answers
82 views

Compare and print data from different text files to one in Shell

How can I compare and print data from different text files to one in Shell. I have captures NAS details of three different boxes using SSH, now I need to combine all the three text files to one file ...
17
votes
3answers
1k views

Why is coreutils sort slower than Python?

I wrote the following script to test the speed of Python's sort functionality: from sys import stdin, stdout lines = list(stdin) lines.sort() stdout.writelines(lines) I then compared this to the ...
1
vote
2answers
330 views

How batch mv files to folders created and named by modification date?

I am quite new to the terminal stuff and I just couldn't find any answer on the aforementioned subject. On my OS X, I have a folder with about 4000 pics that I'd like to move into folders that are ...
3
votes
1answer
150 views

Sorting lists when there are no leading zeros

I would like to sort this numerically and the sort command keys off the first character. CPL_V11.01.00_1 CPL_V11.01.00_10 CPL_V11.01.00_2 CPL_V11.01.00_3 CPL_V11.01.00_35 CPL_V11.01.00_36 ...