Sorting text files or other data, for example with the sort utility.

learn more… | top users | synonyms

3
votes
2answers
231 views

How to use output of one command as input in another command

I made a copy of all the backups made on Wed of every week The time stamp of each file is not sorted but the day is Wednesday of every file Now, I need to sort each file based on time Stamp e.g if ...
4
votes
3answers
515 views

Sort using custom pattern

Is there a way to output file's contents using custom patterns? For instance, having a file myfile with following contents: a d b c ..how would one sort it using following pattern: print lines ...
4
votes
2answers
307 views

sort part of a file

How can I sort the file by the second column from the terminal. The content of the file is: Nome Note ------------ Mehdi 0 Sunday 20 Others 10 Shnou 5 Using sort -t' ' -nk2 is not ...
3
votes
2answers
1k views

Specify the sort order with LC_COLLATE so lowercase is before uppercase

Given the file: $ cat file 1 a C B 2 c 3 A b By default sort will: $ sort file 1 2 3 a A b B c C With LC_COLLATE=C so will sort in uppercase letter before lowercase: $ LC_COLLATE=C sort file 1 ...
2
votes
6answers
947 views

sort ls output by users

On Linux, is there a way to sort ls output by users? What I try to achieve is something like this: user_a   file1 user_a   file2 user_b   another_file user_c   this_file user_c ...
1
vote
2answers
223 views

How to remove duplicate entries from a file?

I have a comma-separated list of domain-names in a file; unfortunately there are duplicate entries for some domain-names. Is there a way to find and remove the duplicate entries from the file? The ...
0
votes
3answers
452 views

Sorting IP address according to the second field in file

in my ksh script I need to add the following task ( OS - linux/solaris ) I have the following file more test.txt /etc/backup/app 172.1.120 172.110.120 /etc/backup/app 172.1.120.12 ...
1
vote
3answers
243 views

another question with sort in bash [duplicate]

How can I ignore the space in "Kuala Lumpur" when sorting this list? I cheated it by tabbing the columns and sorted on the tab which gives me the right results but I would like to know how to deal ...
1
vote
3answers
986 views

Sort with unequal whitespace in first column

I need to sort this list by name, high temp and low temp: Kuala Lumpur 78 56 Seoul 85 66 Karachi 95 75 Tokyo 85 60 Lahore 85 75 Manila 90 85 I figured since whitespace is the delimiter for a column ...
5
votes
2answers
491 views

Unix sort out of memory

I'm sorting a 25 GB file containing 200M lines on Fedora 18 using: sort -S 10G -T /bigdisk bigfile and I am getting process killed due to no memory. The process isn't maxing out my RAM before death ...
2
votes
2answers
397 views

Sort lines by unicode value

I'm trying to sort a text file linewise by their Unicode values. As far as I can tell, this means numerals first, then letters, then CJK-Ideographs. However, sort (with LC_ALL=C) fails horribly at ...
2
votes
0answers
35 views

Unexpected ordering of strings in `sort` command [duplicate]

I'm trying to sort a text file using certain columns in the file as a sort key. Since the file doesn't have separators and fields sometimes stick to each other, I have to convert spaces to some ...
1
vote
1answer
86 views

Sort aligned columns

I have a program which output looks like this # Status VAL1 VAL2 Description 1 some text 2.5 4.4 Some long other text 5 some text 2 14.5 2.0 Some long other ...
2
votes
2answers
192 views

Select greatest numbered filename

Simple requirement but can't find anything online which can achieve it. I have a list of dated files as below... filename_20120101.dat filename_20120102.dat filename_20120103.dat I ...
1
vote
3answers
466 views

Reading a directory in shell script

I want to read a numbered directory which is under a path, into a variable in shell script. The paths are something like this: .../releases/R1/... .../releases/R2/... .../releases/R3/... Each time ...
3
votes
2answers
181 views

Delete duplicate images. Need Software for computing average color of an image

I want to delete duplicates in a massive bunch of images. Well as I have dups of the same picture in a different resolution I will make the deletion myself. BUT I want to do this in linear time. So I ...
1
vote
1answer
1k views

How to match 2 large file and print the difference in shell script

I have 2 files. File1 has 400k numerical records. e.g: 1 2 3 4 5 6 .. and so on File 2 also has 420k numerical records. e.g: 1 2 3 4 6 .. and so on Both these file are in unsorted manner. I want ...
22
votes
1answer
374 views

Why is sort changing the order of lines with identical sort keys?

Here is the data: D 2 B 2 A 2 When I run this command: sort -k2,2 file it outputs: A 2 B 2 D 2 My question is that when I specify only the second column -k2,2, why is it that it sorts by the ...
1
vote
1answer
846 views

Sorting by one column then the

Say i have a output file with following: 010513 08:04 ROUTER1 192.168.1.1 68.130.5.10 HIT(1) 010513 08:04 ROUTER1 172.168.1.1 28.130.5.10 HIT(1) 010513 08:04 ROUTER1 122.168.1.1 ...
0
votes
1answer
241 views

How does -k flag work in sort command? [duplicate]

I have got a dataset like this: manufacturer,model,year,mileage,price plym fury 1970 73 2500 chevy malibu 1999 60 3000 ford mustang 1965 45 10000 volvo s80 1998 102 9850 ford thundbd 2003 15 10500 ...
4
votes
1answer
273 views

Why is uniq ignoring Unicode and lines with a single letter?

I'm trying to combine both the American and British dictionaries into one large dictionaries, and I'm trying to remove all the duplicates from the superset, but it seems that uniq is not outputting ...
3
votes
1answer
516 views

Sort command not working as expected

I have got a dataset like this tack2@domain.com,2009-11-27 overflow@domain2.com,2009-11-27 overflow@domain2.com,2009-11-27 When I am running command to delete all of the same entries of column2 ...
3
votes
2answers
352 views

sort a file based on length of the column/row

I need to sort a file based on the number of chars in the first column. I have no idea on how to go about this. (On Linux, so sed/awk/sort is available). An example: .abs is bla bla 12 .abc is bla ...
1
vote
2answers
199 views

`echo -e 'Q\ns\nV' | sort` doesn't sort

Why do : $ echo -e 'Q\ns\nV' | sort outputs Q s V without changing the order of my list (taking in account the lower/uppercase ?)
2
votes
2answers
434 views

Reorder folders on FAT32 drive

My car can read MP3 files from a FAT32 flash drive. I have a folder for each artist on the drive, then MP3 files in those folders. However the car does not read the folders alphabetically, but by the ...
0
votes
1answer
109 views

Remove repeated words with suffix in a file

I have a list of records in a file as in below example word_word1 word_word1_acc word_word2_acc word_word3_acc word_word4 I need to remove only word_word1_acc line in the file as it is only a ...
1
vote
2answers
436 views

Deleting specific lines and duplicates from a 11gb wordlist text file

I have an 11gb wordlist file which is already sorted as each word is on its own line. I need to remove duplicates and lines starting from 077. I guess I need to run sed and sort -u together but I ...
2
votes
1answer
706 views

How can I list my `at` jobs by date?

I am using atq command, but I need to list my at jobs by date. How is that possible ?
1
vote
3answers
2k views

strange problem: join: file 2 is not in sorted order

sort flash_int_list.txt|join finish_comm - > t1 join: file 2 is not in sorted order I have sorted flahs_int_list.txt but it still says file 2 is not in sorted order what is wrong ? the ...
5
votes
1answer
1k views

comm: file is not in sorted order

I used comm to compare two sorted files. Each line in these files are positive integer numbers. But the results show comm: file 1 is not in sorted order comm: file 2 is not in sorted order How ...
5
votes
1answer
189 views

Alphabetizing names in `ls` alias with .files not intermingled

I am currently using the following ls alias: alias ls='ls -alhGkpsq --color=auto'. This results in following directory listing. [03:35] bryson@brys ~ :$ ls total 48K 4.0K drwx------ 4 bryson 4.0K ...
11
votes
2answers
9k views

Trying to sort on two fields, second then first

I am trying to sort on multiple columns. The results are not as expected. Here's my data (people.txt): Simon Strange 62 Pete Brown 37 Mark Brown 46 Stefan Heinz 52 Tony Bedford 50 John Strange 51 ...
3
votes
1answer
4k views

The simplest method to count lines matching specific patterns, including '0' if line is not found?

I have very big logs (several gigabytes per day), that can (but do not need to) contain specific lines. I have to count the number of occurences of every one of these lines on a daily basis. I have a ...
1
vote
3answers
183 views

How to sort and join at the same time?

I have 2 files, one having 2 columns, another having 1 column. The second file is sorted using sort -u. Now the task is I need to join this column with the first column of the first file, which is not ...
5
votes
4answers
507 views

How to shift filename numbers without collisions?

Consider this list of files: $ touch index-{1,2,3,4,5,6,7,8,9}.txt If I want to shift them down so they start at zero, it's relatively easy: $ rename --verbose 's/^index-([1-9])\.txt$/$1/; ...
4
votes
1answer
2k views

How to find out common elements between two files?

For an example, I have 2 files having following info: File #1: 12 13 14 15 File #2: 12 1 13 2 14 2 15 6 16 7 17 8 Output File: 1 2 2 6 In the output file, I want only the second column ...
1
vote
1answer
315 views

Sort -u Function Command Not Working

As you can see in the picture attached screen shot After I used the sort command to sort a VERY large wordlist file I then split them up using the Split command. From there I confirmed that the ...
10
votes
3answers
3k views

How can I remove duplicates in my .bash_history, preserving order?

I really enjoying using control+r to recursively search my command history. I've found a few good options I like to use with it: # ignore duplicate commands, ignore commands starting with a space ...
3
votes
1answer
1k views

File size limit exceeded in bash [closed]

I have tried this shell script on a SUSE 10 server, kernel 2.6.16.60, ext3 filesystem. The script has a line like this: cat file | awk '{print $1" "$2" "$3}' | sort -n > result The file's size ...
0
votes
1answer
695 views

Sort a file based on 1 column [duplicate]

Possible Duplicate: how do you sort du output by size? I have a text file generated from the output of du --max-depth 1 -h > sizeOfHomeFolder.txt It's contents are in this format: $ ...
3
votes
4answers
707 views

Sort input file by the results of a regex

I'd like to sort a file based on the results of a regex expression. For example, if I have the following property declarations in Obj-C @property (nonatomic, strong) id <AlbumArtDelegate, ...
5
votes
3answers
2k views

What's the default order of Linux sort?

For a long period I thought the default behavior of the sort program was using ASCII order. However, when I input the following lines into sort without any arguments: # @ I got: @ # But ...
10
votes
3answers
4k views

How to sort the string which combined with string + numeric using bash script?

This is the data what I want to sort. But sort treats the numeric to string, the data it no sorted as I expected. /home/files/profile1 /home/files/profile10 /home/files/profile11 ...
3
votes
2answers
809 views

Using sort with alphanumeric filenames

I'm sorting the results of a find command which finds all of the files in the current directory: find . -maxdepth 1 -type f -iname "*.flac" | sort What I'm expecting is a list like this: ./Track ...
2
votes
2answers
1k views

Sort by regular expressions

I've got a set of POSIX regular expressions* ^BEGIN:VCARD\r$ ^VERSION[^A-Z] ^FN[^A-Z] ^N[^A-Z] ^NICKNAME[^A-Z] ^EMAIL[^A-Z] ^X-\([A-Z-]*\) ^TEL[^A-Z] ^ADR[^A-Z] ^ORG[^A-Z] ^TITLE[^A-Z] ^BDAY[^A-Z] ...
9
votes
4answers
4k views

How do I make ls sort underscore characters first?

I like being able to name files and directories with an underscore prefix if it's something I want to keep separate from other files and directories at the same level. On Windows and Mac, for example, ...
4
votes
6answers
6k views

Get text-file word occurrence count of all words & print output sorted

I had a command which would work through a text file, count all the occurrences of the words and print it out like this: remy@box $˜ magic-command-i-forgot | with grep | and awk | sort ./textfile.txt ...
1
vote
3answers
252 views

Numerically sorting files [duplicate]

Possible Duplicate: List files sorted numerically How can I sort this according to the number of last field? I tried with sort -t ' ' -k9,10n: -rw-r--r-- 1 root root 440 May 14 08:52 ...
7
votes
4answers
197 views

Sort fields inline

I'm trying to sort within a line of input over an unknown number of fields: Input: ab bc bc ab cd ef bc bc cd ef cd bc ab ef ab bc cd gh Output: ab bc ab bc bc cd ef bc cd ef ab cb cd ab bc cd ...
5
votes
4answers
798 views

For each subfolder, sort files by name and rename them to sequential padded numbers (regardless of extension)

I have the following directory structure: . ├── event-a │   ├── album-a │   │   ├── a.jpg │   │   └── x.png │   └── album-b │   ├── a.jpg │   ├── x.png │   └── y.gif └── event-b ├── ...