Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Is there a way to output file's contents using custom patterns?

For instance, having a file myfile with following contents:

a
d
b
c

..how would one sort it using following pattern: print lines starting with "b" first, then print lines starting with "d" and then print lines in normal alphabetical order, so expected output is:

b
d
a
c
share|improve this question
add comment

3 Answers

up vote 3 down vote accepted

You would need to use something more than just the sort command. First grep the b lines, then the d lines and then sort anything without the b or d at the end of that.

grep '^b' myfile > outfile
grep '^d' myfile >> outfile
grep -v '^b' myfile | grep -v '^d' | sort >> outfile
cat outfile

will result in:

b
d
a
c

This is assuming that the lines start with the 'pattern' b and d if that is the whole pattern or something inside the line you can leave out the caret (^)

A one-line equivalent would be:

(grep '^b' myfile ; grep '^d' myfile ; grep -v '^b' myfile | grep -v '^d' | sort)
share|improve this answer
    
This method works, but can it be refactored into a one-liner that only outputs to STDOUT instead of writing to a file in chunks? –  Sergey May 12 '13 at 11:45
    
@Sergey Just don't use redirections. –  Rany Albeg Wein May 12 '13 at 12:07
    
I see, so: grep '^b' myfile; grep '^d' myfile; cat myfile| grep -v '^b' | grep -v '^d' | sort –  Sergey May 12 '13 at 12:20
    
@Sergey you would want to inclose the stuff in ( and ) otherwise you cannot redirect the combined output –  Anthon May 12 '13 at 12:52
    
Yes, also I can see that cat was unnecessary, thanks! I think we've got a bit overwhelmed but a perfectly working one-liner. Can we utilize awk to make it more efficient? –  Sergey May 12 '13 at 13:13
show 3 more comments

When you need data to be sorted beyond sort's capability, a common approach is to pre-process the data to prepend a sort key, then sort, and finally remove the extra sort key. For example, here, add a 0 if a line starts with b, a 1 if a line starts with d, and a 2 otherwise.

sed 's/^b/0&/; t; s/^d/1&/; t; s/^/2/' |
sort |
sed 's/^.//'

Note that this sorts all the b and d lines. If you want these lines in the original order, then the easiest approach is to split off the lines that you want to leave unsorted. You can, however, work the original line into a sort key with nl — but here it's more complicated. (Replace \t by a literal tab character throughout if your sed doesn't understand that syntax.)

nl -ba -nln |
sed 's/^[0-9]* *\t\([bd]\)/\1\t&/; t; s/^[0-9]* *\t/z\t0\t/' |
sort -k1,1 -k2,2n |
sed 's/^[^\t]*\t[^\t]*\t//'

Alternatively, use a language such as Perl, Python or Ruby that lets you easily specify a custom sort function.

perl -e 'print sort {($b =~ /^[bd]/) - ($a =~ /^[bd]/) ||
                     $a cmp $b} <>'
python -c 'import sys; sys.stdout.write(sorted(sys.stdin.readlines(), key=lambda s: (0 if s[0]=="b" else 1 if s[0]=="d" else 2), s))'

or if you want to leave the b and d lines in the original order:

perl -e 'while (<>) {push @{/^b/ ? \@b : /^d/ ? \@d : \@other}, $_}
         print @b, @d, sort @other'
python -c 'import sys
b = []; d = []; other = []
for line in sys.stdin.readlines():
    if line[0]=="b": b += line
    elif line[0]=="d": d += line
    else: other += line
other.sort()
sys.stdout.writelines(b); sys.stdout.writelines(d); sys.stdout.writelines(other)'
share|improve this answer
    
The idea of pre-processing sounds promising, however neither of your sed examples really work for me –  Sergey May 13 '13 at 7:54
    
The second sed example was nonsense and I've fixed it, but the first works for me. What's wrong with it? –  Gilles May 13 '13 at 21:55
    
With first example I get an error when t; is inside, when I replace t with \t, literal tab or take it off I get following output: 0b\n 1d\n a\n c\n which is close but b and d are left with integers –  Sergey May 14 '13 at 17:59
add comment

One way of solving this using awk would be:

sort myfile | awk '$0 ~ /^b/ || $0 ~ /^d/ {print} $0 !~ /^b/ && $0 !~ /^d/ { a[f++] = $0 } END { for (word = 0; word < f; word++) { print a[word] } }'
share|improve this answer
    
+1 have you tried if this runs faster than the (multiple) grep invocations? –  Anthon May 12 '13 at 14:28
    
@Anthon Yes, according to my results AWK method runs in average 2 times faster. Check out this gist for details: gist.github.com/sergeylukin/5564524 –  Sergey May 12 '13 at 18:57
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.