New answers tagged

3

Because it's not used as much, I'll mention a comm-based solution: comm -12 <(sort A) <(sort B) This uses process substitution <( ... ) to sort files A and B and provide them as inputs to comm, which then uses -12 to: -1 suppress column 1 (lines unique to FILE1) -2 suppress column 2 (lines unique to FILE2) ... leaving only lines ...


1

Depending on the file size you could just use grep: grep -Fxf A B -f specifies a file from which to get a list of patterns. -x means to match the whole line only (disallow matching a part of a line). -F means to treat the patterns as fixed strings rather than regular expressions. If B is smaller than A you may get slightly faster results by naming B as ...


2

As the comments indicate, the problem seems likely to be blanks or carriage returns. Either of the following should do the trick: $ (sort -n A B) | sed -E 's/[^[:alnum:]]+$//' | uniq -d $ (sort -n A B) | tr -d '\r ' | uniq -d Some flavors of GNU sed use -r instead to get Extended Regular Expressions. tr is certainly simpler but also more brutal in ...


2

In addition to what don_crissti mentions about trailing spaces you may want to check the filetype / newline style as well. The man page for uniq states that it can: uniq - report or omit repeated lines If you have say CRLF i.e. Windows style newline characters instead of the expected LF, you might get surprises. You can quickly check the type with: ...


0

The program you're looking that can easily do what you want is called awk. :-) It can do programmed actions on matched RE patterns. Untested, simplified, rote, example awk program that should work with your example input and specified patterns: BEGIN { eights = 0; fives = 0; threes = 0; } /8888/ { eightln[eights] = $0; eights++; } ...


0

As a pure awk solution, try: tail -f log.file | awk ' $0!=last{ print $1 " " $4 " " $9} {last=$0}' This one prints a new output line only if the input line is different from the previous input line. As a slight variation, this one prints a new output line only if this output line differs from the previous output line: tail -f log.file | awk '{$0=$1" ...


5

One way in awk: awk '$0 != x ":FOO" && NR>1 {print x} {x=$0} END {print}' file Saves the line, then checks at the start of every line that it doesn't contain the saved string + :FOO. Print last line as it can't possibly have the next line have :FOO as there is none.


4

In the simplest case, to keep the lines without :FOO, you could just remove :FOO and then pass through uniq: $ sed 's/:FOO$//' file | uniq red.7 green.2 blue.6 yellow.9 If you prefer to keep the :FOO lines and assuming that they always come after their non-suffixed brethren, you could try: $ rev file | sed 's/:/ /' | uniq -f1 | sed 's/ /:/' | rev red.7 ...


3

How about joining adjacent pairs of lines, and then using a backreference to find the non-unique prefix? $ sed '$!N; /\(.*\)\n\1:FOO/D; P;D' file red.7 green.2:FOO blue.6 yellow.9:FOO Explanation: $!N - if we are not already at the last line, append the next line to the pattern space, separated by a newline /\(.*\)\n - match everything up to the newline ...



Top 50 recent answers are included