0

I'm looking for a way to find all duplicate lines which are not "not a color" and add "is a color" at the end of second occurrence of them.

Here is a diff -y of what I'm talking about.

orginal file  - final resault

pink            pink
pink          | pink is a color
not a color     not a color
not a color     not a color 
violet          violet
violet        | violet is a color
not a color     not a color
not a color     not a color
orange          orange
orange        | orange is a color
not a color     not a color
  • but some duplicates remain unchanged like not a color. So it's an exclusion item. Or what? – RomanPerekhrest May 15 '17 at 20:13
  • Yeah forget about that... I'll edit it right now ;) – Ravexina May 15 '17 at 20:14
3

awk approach:

awk '{print $0; if((getline nl) > 0){ print ($0!="not a color" && $0 == nl)? 
     nl=$0" is a color" : nl }}' file

The output:

pink
pink is a color
not a color
not a color
violet
violet is a color
not a color
not a color
orange
orange is a color
not a color

You can use ‘getline var’ to read the next record from awk’s input into the variable var.

The getline command returns 1 if it finds a record and 0 if it encounters the end of the file.

$0!="not a color" && $0 == nl - if current record is not not a color string AND 2 consequtive lines are equal (dupicates)


An additional approach using substr() function (to insert " is a color " string after the first 2 chars of the crucial duplicate "color"):

awk '{print $0; if((getline nl) > 0){ print ($0!="not a color" && $0 == nl)? 
     nl=substr($0,1,2)" is a color "substr($0,3) : nl }}' file

The output would be:

pink
pi is a color nk
not a color
not a color
violet
vi is a color olet
not a color
not a color
orange
or is a color ange
not a color
  • Here is a thing, can we instead of adding "is a color" at the end of the line "$0", put it between $0 characters? say "vi is a color olet" instead of "violet is a color" (I really should learn awk and sed :\ ) – Ravexina May 15 '17 at 20:41
  • @Ravexina, yes, I can do it for violet, but how about pink, orange. There should be the common rule of replacement. You know what I mean – RomanPerekhrest May 15 '17 at 20:43
  • Yeah I know, a sample on violent is good enough I guess; then I can work on it myself. – Ravexina May 15 '17 at 20:44
  • @Ravexina, should it insert "is a color" after the first 2 chars? – RomanPerekhrest May 15 '17 at 20:45
  • that would be great ;) – Ravexina May 15 '17 at 20:46
1

From the information provided so far:

sed 'N;s/^\([a-z]*\)\n\1$/& is a colour/;$! P;$! D' file

The pattern [a-z]* may need to be adapted to your needs. Of course it doesn't match only colors, here it matches every low-letter word.

Explanation: What the script for each line is to append the following line with command N, so you alway have teo consecutive lines with a newline in between. Then the s has the pattern for the first line as a backreference \1 after the newline, thus it matches for repeated lines only. In this case the & in the replacement string inserts the wholemmatch and appends the given text to the second line. Then P prints up to the first newline and D removes this part, so the second line remains to start over. $! makes those commands executed for all but the last line, because for the last line we need to output both lines, which happends by default at the end of the script.

Testing for input:

pink
pink
not a colour
not a colour
orange
orange
not a colour
red
blue
blue

Gives output:

pink
pink is a colour
not a colour
not a colour
orange
orange is a colour
not a colour
red
blue
blue is a colour
1
sed -e '
   # not interested in empty lines or blank lines
   /^$/b
   /\S/!b

   N;                        # get the next line into pattern space
   /^\(.*\)\n\1$/!{P;D;};    # compare 2 in the pattern space as string eq
   /\nnot a color$/b;        # 2 EQUAL, they are "not a color" => NOP
   s/$/ is a color/;         # 2 EQUAL, but not "not a color" => suffix
' your_colors.file
0
awk '/Not a color/ { print } /pink|red|blue|red|orange/ { if( found[$1] ) { print $1, "is a color" } else { print $1; found[$1]=1 } }' /path/to/input
  • If your list of "colors" is known, add them to the conditional match. – DopeGhoti May 15 '17 at 20:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.