5

I want to add a dummy IP address but only after two consecutive duplicate lines are found.

I am working on a Linux system and this is my input file:

  IP_Remote_Address
     Address : 192.168.1.1
  IP_Remote_Address
     Address : 192.168.1.2
  IP_Remote_Address
     Address : 192.168.1.3
  IP_Remote_Address
  IP_Remote_Address
     Address : 192.168.1.4
  IP_Remote_Address
     Address : 192.168.1.5
  IP_Remote_Address
     Address : 192.168.1.6
  IP_Remote_Address
     Address : 192.168.1.7
  IP_Remote_Address
  IP_Remote_Address
     Address : 192.168.1.8

My desired output:

  IP_Remote_Address
     Address : 192.168.1.1
  IP_Remote_Address
     Address : 192.168.1.2
  IP_Remote_Address
     Address : 192.168.1.3
  IP_Remote_Address
     Address : NOT_FOUND
  IP_Remote_Address
     Address : 192.168.1.4
  IP_Remote_Address
     Address : 192.168.1.5
  IP_Remote_Address
     Address : 192.168.1.6
  IP_Remote_Address
     Address : 192.168.1.7
  IP_Remote_Address
     Address : NOT_FOUND
  IP_Remote Address
  Address : 192.168.1.8

I have this line line but it replaces only the first duplicate found:

awk '{print $0; if((getline nl) > 0){ print ($0!="IP_Remote_Address" && $0 == nl)? nl=$0"INSERT_NOT_FOUND_ABOVE" : nl }}' file.txt

I can later then use sed to replace the string INSERT_NOT_FOUND_ABOVE" with this:

sed '/INSERT_NOT_FOUND_ABOVE/i Address : NOT_FOUND' file.txt > new_file.txt

My only issue is that it can't detect all consecutive duplicates; it finds only the first one.

3
  • 2
    If your file can have multiple consecutive duplicates, please edit your example and show that. – terdon Jun 7 at 19:45
  • 2
    The last IP may be missing too, right? I.e. can the last line be IP_Remote_Address? In that case, should "Address : NOT_FOUND" be appended after it? (Despite it not being necessarily part of two or more consecutive duplicate lines). – fra-san Jun 7 at 21:42
  • The last line is fine. It already has an IP Addr. Its just only on duplicates that I need to inject the line. Works perfect! – Javier Gonzalez Jun 8 at 16:58
13

awk:

awk 'p==$0{print "     Address : NOT_FOUND"}{p=$0}1'

A rather naive solution.

  • p==$0 IF p == current line
    • THEN print not found
  • p=$0 SET p = current line
  • 1: print

Handles consecutive duplicate lines.


And as noted by @san-fran in comments under question, "The last IP may be missing too, right?" – Ups. Should have thought of that.

So:

awk -v e='Address : NOT_FOUND' 'p==$0{print e}{p=$0}END{if($1 ~ "IP")print e}1'
  • Set e = text to inject
  • p==$0 IF p == current line
    • THEN print variable e
  • p=$0 SET p = current line
  • END print e if current line contains IP
  • 1: print

Here the error-string has been added as a variable as we use it twice. (And trimmed for readability in this post).

3
  • Amazing!... That works for me!.. Thanks a LOT!... Im still wondering how that simple line detects the "two consecutive repeated lines" :) – Javier Gonzalez Jun 7 at 20:07
  • @JavierGonzalez: Yes, awk is powerful. Would have liked to add a sed option (mostly for fun) as well, but stuck if there are consecutive duplicates. – ibuprofen Jun 7 at 21:07
  • 1
    @JavierGonzalez : in that solution, p is meant to mean "previous line", and is compared to $0 (the whole current line) : if both are equal, it means we saw 2 consecutive identical lines, and we therefore print the e message (print will also add a newline character), before the last conditionnal ( 1, always true ) prints the current line ( an awk line is : ( condition) { action } . if action is not precised, it does the default action : print the current line if condition is true.) The answer would be clearer (it's a bit too "golfed") by adding a few newlines here and there... – Olivier Dulac Jun 8 at 9:10
6

Her is one way you could do it with a sliding-window in GNU sed:

parse.sed

# Handle last-line-error
$ { /IP/ s/$/\n   Address : NOT_FOUND/; }

# Always keep 2 lines in pattern-space
N

# If the lines are identical
/^([^\n]*)\n\1$/ { 

  # Add error text
  s/\n/\n   Address : NOT_FOUND\n/

  # Ensure we still only have 2 lines in pattern-space
  P
  s/[^\n]*\n//
}

# Print line 1 and delete it from pattern-space
P
D

Here is a modified test-text with tripple-error and last-line-error:

IP_Remote_Address
   Address : 192.168.1.1
IP_Remote_Address
   Address : 192.168.1.2
IP_Remote_Address
   Address : 192.168.1.3
IP_Remote_Address
IP_Remote_Address
IP_Remote_Address
IP_Remote_Address
   Address : 192.168.1.4
IP_Remote_Address
   Address : 192.168.1.5
IP_Remote_Address
   Address : 192.168.1.6
IP_Remote_Address
   Address : 192.168.1.7
IP_Remote_Address
IP_Remote_Address
   Address : 192.168.1.8
IP_Remote_Address

Run it like this:

sed -Ef parse.sed infile

Or as a one-liner:

<infile sed -E '${/IP/ s/$/\n   Address : NOT_FOUND/};N;/^([^\n]*)\n\1$/{s/\n/\n   Address : NOT_FOUND\n/;P;s/[^\n]*\n//};P;D'

Output in both cases:

IP_Remote_Address
   Address : 192.168.1.1
IP_Remote_Address
   Address : 192.168.1.2
IP_Remote_Address
   Address : 192.168.1.3
IP_Remote_Address
   Address : NOT_FOUND
IP_Remote_Address
   Address : NOT_FOUND
IP_Remote_Address
   Address : NOT_FOUND
IP_Remote_Address
   Address : 192.168.1.4
IP_Remote_Address
   Address : 192.168.1.5
IP_Remote_Address
   Address : 192.168.1.6
IP_Remote_Address
   Address : 192.168.1.7
IP_Remote_Address
   Address : NOT_FOUND
IP_Remote_Address
   Address : 192.168.1.8
IP_Remote_Address
   Address : NOT_FOUND
4

Using the GNU version of the stream editor utility sed in its extended regex mode:

sed -Ee '
  x;1d;G
  ${/\n\s*IP_/ba;}
  /^(.*)\n\1$/{
    g;:a;p;c\
   Address : NOT_FOUND
    b
  }
  $!s/\n.*//
' file
  • the current line goes to hold space and the previous line in the pattern space.
  • when there is a match between hold and pattern space (meaning, previous line and current line) we print the line + the not found line.
  • last line is /IP_/ then also we add the not found line.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.