How to edit the entire file after match a grep pattern?

Question

To simplify, I want to edit the whole file after matched pattern, example of a file:

$ cat file
ip=x.x.x.a
mask=255.0.0.0
host=a
ip=x.x.x.b
mask=255.0.0.0
host=b
ip=x.x.x.c
mask=255.0.0.0
host=c
ip=x.x.x.x
blahblah
mask=255.0.0.0
host=d

Let's suppose that I want to edit the IP from host c, but note that this may be variable, so I don't know this value. If I grep host c and use the -B2 to print the lines I can't edit it in the original file! Another point the line may not have the same structure, that's the case of host d, between ip and mask info there's some text, so I can't assume that the IP pattern always will be 2 lines before my search pattern.

Resuming, I can't grep the IP directly because I don't know it, instead I need to search for host, and edit the line before this match to change the value of IP. How can I do this?

Answer 1

This gets a lot easier if you go through the file backwards. Fortunately, you can do so easily with tac (which happens to be cat backwards). We can then use a relatively simple awk script to look for your host, and change only its ip:

$ tac input | awk -v OFS="=" -v myip="changed_address" -v myhost="d" -F"=" '$1 == "host" { if( $2 == myhost ) { sw = "on" } else { sw="off" } } sw == "on" && $1 == "ip" { $2=myip } { print $0 }' | tac
ip=x.x.x.a
mask=255.0.0.0
host=a
ip=x.x.x.b
mask=255.0.0.0
host=b
ip=x.x.x.c
mask=255.0.0.0
host=c
ip=changed_address
blahblah
mask=255.0.0.0
host=d

I shall explain in detail how the awk works:

First, we declare a few variables: one each for the host and new value for the ip:

-v myip="changed_address" -v myhost="d"

Further, we declare the field separator for the input and output:

-v OFS="=" -F"="

Now, the actual awk script itself:

$1 == "host" {             // If we see the "host" line..
  if( $2 == myhost ) {     // And it matches the one we're looking for..
    sw = "on"              // Set a flag to swap the next IP
  } else { 
    sw="off"               // Otherwise, unset the flag
  }
} 

sw == "on" && $1 == "ip" { // If the flag is set and this is an IP line.. 
    $2=myip                // Swap in the new IP 
}

{
   print $0                // Finally, print out the processed line
}

Once that's all done, we just use tac again to re-reverse it, making it forwards again.

Answer 2

This will change the IP associated with host c to 1.2.3.4:

$ sed 's/^ip/\nip/' file | perl -00pe 'if(/\nhost=c\n/){s/ip=\S+/ip=1.2.3.4/} s/\n\n/\n/' 
ip=x.x.x.a
mask=255.0.0.0
host=a
ip=x.x.x.b
mask=255.0.0.0
host=b
ip=1.2.3.4
mask=255.0.0.0
host=c
ip=x.x.x.x
blahblah
mask=255.0.0.0
host=d

Explanation:

• sed 's/^ip/\nip/' file : add an extra newline (\n) to each line beginning with ip. I think this might not work with all implementations of sed, so if yours doesn't support this, replace the sed command with perl -pe 's/^ip/\nip/'. We need this in order to use Perl's "paragraph mode" (seen below).

• perl -00pe : the -00 makes perl run in "paragraph mode" where a "line" is defined by two consecutive newlines. This enables us to treat each host's block as a single "line". The -pe means "print each line after applying the script given by -e to it".

• if(/\nhost=c\n/){s/ip=\S+/ip=1.2.3.4/} : if this "line" (section) matches a newline followed by the string host=c and then another newline, then replace ip= and 1 or more non-whitespace characters (\S+) following it with ip=1.2.3.4.

• s/\n\n/\n/ replace each pair of newlines with a single newline to get the original file's format back.

If you want this to change the file in place, you can use:

tmp=$(mktemp); sed 's/^ip/\nip/' file > $tmp; 
perl -00pe 'if(/\nhost=c\n/){s/ip=\S+/ip=1.2.3.4/} s/\n\n/\n/' $tmp > file

Answer 3

Another option is to send some commands to ed:

h=c
ed -s file <<< $'/^host='"${h}"$'$\n?^ip=\nc\nip=new.ip.here\n.\nw\nq' > /dev/null

The basic idea is to edit the file and pipe in a newline-separated list of commands (in two ANSI C quote strings $' ... ') that search for the given host (in variable $h), then search backwards for a line that begins with ip=, then change that line to be something new, then save and quit ed. Broken out by newlines (\n), the commands are:

1. /^host= ... $h $ -- start a search (/) for the string host= at the beginning of the line (^), followed by the contents of $h, and ending the line ($). Relax the end-of-line requirement if your host isn't a complete match.

2. ?^ip= -- Now that we're on the matching line, search backwards for the text ip= at the beginning of a line.

3. c -- change this line

4. insert the text ip=new.ip.here

5. . -- end the insertion text

6. w -- write the file to disk

7. q -- quit ed

Invoking ed with -s silences the byte-count it emits when opening and saving the file. Redirecting the entire command to /dev/null silences the default output when ed finds the host= and ip= lines we're searching for.

Answer 4

I know you are expecting something very fast, and simple like a one-line sed command or a smart awk code, but if you don't care...

#!/bin/bash
#Note: Adjusted to run with a posix shell (tested in dash)
filename='file'
newip='127.0.0.1'
hostchar='d'

tac "$filename" | while IFS= read -r line ; do 
     case $line in 
          host=${hostchar})  
              flag=on 
          ;;
          host=*)
              flag=off
          ;;
     esac
     if [ "$flag" = "on" ]; then
     case $line in 
          ip=*) 
              echo ip=$newip
              continue 
          ;; 
          #you can replace more variables at once by adding it here
          #in the same standard. 
          #for ex: mask=*) echo mask=$newmask; continue ;; etc...
     esac  
     fi
     echo $line 
done | tac 

Results:

ip=x.x.x.a
mask=255.0.0.0
host=a
ip=x.x.x.b
mask=255.0.0.0
host=b
ip=x.x.x.c
mask=255.0.0.0
host=c
ip=127.0.0.1
blahblah
mask=255.0.0.0
host=d

Answer 5

$ awk -v host=c -v newip=zzz.zzz.zzz.zzz '$0 ~ "^host=" host "$" { print; getline; $0 = sprintf("ip=%s\s", newip) }; 1' file
ip=x.x.x.a
mask=255.0.0.0
host=a
ip=x.x.x.b
mask=255.0.0.0
host=b
ip=x.x.x.c
mask=255.0.0.0
host=c
ip=zzz.zzz.zzz.zzzs
blahblah
mask=255.0.0.0
host=d

This assumes that you'd like to change the IP address of some named host and that the IP address line is always occurring after the host= line for that host.

The awk program takes the host name and the new IP address on the command line by setting the two awk variables host and newip. The code then locates the host= line corresponding to the given hostname, reads and discards the following line (the ip= line) and creates a new ip= line with the new IP address. The data (modified or not) is outputted by the trailing 1 in the program.

Answer 6

A relatively short solution using negative lookahead assertions in a perl regular expression:

perl -p0e 's/(ip=)[^\n]*((?:(?!ip=).)*?host=c)/\1new_ip_for_host_c\2/gs' hosts.txt

Explanation:

First, a print-loop is used using the -p option. This iterates through lines of a file provided as argument to perl and prints them. If the line is changed, the changed variant is printed, as with sed.

The file contents are not actually treated as separate lines, the -p option merely serves to make perl read in the file at once and use the contents as the default string to match in, without using a variable. This is achieved via the -0 option, which makes perl treat multiline input as a single line, substituting newline characters by null bytes.

In the pattern, parts excluding the value to be changed are saved.

The value is recognized as anything until the next newline, which I assume is understood as the next null-byte in this situation. Someone who knows this could perhaps explain.

Then, using a negative lookahead assertion, all characters are matched that are not preceded by an ip= part, until the host=d part is encountered. This is necessary, otherwise perl would prefer the left-most match over the shortest, and would match from the very first ip= to the required host=c part.

The characters not preceded by ip= are matched in a non-greedy manner, otherwise again, the match would encompass everything from the first ip= until host=c, because by default, the longest match is preferred. In other terms, this ensures that text is matched in sections terminated by host=c, and started by the ip= segment that is closest above host=c, not furthest.

The character restricted by the negative lookahead expression inside the (?:) construct. It denotes a non-capturing group, meaning that it won't count toward the total number of backreferences in the substitution string. Grouping is used so the * sign can properly quantify the whole negative lookahead - dot character combo as a single atom. In the example, backreferences range from \1 - \2, whilst without the non-capturing group, they would be \1 - \3, with \3 denoting the nested group.

The backreferences \1 and \2 in the substitution string are the matched parts of the section that enclose exactly the ip address for the host found in the matched section.

The ip address will thus be substituted by new_ip_for_host_c.

General blablablah:

I am not sure if the data format was the OP's choice, just putting this out in general: it is better to store such simple data in such a manner that records are separated by newlines, i.e. there are no multiline records. Refer to the format of /etc/passwd . Most standard Unix filters are line-oriented, and data manipulation is much easier if you follow "da Unix wae". If your data is more complex (nested data items, etc.), you are better off using a format such as JSON and manipulating it with a proper parser.