3

I have a file with a list of IP addresses, but some of the strings are not IP addresses and I want to replace such strings with a dummy IP address.

I am using this grep to search for IP; but don't know how to replace what does not match with a dummy IP address. I believe this can be done with sed. I tried few things but none of them worked.

cat file.txt | grep -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'

I have this sed to match IP addresses in the file, but I have no idea how to replace non IP addresses with a dummy IP.

sed -rn '/([0-9]{1,3}\.){3}[0-9]{1,3}/p' file.txt

Input:

192.168.10.20
00 03
10.28.214.5
192.168.10.40
BF
192.168.10.50

Desired Output:

192.168.10.20
192.168.0.0
10.28.214.5
192.168.10.40
192.168.0.0
192.168.10.50

Thanks!

2

The following answer uses awk and addresses the fact that a valid IPv4 address is not simply 4 tuples of up to 3 digits, but also the constraint on digits being smaller than 256:

awk -v dummy="192.168.0.0" -F'.' 'NF!=4 {$0=dummy}
    NF==4 {for (i=1;i<=4;i++) {if ( !($i~/^[0-9]{1,3}$/) || $i>255) {$0=dummy;break}}} 1' input.txt

This will specify the dummy IP as variable dummy. It will then consider the . as field separator and check if

  • there are exactly 4 fields, and
  • if each field consists of only 1-3 digits, with the resulting number being smaller than 256

If any condition is not met, replace the line with the dummy IP. After processing, the line (including any modifications made) is printed.

Note that your proposed RegExp would not only accept byte fields larger than 255, but for lack of anchoring also lines where only a sub-string happens to match "4 numbers of 1-3 digits, separated by periods".

7

Your regexp will match things which are not IPv4 dotted-quad addresses but look similar to them (e.g. 256.256.256.256 - looks a lot like an IPv4 address, but isn't).

To match only valid IPv4 addresses, you need to use a regex like:

(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)

or (without the perlish ?: non-capturing group modifiers):

((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)

and this needs to be properly anchored, e.g. with ^ and $, \b on both ends, or \< and \>.

See: Regular Expressions Cookbook by Jan Goyvaerts and Steven Levithan, published by O'Reilly Media, Inc.

e.g.

$ sed -E '/^((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$/!s/.*/192.168.0.0/' file.txt 
192.168.10.20
192.168.0.0
10.28.214.5
192.168.10.40
192.168.0.0
192.168.10.50
$ perl -p -e 's/.*/192.168.0.0/ unless m/^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$/' file.txt 
192.168.10.20
192.168.0.0
10.28.214.5
192.168.10.40
192.168.0.0
192.168.10.50

Both of these change the entire input line to 192.168.0.0 unless the line matches a valid IPv4 address.

Personally, I would use perl's Regexp::Common module, which is a large collection of regexes for common pattern-matching tasks, conveniently available in a hash called%RE.

$ perl -MRegexp::Common -p -e 's/.*/192.168.0.0/ unless m/^$RE{net}{IPv4}$/' file.txt 
192.168.10.20
192.168.0.0
10.28.214.5
192.168.10.40
192.168.0.0
192.168.10.50
6

NOTE: others have pointed out that your chosen regular expression for an IPv4 address is flawed. I will not attempt to address that here since it is covered well elsewhere.

You can use sed's change command on lines that do not (!) match your RE, ex.

$ sed -r '/([0-9]{1,3}\.){3}[0-9]{1,3}/!c\
192.168.0.0
' file.txt
192.168.10.20
192.168.0.0
10.28.214.5
192.168.10.40
192.168.0.0
192.168.10.50

With GNU sed, you can simplify to

sed -r '/([0-9]{1,3}\.){3}[0-9]{1,3}/!c192.168.0.0' file.txt
6
  • OMG THANKS A LOT.. I was so close; but guys are great! – Javier Gonzalez Jun 24 at 23:07
  • 1
    -r is GNU specific anyway. See -E for the (soon-to-be) standard equivalent. – Stéphane Chazelas Jun 25 at 6:49
  • Unfortunately, this would not only accept strings where the individual fields are larger than 255, but also strings like 123.4532.432.2.4 where only the trailing 532.432.2.4 actually "matches" (i.e. in general also lines where only a sub-string contains four 1-to-3-digit numbers separated by periods). – AdminBee Jun 25 at 11:14
  • @AdminBee IMHO the focus of the question is the mechanism for text replacement - I will leave critique of the OP's regex to others – steeldriver Jun 25 at 11:28
  • The same approach can be implemented with an s command instead of a c command. I tend to prefer s over c for single-line replacements because it does not require you to embed backslashes and newlines in the command. – John Bollinger Jun 26 at 12:29
5
sed '/^\([[:digit:]]\{1,3\}\.\)\{3\}[[:digit:]]\{1,3\}$/!s/.*/192.168.0.0/' data

If the line is not a valid IPv4 address then replace the current line with : 192.168.0.0.

I advise you to use another value for the dummy address instead of 192.168.0.0 but this is up to you and to youts needs.

Or the same but with awk:

awk '!/^([0-9]{1,3}\.){3}[0-9]{1,3}$/ {$0="192.168.0.0"}1' data

Or perl:

perl -MNet::IP -ple '$_ = "192.168.0.0" unless new Net::IP($_)' data
0
1

With Perl:

  • define the regex of an octet which comprises one to three digits with the constraint that it be nonzero leading unless it's single digit.
  • match that the line comprises exactly 4 octets separated by dots.
  • split the Line on dots and that all the tuples are less than 256.
dummy=192.168.0.0 \
perl -MList::Util=all -lpe '
  $octet //= qr/(?!0\d)\d{1,3}/;

  /^$octet(?:[.]$octet){3}$/ &&
   all { $_<256 } split /[.]/ or
     $_ = $ENV{dummy};
' file

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.