How to make grep for a regex that appear multiple times in a line

Question

I want to grep a regex. The pattern I am searching for may appear multiple times in a line. If the pattern appeared many times, I want to separate each occurrence by a comma and print the match only not the full line in a new file. If it did not appear in a line I want to print n.a.

Example. I want to use this regex to find numbers in the pattern: [12.123.1.3].

grep -oh "\[\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\]" 'filename'

input file (input.txt)

blabla [11.335.2.33] xyuoeretrete [43.22.11.88] jfdfjkfbs [55.66.77.88]
blabla [66.223.44.33]
foo bar
blabla [1.2.33.3] xyuoeretrete [42] bla[1.32.2.4]

intended result in a new file (output.csv):

11.335.2.33,43.22.11.88,55.66.77.88
66.223.44.33
n.a.
1.2.33.3,1.32.2.4

Note: I use Ubuntu

Answer 1

With perl:

perl -lne '
  if (@ips = /\[(\d{1,3}(?:\.\d{1,3}){3})\]/g) {
    print join ",", @ips;
  } else {
    print "n.a.";
  }'

Or using the regexp for dotted-quad-decimal IPv4 addresses from Regexp::Common (libregexp-common-perl package on Debian-based systems such as Ubuntu):

perl -MRegexp::Common=net -lne '
  if (@ips = /\[($RE{net}{IPv4})\]/g)
    print join ",", @ips;
  } else {
    print "n.a.";
  }'

With -n, input can be given either on stdin or read from files whose paths are given as extra arguments or from the output of some commands if passed arguments like some commands|. perl's print prints to stdout by default, which from your shell you could redirect into a file with redirection operators such as >, >> (append), or 1<> (like > except the file is not truncated first, and is opened in read+write mode), and possibly more depending on your shell.

You can also add the -i option for the output to end up replacing the contents of the input files (whose paths must then be given as arguments).

Here, to take the input from a file called input.txt, and overwrite or create a output.csv file with the output:

< input.txt perl... > output.csv

Answer 2

Using GNU awk for the FPAT:

awk -v FPAT='\\[([0-9]{1,3}[.]){3}[0-9]{1,3}\\]' -v OFS=, '
{
    $1=$1; print (gsub(/[][]/, "")?$0:"N/A")
}' <infile >output

or with any POSIX awk (which all supports the {x,y} RE intervals):

awk '
{
    bkup=$0;
    gsub(/\[([0-9]{1,3}[.]){3}[0-9]{1,3}\]/, "|")
    gsub(/[][()\\.{}?+*$^]/, "\\\\&")
    n=split(bkup, tmp, $0)
    for(i=1; i<=n; i++){
        if(tmp[i]!=""){
            gsub(/[][]/, "", tmp[i])
            printf ("%s", (sep?",":"") tmp[i])
            sep=","
        }
    }; print (sep?"":"N/A"); sep=""
}' <infile >output

---

Output is written into the file output.

$ cat output
11.335.2.33,43.22.11.88,55.66.77.88
66.223.44.33
N/A
1.2.33.3,1.32.2.4

Note that your input should not contain | and & characters for the second approach.

---

Same code with inline explanation:

awk '
{
    #backup from the current record
    bkup=$0;

    #replace desired pattern all with "|" characters 
    #to build regexp patterns of everything other than our desired pattern
    gsub(/\[([0-9]{1,3}[.]){3}[0-9]{1,3}\]/, "|")

    #escape all regexp operators except "|"
    gsub(/[][()\\.{}?+*$^]/, "\\\\&")

    #split the original record (from bkup) into tmp on regexp
    # from the result of the first gsub() above
    n=split(bkup, tmp, $0)

    #loop through the splitted fields on the tmp array
    for(i=1; i<=n; i++){

        #if the current field is not empty
        if(tmp[i]!=""){

            #remove the ], [ characters from it
            gsub(/[][]/, "", tmp[i])

            #and print it (will add comma when it is the second or the next one)
            printf ("%s", (sep?",":"") tmp[i])

            #set comma as the field seperator when at least one field was printed
            sep=","
        }

    #print "N/A" in case there was no field and var "sep" did not set above
    # and then unset the "sep" var
    }; print (sep?"":"N/A"); sep=""

}' <infile >output

Answer 3

Executable awk file filter.awk:

#! /usr/bin/awk -f
{
    ret = ""
    line = $0
    while (match(line, /\[([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}\]/) > 0) {
        if (ret != "") {
            ret = ret ","
        }
        ret = ret substr(line, RSTART, RLENGTH)
        line = substr(line, RSTART + RLENGTH)
    }
    if (ret != "") {
        print ret
    }
}

Executed like that:

./filter.awk filename

Answer 4

Using Perl in sed mode (-p), (-l) shall remove LF (\n) from every record read and also append to every record printed.

perl -plse '
  $_ = "@{[/\[\K\d{1,3}(?:\.\d{1,3}){3}(?=])/g]}" || "n.a.";
' -- -\"=, file

---

GNU sed in extended regex mode (-E)

sed -E '
  s/\[([0-9]{1,3}(\.[0-9]{1,3}){3})]/\n\1\n/g

  y/\n_/_\n/
    s/[^_]*_([^_]*_)/\1/g
    s/_?[^_]*$//
  y/\n_/_,/

  /./!c n.a.
' file

---

Using awk with progressive matching of the regex

_m0="n.a." \
awk -v OFS=, '
BEGIN {
  msg = ENVIRON["_m0"]
  _d = "[[:digit:]]{1,3}"
  fmt = sprintf("%s%%s([.]%%s){%%d}%s", "[[]", "[]]")
  ip_re = sprintf(fmt, _d, _d, 3)
}
{
  t=$0;$0=""
  for (i=1; match(t,ip_re)>0; i++) {
    $i = substr(t, RSTART+1, RLENGTH-2)
    t = substr(t, RSTART+RLENGTH)
  }
}
!NF{print msg};NF' file

---

Output:

11.335.2.33,43.22.11.88,55.66.77.88
66.223.44.33
n.a.
1.2.33.3,1.32.2.4

---

Answer 5

Using Raku (formerly known as Perl_6)

No value-checking:

raku -ne 'if m:g/ ( \d**1..3 )**4 % "." / { $/.join(",").put } else {"n.a.".say};'    

OR

raku -ne 'm:g/ ( \d**1..3 )**4 % "." / ?? $/.join(",").put !! "n.a.".say;'  

Sample Input:

blabla [11.335.2.33] xyuoeretrete [43.22.11.88] jfdfjkfbs [55.66.77.88]
blabla [66.223.44.33]
foo bar
blabla [1.2.33.3] xyuoeretrete [42] bla[1.32.2.4]

Sample Output (both examples):

11.335.2.33,43.22.11.88,55.66.77.88
66.223.44.33
n.a.
1.2.33.3,1.32.2.4

At least in Perl-family languages, what you're asking for is a match, not grep. hence use of the m/.../ match operator, made 'global' as in m:g/.../ to return multiple instances of the match [which differs from grep, which returns the full element (such as a line, etc.), containing a match].

Briefly, clusters of 1-to-3 digits ( \d**1..3 ) are searched for, with these digits **4 repeated 4 times, and having % "." a period between each instance, and this regex match is searched for globally (m:global or m:g), meaning to get all instances of the match per element (line, etc.), not just the first match. First example: if found (if) output the match contained in the $/ match variable, else say n.a.. Second example: same match conditional used in Raku's ternary operator, i.e. condition ?? True !! False. Thus if the conditional is ?? (True), output the match contained in the $/ match variable, if the conditional is !! False say n.a..

---

Below WITH value-checking:

raku -ne 'if m:g/ ( \d**1..3 <?{ $/ < 256 }> )**4 % "." / { $/.join(",").put } else {"n.a.".say};'    

OR

raku -ne 'm:g/ ( \d**1..3 <?{ $/ < 256 }> )**4 % "." / ?? $/.join(",").put !! "n.a.".say;'  

Sample Input: same as above

Sample Output (both examples):

43.22.11.88,55.66.77.88
66.223.44.33
n.a.
1.2.33.3,1.32.2.4

Immediately above, Raku code is shown that checks each 1-to-3 digit cluster to ensure it is an integer less than 256. The additional regex element, <?{ $/ < 256 }> , is a positive assertion containing a {...} code-block that checks if $/ match-variable is less than 256. Reference here.

https://raku.org