2

I am trying to get only matching strings (match_E2 and pattern_2) along with 1st column.

abcd.corp;;a123,Virtual,aws,Linux,Linux,match_E2,Database
web1.corp;;,Virtual,azure,match_E2,Linux,corpo,Database
web2.corp;;match_E2,Virtual,a2responsible,Linux_Suse,Linux,corpo,Database
web3.corp;;Virtual,Virtual,corpo,pattern_2,Linux,corpo,Database
web4.corp;;Virtual,Virtual,corpo,,Linux,pattern_2,Database

expected output could be below

abcd.corp,match_E2
web1.corp,match_E2
web2.corp,match_E2
web3.corp,pattern_2
web4.corp,pattern_2

I tried to use option -o in grep but it gives only matching strings.

5
  • 2
    Does it have to be grep, or is awk also an option? Your question tags seem to indicate this, just want to make sure. Also, how exactly are the patterns defined. Are the simple strings or regular expressions? Are they always the same for one processing run, or are you looking for multiple matches to be found in one pass? Also, do you actually have mixed delimiters (; and ,), and if so, can the match occur only in the ,-separated part? – AdminBee Aug 20 '20 at 10:47
  • anything either awk or grep or both would be ok, but more i am preferring in one line command, pattern is similar what i mentioned – Sin15 Aug 20 '20 at 10:53
  • The problem is you mentioned two patterns. Do you want to look for both in one single run of the command you are looking for, or is it okay to call the command twice (i.e. once for each search pattern). Also, is it ensured that the pattern can occur only once on every line, or could there be multiple occurences on the same line? – AdminBee Aug 20 '20 at 10:55
  • yes, pattern occurs only once in every line either match_E2 or pattern_2 or none. But not both at a time . – Sin15 Aug 20 '20 at 11:00
  • 1
    Does mismatch_E2 match match_E2? What about pattern_275 matching pattern_2? You said you wanted to match 2 strings but accepted an answer that matches 2 partial regexps - just confirming that a partial regexp match is really what you wanted rather than a full string match. – Ed Morton Aug 20 '20 at 23:24
5

I daresay your case might be better handled with sed.

For match_E2 pattern:

$ sed -nE 's/^([^;]+).*(match_E2).*/\1,\2/p' file.txt

For pattern_2 pattern:

$ sed -nE 's/^([^;]+).*(pattern_2).*/\1,\2/p' file.txt

For both those patterns in one go:

$ sed -nE 's/^([^;]+).*(match_E2|pattern_2).*/\1,\2/p' file.txt

That is, basically:

$ sed -nE 's/^([^;]+).*(    ).*/\1,\2/p' file.txt
#                       ^  ^
#                       |  |
#            ---------------------
# put within these two parentheses the same (Extended Regular Expression) pattern you would use with `grep -E`

Note that it only relies on at least one ; being the separator between the first field and the rest of the line.

4

The following awk command should do what you want:

awk -F'[;,]' -v pat="match_E2" '$0~pat{for (i=3;i<NF;i++) {if ($i ~ pat) printf("%s,%s\n",$1,$i)}}' file.txt
  • The -F'[;,]' option will tell awk to recognize both ; and , as field separator and split the line accordingly. Note that while the POSIX standard mandates that such a multi-character field separator is interpreted as full regular expressions, there may still be awk versions around that don't implement this correctly.

  • The pattern is passed to awk via the -v pat="match_E2" command-line option. Note that this will interpret the pattern as full regular expression. If you have characters in there that have a special meaning in this context, you need to escape them!

  • If the current line matches the pattern ($0 ~ pat means "if the entire line matches the regular expression stored in pat somewhere"), it will loop over all relevant fields (field 3 is the first field after the last ;) and identify the one that actually matched (the if ($i ~ pat) condition). It then prints the first field ($1) and the matching field ($i) via printf(). This assumes that there can only be one such field on a matching line!

If you look for multiple patterns, you can either formulate the regular expression in pat accordingly, as in

awk -F'[;,]' -v pat="match_E2|pattern_2" ' ... etc ... '

or run the command twice, once for each pattern.

2
  • Thanks @AdminBee , it works as expected. Can you please explain line - '$0~pat{for (i=2;i<NF;i++) {if ($i ~ pat) printf("%s,%s\n",$1,$i)}}' – Sin15 Aug 20 '20 at 11:12
  • 2
    @Sin15 I edited the answer to include more explanation. You may want to look at the GNU awk user's guide for more insight if you are interested. – AdminBee Aug 20 '20 at 11:18
4

A somewhat more grep -o like awk version, using the match function:

$ awk -F';' 'match($0,/match_E2|pattern_2/) {print $1 "," substr($0,RSTART,RLENGTH)}' file
abcd.corp,match_E2
web1.corp,match_E2
web2.corp,match_E2
web3.corp,pattern_2
web4.corp,pattern_2
1
  • could you please explain your command.. – Sin15 Aug 21 '20 at 4:57
1

Best I can offer is a very VERY ugly workaround:

# Finds first columns
first=(`grep "match_E2" text | awk -F';;' '{print $1}'`)

# Lets join them together now
for column in ${first[@]}; do
  echo "$column,match_E2"
done

Result:

abcd.corp,match_E2
web1.corp,match_E2
web2.corp,match_E2

You can also make a script or function and replace serach string with $1, and then call it with search string as argument. Like this:

Script:

#!/bin/bash
first=(`grep "$1" text | awk -F';;' '{print $1}'`)
for column in ${first[@]}; do
  echo "$column,$1"
done

And then you call it like this:

xxxx@ubuntu:~/test# ./script.sh match_E2
abcd.corp,match_E2
web1.corp,match_E2
web2.corp,match_E2
1
  • 1
    Thanks @pormulsys, i am looking one line command either awk/grep/or combination of both. btw, there are two matching strings (match_E2 and pattern_2) as mentioned in expected output. – Sin15 Aug 20 '20 at 10:59
1

This does full, literal string matching so it'll work even if your target strings contain regexp metacharacters or appear as substrings in your input:

$ awk '
    BEGIN { strs["match_E2"]; strs["pattern_2"]; FS=";"; OFS="," }
    { for (str in strs) if (index(","$NF",",","str",")) print $1, str }
' file
abcd.corp,match_E2
web1.corp,match_E2
web2.corp,match_E2
web3.corp,pattern_2
web4.corp,pattern_2

For an example of partial vs full matching, consider the following input:

$ cat file
abcd.corp;;a123,Virtual,aws,Linux,Linux,mismatch_E2,Database
web1.corp;;,Virtual,azure,match_E2,Linux,corpo,Database
web2.corp;;match_E2,Virtual,a2responsible,Linux_Suse,Linux,corpo,Database
web3.corp;;Virtual,Virtual,corpo,pattern_275,Linux,corpo,Database
web4.corp;;Virtual,Virtual,corpo,,Linux,pattern_2,Database

Note that now the first line of input contains mismatch_E2 instead of match_E2 and the 4th line is pattern_275 instead of pattern_2. Now run the above awk script on it and see that it produces the expected output:

$ awk '
    BEGIN { strs["match_E2"]; strs["pattern_2"]; FS=";"; OFS="," }
    { for (str in strs) if (index(","$NF",",","str",")) print $1, str }
' file
web1.corp,match_E2
web2.corp,match_E2
web4.corp,pattern_2

For an example of regexp vs string matching, change the match_E2 on line 1 to m.*2 and patch_2 on the 4th line to p.*2 in the input:

$ cat file
abcd.corp;;a123,Virtual,aws,Linux,Linux,m.*2,Database
web1.corp;;,Virtual,azure,m.*2,Linux,corpo,Database
web2.corp;;m.*2,Virtual,a2responsible,Linux_Suse,Linux,corpo,Database
web3.corp;;Virtual,Virtual,corpo,pattern_2,Linux,corpo,Database
web4.corp;;Virtual,Virtual,corpo,,Linux,pattern_2,Database

and modify the above awk script to look for m.*2 and p.*2 instead of match_E2 and pattern_2 and see that again it produces the expected output:

$ awk '
    BEGIN { strs["m.*2"]; strs["p.*2"]; FS=";"; OFS="," }
    { for (str in strs) if (index(","$NF",",","str",")) print $1, str }
' file
abcd.corp,m.*2
web3.corp,p.*2
0
awk -F "," '{for(i=1;i<=NF;i++){if($i ~ /match_E2|pattern_2/){print $1,$i}}}'  filename|sed "s/;.*[ ;]/,/g"

output

abcd.corp,match_E2
web1.corp,match_E2
web2.corp,match_E2
web3.corp,pattern_2
web4.corp,pattern_2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.