Linux - Identify a multi-pattern in a set of lines and replace

Question

Input Contents:

objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetorgperson
objectClass: org-abc
objectClass: org-xyz

objectClass: top
objectClass: inetOrgPerson
objectClass: org-abc
objectClass: organizationalPerson
objectClass: person

objectClass: top
objectClass: org-abc
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: org-xyz

objectClass: top
objectClass: inetOrgPerson
objectClass: org-xyz
objectClass: organizationalPerson
objectClass: person

---

Read a LDIF file of size 50 MB.

The contents between two new-lines are considered as a BLOCK.

• if both the lines (objectClass: org-abc & objectClass: org-xyz) are present in any order in a BLOCK, then remove those 2 lines in a BLOCK and add a new line as "objectClass: org-111"

(OR)

• if this line "objectClass: org-abc" alone is present in a BLOCK, then replace that line with "objectClass: org-222"

(OR)

• if this line "objectClass: org-xyz" alone is present in a BLOCK, then replace that line with "objectClass: org-333"

Expected Output:

objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetorgperson
objectClass: org-111

objectClass: top
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: org-222

objectClass: top
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: org-111

objectClass: top
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: org-333

How can I get such output using Linux commands (sed or awk) or suggest me a better approach?

Answer 1

Complex AWK solution:

awk 'function process(a,c) {       # process the lines of one passed block
         for (i=1; i<=c; i++) {    
             split(a[i], fields);  # split the line into 2 fields
             if (fields[2]=="org-abc") abc="222"; 
             else if (fields[2]=="org-xyz") xyz="333"; 
             else print a[i] 
         } 
         if (abc || xyz) printf "objectClass: org-%s\n",(abc && xyz? "111" : (abc? "222":"333")) 
     }
     !NF{ process(a, c); c=abc=xyz=0 }
     { a[++c]=$0 }
     END{ process(a, c) }' file

---

This is Memory sufficient solution, cause the array a will hold the lines of one single block only during the whole processing time. (counter c is getting reset on each next block)

The output:

objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetorgperson
objectClass: org-111

objectClass: top
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: org-222

objectClass: top
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: org-111

objectClass: top
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: org-333

Answer 2

This is a typical use case of Perl's "paragraph mode" (-00) where "lines" are defined by \n\n, so each paragraph is treated as a single line:

$ perl -00 -lpe 'if(/: org-abc/ && /: org-xyz/){
                    s/(^|\n)[^\n]+: (org-abc|org-xyz)\s*(?=$|\n)//g;
                    s/$/\nobjectClass: org-111/;
                 }
                 else{
                    s/objectClass: org-abc/objectClass: org-222/; 
                    s/objectClass: org-xyz/objectClass: org-333/
                 }' file
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetorgperson
objectClass: org-111

objectClass: top
objectClass: inetOrgPerson
objectClass: org-222
objectClass: organizationalPerson
objectClass: person

objectClass: top
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: org-111

objectClass: top
objectClass: inetOrgPerson
objectClass: org-333
objectClass: organizationalPerson
objectClass: person

Here's the same thing uncondensed into a script for clarity:

#!/usr/bin/env perl

## Paragraph mode
local $/="\n\n";

my $pat1 = 'objectClass: org-abc';
my $pat2 = 'objectClass: org-xyz';

## Read input file
while (my $line = <>) {
  ## Remove trailing newlines
  chomp($line);

  if($line =~ /$pat1/ && $line=~ /$pat2/){
    $line =~ s/(^|\n)($pat1|pat2)\s*(?=$|\n)//g;
    $line =~ s/$/\nobjectClass: org-111/;
  }
  else{
    $line =~ s/$pat1/objectClass: org-222/;
    $line =~ s/$pat2/objectClass: org-333/
  }
  print "$line\n\n";
}

Answer 3

Also easy with sed:

sed '/^$/!{H;1h;$!d;};x
  /objectClass: org-abc/!{s/\(objectClass: org-\)xyz/\1333/;p;d;}
  s/\(objectClass: org-\)xyz/\1111/;t1
  s/\(objectClass: org-\)abc/\1222/;:b
  :1
  s/\nobjectClass: org-abc//'

The first line is to collect one block in the pattern space, the rest does the obvious replacements.