0

Say I have a text file with the following lines:-

abcd/efgh/a.jar
{
abcd/efgh/a.class
cdef/ghij/b.class
klmn/opqr/c.class
}
lkmn/opqr/b.zip
{
abcd/efgh/a.class
cdef/ghij/b.class
}
abcd/efgh/a.jar
{
cdef/ghij/b.class
}

Now abcd/efgh/a.jar in the first case has abcd/efgh/a.class, cdef/ghij/b.class and klmn/opqr/c.class within the curly braces. Consider it as 1 block of text. Now abcd/efgh/a.jar below again has cdef/ghij/b.class within curly braces. I want to remove this section/block of text. So the final output needs to be like:-

abcd/efgh/a.jar
{
abcd/efgh/a.class
cdef/ghij/b.class
klmn/opqr/c.class
}
lkmn/opqr/b.zip
{
abcd/efgh/a.class
cdef/ghij/b.class
}

Any help would be highly appreciated :)

3
  • What if the second a.jar block contains d.class? Add it to the first block or leave it as a block of it's own?
    – Philippos
    Jun 29, 2017 at 6:23
  • Good question. If the second a.jar has a d.class, then it needs to be merged to the first block. So the first a.jar block needs to have d.class as well
    – AJS
    Jun 29, 2017 at 6:37
  • I expanded your example to include this case
    – Philippos
    Jun 29, 2017 at 7:48

3 Answers 3

2

Use

for i in `awk '/}/ {if (NR!=1) print "";next} \
                {printf "%s ",$0,"}"}END{print ""}' yt.txt \
        |awk '{print $1}'|sort|uniq \
    `; \
    do \
        awk '/}/ {if (NR!=1) print "";next} \
            {printf "%s ",$0,"}"}END{printf ""} \
            ' yt.txt \
         |grep "$i"|sed 's/ /\n/g'|grep -v "$i"|sort|uniq \
            |awk -v var="$i" ' NR==1 {printf var} {print $0} END {print "}"}'  \
    ;done \

Same command in 1 line below (for copying purpose)

for i in `awk '/}/ {if (NR!=1) print "";next} {printf "%s ",$0,"}"}END{print ""}' yt.txt|awk '{print $1}'|sort|uniq` ; do awk '/}/ {if (NR!=1) print "";next} {printf "%s ",$0,"}"}END{printf ""}' yt.txt|grep "$i"|sed 's/ /\n/g'|grep -v "$i"|sort|uniq|awk -v var="$i" ' NR==1 {printf var} {print $0} END {print "}"}' ;done

Explanation:

The for part will return you the unique heading of the block (abcd/efgh/a.jar,lkmn/opqr/b.zip) and pass it to do block. The do part will first grep all the rows for each heading, which would include duplicates also. Then it will exclude the heading and merge all the remaining rows under that heading block, then add the heading at first row. And hardcode } at the end.

Example

bash-4.2$ cat yt.txt
abcd/efgh/a.jar
{
abcd/efgh/a.class
cdef/ghij/b.class
klmn/opqr/c.class
}
lkmn/opqr/b.zip
{
abcd/efgh/a.class
cdef/ghij/b.class
}
abcd/efgh/a.jar
{
cdef/ghij/b.class
d.class
}



bash-4.2$ for i in `awk '/}/ {if (NR!=1) print "";next} {printf "%s ",$0,"}"} \
> END{print ""}' yt.txt |awk '{print $1}'|sort|uniq` \
> ; do awk '/}/ {if (NR!=1) print "";next} {printf "%s ",$0,"}"}END{printf ""}' yt.txt \
>  |grep "$i"|sed 's/ /\n/g'|grep -v "$i"|sort|uniq \
> |awk -v var="$i" ' NR==1 {printf var} {print $0} END {print "}"}'\
> ;done
abcd/efgh/a.jar
{
abcd/efgh/a.class
cdef/ghij/b.class
d.class
klmn/opqr/c.class
}
lkmn/opqr/b.zip
{
abcd/efgh/a.class
cdef/ghij/b.class
}
1
  • Happy to help. Feel free to accept the answer so the question could be closed
    – Utsav
    Jun 29, 2017 at 9:19
1

After I saw the solution utilizing a for loop with awk and sort and uniq and grep and sed I tried a solution with one tool instead of six:

sed ':a
  N;$!ba
  y/\n_/_\n/;s/^/_/
  :b
  s/\(_[^_]*_{\)\([^}]*\)\(_[^_}]*\)\(_[^}]*\)\(_}.*\)\1\([^}]*\)\3_/\1\2\3\4\5\1\6_/;tb
  :c
  s/\(_[^_]*_{\)\([^}]*\)_}\(.*\)\1\([^}]*\)_}/\1\2\4_}\3/;tc
  s/^_//
  y/\n_/_\n/' yourfile

does the job, but I have to admit that the regular expressions are easiert to write than to read ... (-;

2
  • So you started to use the y/\n_/_\n/ now ;-)
    – user218374
    Jun 29, 2017 at 12:07
  • @RakeshSharma Indeed. But I still dislike it! (-; Well, for a messy script like this it doesn't matter. In real life I would use python instead for a task like this. Thank you once again for introducing me to that workaround.
    – Philippos
    Jun 29, 2017 at 14:50
0
perl -alF'/\n[}{]\n/' -0777ne '
   for ( 0 .. $#F/2 ) {
      my $i = 2*$_;
      my($k,$v) = @F[$i,$i+1];
      if ( exists $h{$k} ) {
         $h{$k} .= join $\, grep { ! exists $seen{$k,$_} } split $\, $v;
      } else {
         push @k, $k;
         $seen{$k,$_}++ for split $\, $h{$k} = $v;
      }
   }
   print "$_\n{\n$h{$_}\n}" for @k;
' yourfile

Results

abcd/efgh/a.jar
{
abcd/efgh/a.class
cdef/ghij/b.class
klmn/opqr/c.class
}
lkmn/opqr/b.zip
{
abcd/efgh/a.class
cdef/ghij/b.class
}

Working

The input file is slurped and then split up into fields based on the field separator mentioned by the -F option. We will be getting even number of elements in the array @F. The even numbered then go in as the keys of the hash %h whilst their corresp. values are grabbed from the next odd value.

The hash %h is populated by splitting up the odd numbered elems on record separator ($\ = \n). At the same time we place the key into the array @k so that we can retrieve hash elements in order they were encountered.

All the while only those odd elements are used that are not already seen.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.