0

I want to remove the sequential lines that begins with the same pattern "CREATE CURRENT" using sed or awk for example if have the folowwing file:

CREATE CURRENT = 'aaaa' ;
CREATE CURRENT = 'bbbb' ;
CREATE CURRENT = 'aaaa' ;
CREATE CURRENT = 'dddd' ;
THIS IS A PARAGRAPH 1 ;
THIS IS A PARAGRAPH 1-1
CREATE CURRENT = 'cccc' ;
THIS IS A PARAGRAPH 2 ;
THIS IS A PARAGRAPH 2-2

the result would be like

CREATE CURRENT = 'dddd' ;
THIS IS A PARAGRAPH 1 ;   
THIS IS A PARAGRAPH 1-1 
CREATE CURRENT = 'cccc' ;
THIS IS a PARAGRAPH 2
THIS IS a PARAGRAPH 2-2 ;
  • 1
    This is just a particular case of a more general question. Anyway, with sed you would simply change the pattern: sed '$!N;/\(CREATE CURRENT\).*\n\1/!P;D' infile – don_crissti Mar 17 '16 at 10:59
  • @don-crissti yes it's working you have to post this answer – malmo Mar 17 '16 at 11:53
0

try

awk '{ if ( $1 "x" != before "x" ) print ; before=$1 ; } ' file

where

  • $1 "x" != before "x" test with previous value of first field
  • before "x" whith implicit string concat allow for uninitialized first before value
  • before=$1 remember previous $1 value.

Update with OP's edit:

awk 'BEGIN {cc=0 } 
    /CREATE CURRENT/ { if ( !cc++) print ; else next ;} 
    {cc=0; print ;} '
  • consecutives lines with CREATE CURRENT are printed only once
  • other lines reinitalize cc value
  • you can add new line within ' ' or put all in one line.
  • I edited my question, the line that shoud be removed if they are sequential are the ones that begin with 'CREATE CURRENT' – malmo Mar 17 '16 at 10:04
0

I tried this sed and it's working

sed '/^CREATE CURRENT/ { :a $! { N; /\nCREATE CURRENT/ { s/.*\n//; ba; }; }; }' file

here's the result

CREATE CURRENT = 'dddd' ;
THIS IS A PARAGRAPH 1 ;
THIS IS A PARAGRAPH 1-1
CREATE CURRENT = 'cccc' ;
THIS IS A PARAGRAPH 2 ;
THIS IS A PARAGRAPH 2-2
0

Open it up in vi and you can use:

:%s/\(\(\_^CREATE CURRENT\).*\n\)\(\2.*\n\)*/\1/g

This may leave an extra newline at the end of the file if you have CREATE CURRENT lines at the end of the file. If so, you can delete the extra line with :$d.

If you're all happy with it, save it with :x, or discard changes and exit using :q!


To keep the last of each set of duplicates instead of the first, use:

:%s/\(\(\_^CREATE CURRENT\).*\n\)\+\(\2.*\n\)/\3/g
  • thanks it's working but it kept the first line I want to keep the last occurence of the duplicate line I edit my question – malmo Mar 17 '16 at 10:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.