1

I'm developing on a max osx 10.8 and have no preferences to which tool to use probably sed but it doesn't matter.

Update after comment: The input file uses windows new lines if that's complicates things I can pre process them to change the new lines to linux style.

Text

Text



text


text

text

And I would like an output file like this

Text

Text
text
text

text
  • 1
    Please format the posted data in a way we can use it for testing our solutions to be proposed. – manatwork Aug 12 '13 at 16:32
  • A new format is posted, let me know if this suffice. – Farmor Aug 12 '13 at 16:52
1

Or a perl variant:

perl -e 'undef $/; $_=<>; s/^\n{2,}//mg; print' < infile

unset (undef) the field separator ($/). This makes $_=<> assign the contents of STDIN into $_. Then search and replace 2 or more newlines with nothing (s/^\n{2,}//mg;), and print the output (these operations are done on $_). The m option on the search s/// above allows newline (multiple line) matching and g applies the pattern for all occurances in the string (and not just the first).

If it is to only work on 2 and 3 new lines (and not effect 4 or 5 etc) then use s/^\n{2,3}//mg;

For DOS format file:

perl -e 'undef $/; $_=<>; s/^(\r\n){2,}//mg; print' < infile
  • Thanks, worked really good. It had a shockingly fast execution time to. The only small thing, and it doesn't matter for me, was that it didn't remove 2 new lines at the beginning of the file. – Farmor Aug 12 '13 at 22:01
  • Odd I'd tested for that in my tests :) May be those lines contain whitespace (tabs, spaces?) Glad it wasn't an issue! – Drav Sloan Aug 12 '13 at 23:53
  • You were correct it removes the first line also if formatted with utf8. Sadly the downloaded file was encoded with utf8 with BOM. Removing the BOM and the script worked fine also for the first two lines. – Farmor Aug 13 '13 at 0:33
  • Same as perl -0777 -pe's/^\n{2,}//mg'. It should be noted that since it slurps the whole file in memory, it's not adapted to huge files. – Stéphane Chazelas Aug 13 '13 at 6:07
  • I don't think that variant works, because doesn't -p split the STDIN across newlines (and so you can't do multiline matching?) – Drav Sloan Aug 13 '13 at 11:04
1
awk -vRS='\r\n' -vORS='\r\n' '
  /./||!++n{if (n==1) print "";print;n=0}
  END{if (n==1) print ""}'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.