1

I'm developing on a max osx 10.8 and have no preferences to which tool to use probably sed but it doesn't matter.

Update after comment: The input file uses windows new lines if that's complicates things I can pre process them to change the new lines to linux style.

Text

Text



text


text

text

And I would like an output file like this

Text

Text
text
text

text
2
  • 1
    Please format the posted data in a way we can use it for testing our solutions to be proposed.
    – manatwork
    Commented Aug 12, 2013 at 16:32
  • A new format is posted, let me know if this suffice.
    – user44971
    Commented Aug 12, 2013 at 16:52

2 Answers 2

1
awk -vRS='\r\n' -vORS='\r\n' '
  /./||!++n{if (n==1) print "";print;n=0}
  END{if (n==1) print ""}'
0

Or a perl variant:

perl -e 'undef $/; $_=<>; s/^\n{2,}//mg; print' < infile

unset (undef) the field separator ($/). This makes $_=<> assign the contents of STDIN into $_. Then search and replace 2 or more newlines with nothing (s/^\n{2,}//mg;), and print the output (these operations are done on $_). The m option on the search s/// above allows newline (multiple line) matching and g applies the pattern for all occurances in the string (and not just the first).

If it is to only work on 2 and 3 new lines (and not effect 4 or 5 etc) then use s/^\n{2,3}//mg;

For DOS format file:

perl -e 'undef $/; $_=<>; s/^(\r\n){2,}//mg; print' < infile
5
  • Thanks, worked really good. It had a shockingly fast execution time to. The only small thing, and it doesn't matter for me, was that it didn't remove 2 new lines at the beginning of the file.
    – user44971
    Commented Aug 12, 2013 at 22:01
  • Odd I'd tested for that in my tests :) May be those lines contain whitespace (tabs, spaces?) Glad it wasn't an issue!
    – Drav Sloan
    Commented Aug 12, 2013 at 23:53
  • You were correct it removes the first line also if formatted with utf8. Sadly the downloaded file was encoded with utf8 with BOM. Removing the BOM and the script worked fine also for the first two lines.
    – user44971
    Commented Aug 13, 2013 at 0:33
  • Same as perl -0777 -pe's/^\n{2,}//mg'. It should be noted that since it slurps the whole file in memory, it's not adapted to huge files. Commented Aug 13, 2013 at 6:07
  • I don't think that variant works, because doesn't -p split the STDIN across newlines (and so you can't do multiline matching?)
    – Drav Sloan
    Commented Aug 13, 2013 at 11:04

You must log in to answer this question.