Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a document with a lot of empty lines.

How can I remove them when there are 2 or more together.

I tried sed "s/\n\n//" file but it didn't work. No error.

share|improve this question
2  
Do I read you correctly if you do not want to remove all blank lines, but only if it is two or more. So not single blank lines? – Runium May 16 '13 at 13:31
1  
And if it's two or more lines shall really all of them be deleted or just all but one? – Hauke Laging May 16 '13 at 13:44
up vote 24 down vote accepted

Just to remove empty lines:

sed  '/^$/d'

sed is line oriented, so thinking in terms of "2 or more of a particular byte" works except when that byte is a newline. Then you have to think of something that works for the whole line.

share|improve this answer
    
Of course! +1 for simple elegance. – terdon May 16 '13 at 13:22
2  
sed is capable of handling several lines via its "pattern space" / "hold space" feature. But I feel that's too complicated. ;-) – Hauke Laging May 16 '13 at 13:42
    
This will not function as desired if the first character of the file is a newline. – Chris Down May 17 '13 at 3:42
    
@ChrisDown - are you certain? I tried it on just such a file, and it seemed to work with (GNU sed) 4.2.2. – Bruce Ediger May 17 '13 at 13:04
1  
To make it work when the first character is a newline (if that's really a requirement), then you can enclose the command with a negative address 1! (match all except line 1), thus: sed '1!{/^$/d'}. – Toby Speight Jan 28 at 19:51

No need for sed. grep will do:

grep .

(that's grep, SPC, dot, that is match any line containing at least one character).

There's also:

tr -s '\n'

(squeeze any sequence of newline characters into one).

As noted by Chris, both are not equivalent because removing empty lines (like the first solution above and most other answers focus on here) is not the same as squeezing sequences of newline characters as requested in the case where the first line is empty as it only takes one leading newline character to make the first line empty.

share|improve this answer
1  
This will not function as desired if the first character of the file is a newline: sprunge.us/FLAJ – Chris Down May 17 '13 at 3:42

sed is not the best tool for that, since it is line based and treats \n as the end-of-line character this gets complicated. Having seen @Bruce Ediger's answer sed may well be the perfect tool for the job, still, here are some other options:

  1. Perl

    perl -ne 'print if /./' file.txt
    

    or

    perl -pe '$/=""; s/\n+/\n/;' file.txt 
    

    Thanks to @ruakh who made me go and read this:

    $/

    The input record separator, newline by default. This influences Perl's idea of what a "line" is. Works like awk's RS variable, including treating empty lines as a terminator if set to the null string (an empty line cannot contain any spaces or tabs). You may set it to a multi-character string to match a multi-character terminator, or to undef to read through the end of file. Setting it to "\n\n" means something slightly different than setting to "" , if the file contains consecutive empty lines. Setting to "" will treat two or more consecutive empty lines as a single empty line. Setting to "\n\n" will blindly assume that the next input character belongs to the next paragraph, even if it's a newline.

  2. gawk/awk

    awk '$1' file.txt
    

    That will work for the example posted but as @Stephane Chazelas pointed out, it will also delete lines whose first field "looks like" 0. This is more robust:

    awk NF file.txt
    
share|improve this answer
    
For Perl, perl -pe 's/\n+/\n/ file.txt will do, the input record separator is irrelevant for this use. – vonbrand Jan 28 at 22:09
    
@vonbrand no, perl -pe or perl -ne work line by line. \n+ will never match because it is only applied on a single line. That's why you need to either set $/ or use -0 ti slurp the file whole: perl -0pe 's/\n+/\n/' file. – terdon Jan 29 at 9:48

What do you mean remove? remove duplicate (many blank line to one) or remove all?

If you want to remove duplicate, here is the method using sed:

sed '$!N; /^\(.*\)\n\1$/!P; D'

It simulates uniq command.

The best choice is using awk:

awk NF <filename>
share|improve this answer

If you wanted to keep a single blank line for any given sequence of blank lines you might do:

sed -e '/./b' -e :n -e 'N;s/\n$//;tn'
share|improve this answer

Try sed -e 's#\\n\\n#\\n#g' input.file > output.file using / both as your field separator and part of your regex could be the problem.

share|improve this answer
    
Just gave this a whirl with one of my files containing double and triple newlines in a sequence. Doesn't work at all for me. – syntaxerror Nov 7 '13 at 19:53

For most of these answers it is first necessary to remove trailing whitespace. Removing doubled up newlines removes all blank lines. (Think about this).

Literally interpreted the OP wants "all blank lines removed from a file if there are any repeated blank lines".

The typical user wants to "remove only duplicated blank lines".

To do this, strip trailing whitepace first, and pipe though cat -s

sed  s/[[:space:]]*$// | cat -s

And yet this will not remove a superflous leading or trailing blank line.

share|improve this answer
    
Downvoted, but this clearly works ? No comment ? – mckenzm Feb 7 '15 at 8:48

Use this command:

tr -s '\r' '\n'
share|improve this answer
    
yes, their answer didn't work for me. – meow Jan 28 at 19:03
4  
AFAIK this answer is incorrect. I recommend yo to delete it. – zuazo Jan 28 at 19:10
    
oh, its because my file contains lots of newlines and carriage returns actually. 0x0d0a – meow Jan 28 at 19:11
1  
Actually, the command removes repeated lines with windows end of line. Test with echo -e 'one\r\n\r\n\r\n\rtwo'| tr -s '\r' '\n'. The command tr will translate all \r to \n and then will squeeze all \n to just one. So, it works, not sure what to do with the fact that this apply to windows, not UNIX. – BinaryZebra Jan 28 at 19:32

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.