45

I have a document with a lot of empty lines.

How can I remove them when there are 2 or more together.

I tried sed "s/\n\n//" file but it didn't work. No error.

Improve this question
3
  • 4
    Do I read you correctly if you do not want to remove all blank lines, but only if it is two or more. So not single blank lines?
    – Runium
    May 16, 2013 at 13:31
  • 1
    And if it's two or more lines shall really all of them be deleted or just all but one?
    – Hauke Laging
    May 16, 2013 at 13:44
  • You're only asking about replacing empty-lines (^$), not (escaped) double newlines within a line: aaa\n\nbbb?
    – smci
    Feb 10, 2020 at 23:21

9 Answers 9

Reset to default
60 
sed '/^$/d'

sed is line-oriented, so thinking in terms of "2 or more of a particular byte" works, except when that byte is a newline. Then you have to think of something that works for the entire line.

Improve this answer
12
  • 2
    sed is capable of handling several lines via its "pattern space" / "hold space" feature. But I feel that's too complicated. ;-)
    – Hauke Laging
    May 16, 2013 at 13:42
  • 1
    This will not function as desired if the first character of the file is a newline.
    – Chris Down
    May 17, 2013 at 3:42
  • 2
    To make it work when the first character is a newline (if that's really a requirement), then you can enclose the command with a negative address 1! (match all except line 1), thus: sed '1!{/^$/d'}.
    – Toby Speight
    Jan 28, 2016 at 19:51
  • 1
    @AaronFranke - yes, but that's a facet of how Linux shells treat '>' redirection. The shell looks at the command line, sees a '>' redirection of stdout to a file, creates that file, and only then runs sed. Creating a file will essentially delete any existing file with the same name. sed '/^&/d' file.txt > otherfile.txt will work.
    – user732
    Dec 10, 2018 at 19:31
  • 3
    This doesn't seem to answer the question as it removes all empty lines, not just consecutive empty lines.
    – Kusalananda ♦
    Mar 5, 2022 at 7:31
27

No need for sed. grep will do:

grep .

(that's grep, SPC, dot, that is match any line containing at least one character).

There's also:

tr -s '\n'

(squeeze any sequence of newline characters into one).

As noted by Chris, both are not equivalent because removing empty lines (like the first solution above and most other answers focus on here) is not the same as squeezing sequences of newline characters as requested in the case where the first line is empty as it only takes one leading newline character to make the first line empty.

Improve this answer
3
  • 2
    This will not function as desired if the first character of the file is a newline: sprunge.us/FLAJ
    – Chris Down
    May 17, 2013 at 3:42
  • Can we only remove two newlines like \n\n and keep single newlines?
    – alper
    Jul 10, 2021 at 14:41
  • The task was not to remove all empty lines but to leave single empty lines alone and remove multiple consecutive empty lines, i.e. \n{3,} --> \n\n.
    – Kusalananda ♦
    Mar 5, 2022 at 7:33
23

If you wanted to keep a single blank line for any given sequence of blank lines you might do:

sed -e '/./b' -e :n -e 'N;s/\n$//;tn'
Improve this answer
2
  • 6
    This is the only answer (besides cat -s) that actually accomplishes exactly what the question asked as I understand it. (And it's better than cat -s because I can use sed -i with it.)
    – Matthew
    Jun 5, 2018 at 15:25
  • 3
    could you explain it plz?
    – Hacktivista
    Aug 28, 2020 at 16:32
11

What do you mean remove? remove duplicate (many blank line to one) or remove all?

If you want to remove duplicate, here is the method using sed:

sed '$!N; /^\(.*\)\n\1$/!P; D'

It simulates uniq command.

The best choice is using awk:

awk NF <filename>
Improve this answer
2
7

sed is not the best tool for that, since it is line based and treats \n as the end-of-line character this gets complicated. Having seen @Bruce Ediger's answer sed may well be the perfect tool for the job, still, here are some other options:

  1. Perl

    perl -ne 'print if /./' file.txt

    or

    perl -pe '$/=""; s/\n+/\n/;' file.txt

    Thanks to @ruakh who made me go and read this:

    $/

    The input record separator, newline by default. This influences Perl's idea of what a "line" is. Works like awk's RS variable, including treating empty lines as a terminator if set to the null string (an empty line cannot contain any spaces or tabs). You may set it to a multi-character string to match a multi-character terminator, or to undef to read through the end of file. Setting it to "\n\n" means something slightly different than setting to "" , if the file contains consecutive empty lines. Setting to "" will treat two or more consecutive empty lines as a single empty line. Setting to "\n\n" will blindly assume that the next input character belongs to the next paragraph, even if it's a newline.

  2. gawk/awk

    awk '$1' file.txt

    That will work for the example posted but as @Stephane Chazelas pointed out, it will also delete lines whose first field "looks like" 0. This is more robust:

    awk NF file.txt
Improve this answer
2
  • For Perl, perl -pe 's/\n+/\n/ file.txt will do, the input record separator is irrelevant for this use.
    – vonbrand
    Jan 28, 2016 at 22:09
  • @vonbrand no, perl -pe or perl -ne work line by line. \n+ will never match because it is only applied on a single line. That's why you need to either set $/ or use -0 ti slurp the file whole: perl -0pe 's/\n+/\n/' file.
    – terdon ♦
    Jan 29, 2016 at 9:48
7

For most of these answers it is first necessary to remove trailing whitespace. Removing doubled up newlines removes all blank lines. (Think about this).

Literally interpreted the OP wants "all blank lines removed from a file if there are any repeated blank lines".

The typical user wants to "remove only duplicated blank lines".

To do this, strip trailing whitepace first, and pipe though cat -s

sed  s/[[:space:]]*$// | cat -s

And yet this will not remove a superflous leading or trailing blank line.

Improve this answer
0
0
  1. Clean up all lines that only have blank spaces and/or tabs by removing the spaces and tabs
  2. Remove all DOUBLE blank lines (i.e. \n\n), while preserving at least one

To do so (with thanks to @cuonglm for part of the code)

sed 's|^[ \t]\+$||' your_filename | sed '$!N; /^\(.*\)\n\1$/!P; D'

Replace your_filename with the actual filename.

Improve this answer
1
  • -1 : This removes any consecutive duplicate lines in addition to the redundant EOLs. So while this (otherwise helpful) command does respond to the OP, it is also data-destructive.
    – TonyG
    Sep 20, 2020 at 20:28
-2

Try sed -e 's#\\n\\n#\\n#g' input.file > output.file using / both as your field separator and part of your regex could be the problem.

Improve this answer
1
  • 2
    Just gave this a whirl with one of my files containing double and triple newlines in a sequence. Doesn't work at all for me.
    – syntaxerror
    Nov 7, 2013 at 19:53
-2

If your file is using CRLF (DOS/Windows format) line-endings, try:

tr -s '\r' '\n'
Improve this answer
5
  • yes, their answer didn't work for me.
    – meow
    Jan 28, 2016 at 19:03
  • 5
    AFAIK this answer is incorrect. I recommend yo to delete it.
    – zuazo
    Jan 28, 2016 at 19:10
  • oh, its because my file contains lots of newlines and carriage returns actually. 0x0d0a
    – meow
    Jan 28, 2016 at 19:11
  • 2
    Actually, the command removes repeated lines with windows end of line. Test with echo -e 'one\r\n\r\n\r\n\rtwo'| tr -s '\r' '\n'. The command tr will translate all \r to \n and then will squeeze all \n to just one. So, it works, not sure what to do with the fact that this apply to windows, not UNIX.
    – user79743
    Jan 28, 2016 at 19:32
  • Works for me. Created a file, words in vi with separated by multiple lines. cat file | tr -s '\r' '\n' returned the correct output.
    – cutrightjm
    Feb 4, 2020 at 1:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .