10

I would like to remove empty lines from begin and the end of file, but not remove empty lines between lines. I think sed or awk would be the solution.

Source:

1:
2:
3:line1
4:
5:line2
6:
7:
8:

Output:

1:line1
2:
3:line2
  • those numbers 1: are not actually there? – RomanPerekhrest Nov 14 at 15:05
  • @RomanPerekhrest it means the line number. – Feriman Nov 14 at 15:06
5

Try this,

To remove blank lines from the begin of a file:

sed -i '/./,$!d' filename

To remove blank lines from the end of a file:

sed -i -e :a -e '/^\n*$/{$d;N;ba' -e '}' file

To remove blank lines from begin and end of a file:

sed -i -e '/./,$!d' -e :a -e '/^\n*$/{$d;N;ba' -e '}' file

From man sed,

-e script, --expression=script -> add the script to the commands to be executed

b label -> Branch to label; if label is omitted, branch to end of script.

a -> Append text after a line (alternative syntax).

$ -> Match the last line.

n N -> Add a newline to the pattern space, then append the next line of input to the pattern space. If there is no more input then sed exits without processing any more commands.

  • 1
    Note that -i is a non-portable extension to the POSIX sed utility and will not be available on all systems. – Andrew Henle Nov 15 at 11:27
  • i see that these commands work, but I'm not quite sure how. Could you explain them in more detail? In particular, in the second example, why doesn't the first clause delete embedded blank lines? Why does the second clause need to loop? It looks like it gets a bunch of newlines at once. Does any of this work on white space-only lines or are you considering them non-blank? – Joe Nov 16 at 16:41
  • Please explain the individual commands, how they are working and what is the meaning of those flags. – Prvt_Yadav Nov 17 at 10:26
6

This little awk program will remove empty lines at the start of a file:

awk 'NF {p=1} p'

So we can combine that with tac that reverses lines and get:

awk 'NF {p=1} p' file | tac | awk 'NF {p=1} p' | tac
line1

line2

Stealing @guillermo chamorro's command substitution trick:

awk 'NF {p=1} p' <<< "$(< file)"
5

If the file is small enough to fit memory requirements:

$ perl -0777 -pe 's/^\n+|\n\K\n+$//g' ip.txt
line1

line2
  • -0777 to slurp entire input file
  • ^\n+ one or more newlines from start of string
  • \n\K to prevent deleting newline character of last non-empty line
  • \n+$ one or more newlines at end of string
  • 2
    or with (\s*\n)+ in stead of \n+ to also remove lines that only contain whitespace. – ilkkachu Nov 14 at 16:04
5

I proppose this:

printf '%s\n' "$(cat file)" | sed '/[a-z]/,$!d'

It will print the whole text except start-end blank lines. So, if we extend the example:

(blank)
(blank)
line1

line2
line1

line2
line1

line2
line1



line2
(blank)
(blank)

It will output:

line1

line2
line1

line2
line1

line2
line1




line2
  • 2
    Clever. The trick here is that command substitution ($(cat file)) strips off trailing newlines. I'd offer 2 suggestions: 1) use the bash builtin $(< file) instead of cat; 2) use a here string: sed '/[^[:blank:]]/,$!d' <<< "$(<file)" – glenn jackman Nov 15 at 18:17
1

A simple 2 pass approach just for completeness:

$ awk 'NR==FNR{if (NF) { if (!beg) beg=NR; end=NR } next} FNR>=beg && FNR<=end' file file
line1

line2

The above treats lines of only blank chars as empty. If instead you only want lines with no chars at all to be considered empty then just change NF to /./.

0

Not written the code, but there has to be an efficient algorithm for any size file along these lines.

(a) Read and ignore empty lines until the first non-empty.

(b) Read and print non-empty lines until the next empty.

(c) Count (n) the empty lines until the next non-empty.

(d) If you hit non-empty, print n newlines and you are back in state (b).

(e) If you hit EOF, you are done -- discarded n empty lines before EOF.

-1

command

sed -n '/[a-zA-Z]/,/[a-zA-Z]/p' file| awk 'OFS=":"{$2=$1;$1=NR;print }'

output

1 line1
2 
3 line2
  • 5
    There's no need to use sed here as awk could easily do the same test. On the other hand, the line numbers are only for illustration in the question, which means that you may not need awk at all. In any case, it's almost never a reason to use both sed and awk in the same pipeline. – Kusalananda Nov 17 at 9:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.