Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Using version control systems I get annoyed at the noise when the diff says No newline at end of file.

So I was wondering: How to add a newline at the end of a file to get rid of those messages?

share|improve this question
    
see also so/q/10082204/155090 – RubyTuesdayDONO Feb 9 '14 at 5:11

14 Answers 14

up vote 98 down vote accepted

Here you go:

sed -i -e '$a\' file

And alternatively for OS X sed:

sed -i '' -e '$a\' file

This adds \n at the end of the file only if it doesn’t already end with a newline. So if you run it twice, it will not add another newline:

$ cd "$(mktemp -d)"
$ printf foo > test.txt
$ sed -e '$a\' test.txt > test-with-eol.txt
$ diff test*
1c1
< foo
\ No newline at end of file
---
> foo
$ echo $?
1
$ sed -e '$a\' test-with-eol.txt > test-still-with-one-eol.txt
$ diff test-with-eol.txt test-still-with-one-eol.txt
$ echo $?
0
share|improve this answer
1  
While this does work, please note that the documentation at your link is misleading. It says '$' denotes the end of file, but in this case $ is matching the end of line (for every line), and is taking advantage of the fact that sed will add the newline if its not there. Actually, removing the . from the regex works fine, too. – jwd Feb 17 '12 at 17:28
2  
A better version, which seems like what the original author was trying for, would be sed -i -e '$a\' file, which really does match only the last line, and is possibly more efficient. – jwd Feb 17 '12 at 17:30
    
@jwd: From man sed: $ Match the last line. But maybe it works only by accident. Your solution also works. – l0b0 Feb 20 '12 at 11:54
14  
There are two different meanings of $. Inside a regex, such as with the form /<regex>/, it has the usual "match end of line" meaning. Otherwise, used as an address, sed gives it the special "last line in file" meaning. The code works because sed by default appends a newline to its output if it is not already there. The code "$a\" just says "match the last line of the file, and add nothing to it." But implicitly, sed adds the newline to every line it processes (such as this $ line) if it is not already there. – jwd Feb 22 '12 at 19:07
3  
For those of you that are trying this on a Mac, the -i parameter needs to be specified as -i ''. – M. Scott Ford Oct 17 '13 at 17:41

Have a look:

$ echo -n foo > foo 
$ cat foo
foo$
$ echo "" >> foo
$ cat foo
foo

so echo "" >> noeol-file should do the trick. (Or did you mean to ask for identifying these files and fixing them?)

edit removed the "" from echo "" >> foo (see @yuyichao's comment) edit2 added the "" again (but see @Keith Thompson's comment)

share|improve this answer
2  
the "" is not necessary (at least for bash) and tail -1 | wc -l can be used to find out the file without a new line at the end – yuyichao Feb 17 '12 at 14:42
2  
@yuyichao: The "" isn't necessary for bash, but I've seen echo implementations that print nothing when invoked without arguments (though none of the ones I can find now do this). echo "" >> noeol-file is probably slightly more robust. printf "\n" >> noeol-file is even more so. – Keith Thompson Feb 17 '12 at 17:17
    
@KeithThompson, csh's echo is the one known to output nothing when not passed any argument. But then if we're going to support non-Bourne-like shells, we should make it echo '' instead of echo "" as echo "" would ouput ""<newline> with rc or es for instance. – Stéphane Chazelas Feb 19 at 11:49
    
@StéphaneChazelas: And tcsh, unlike csh, prints a newline when invoked with no arguments -- regardless of the setting of $echo_style. – Keith Thompson Feb 19 at 18:55

Another solution using ed. This solution only affect the last line and only if \n is missing:

ed -s file <<< w

It essentially works opening the file for editing through a script, the script is the single w command, that write the file back to disk. It is based on this sentence found in ed(1) man page:

LIMITATIONS
       (...)

       If  a  text (non-binary) file is not terminated by a newline character,
       then ed appends one on reading/writing it.  In the  case  of  a  binary
       file, ed does not append a newline on reading/writing.
share|improve this answer
    
This does not add a newline for me. – Olhovsky Apr 12 '13 at 1:46
2  
Works for me; it even prints "Newline appended" (ed-1.10-1 on Arch Linux). – Stefan Majewsky Mar 10 '15 at 10:00

You're better off correcting the editor of the user who last edited the file. If you are the last person to have edited the file - what editor are you using, I'm guessing textmate ..?

share|improve this answer
    
Vim is the editor in question. But in general, you are right, I should not only fix the symptons ;) – k0pernikus Feb 17 '12 at 13:46
3  
for vim, you have to go out of your way and do the binary-file-on-save dance to get vim to not add a new line at the end of the file - just don't do that dance. OR, to simply correct existing files open them in vim and save the file and vim will 'fix' the missing newline for you (can be easily scripted for multiple files) – AD7six Feb 17 '12 at 13:50
    
My emacs do not add a newline at end of file. – enzotib Feb 20 '12 at 17:45
    
Thanks for the comment @AD7six, I keep getting phantom reports from diffs when I commit things, about how the original file does not have a newline at the end. No matter how I edit a file with vim i can't get it to not put a newline there. So it's just vim doing it. – Steven Lu Jun 21 '13 at 19:39
    
@enzotib: I have (setq require-final-newline 'ask) in my .emacs – Keith Thompson Feb 19 at 18:59

Provided there are no nulls in input:

paste - <>infile >&0

...would suffice to always only append a newline to the tail end of an infile if it didn't have one already. And it need only read the input file through the one time to get it right.

share|improve this answer

Although it doesn't directly answer the question, here is a related script I wrote to detect files which do not end in newline. It is very fast.

find . -type f | # sort |        # sort file names if you like
/usr/bin/perl -lne '
   open FH, "<", $_ or do { print " error: $_"; next };
   $pos = sysseek FH, 0, 2;                     # seek to EOF
   if (!defined $pos)     { print " error: $_"; next }
   if ($pos == 0)         { print " empty: $_"; next }
   $pos = sysseek FH, -1, 1;                    # seek to last char
   if (!defined $pos)     { print " error: $_"; next }
   $cnt = sysread FH, $c, 1;
   if (!$cnt)             { print " error: $_"; next }
   if ($c eq "\n")        { print "   EOL: $_"; next }
   else                   { print "no EOL: $_"; next }
'

The perl script reads a list of (optionally sorted) file names from stdin and for every file it reads the last character to determine if the file ends in a newline or not. It is very fast because it avoids reading the entire contents of each file. It outputs one line for each file it reads, prefixed with "error:" if some kind of error occurs, "empty:" if the file is empty (doesn't end with newline!), "EOL:" ("end of line") if the file ends with newline and "no EOL:" if the file doesn't end with newline.

Note: the script doesn't handle file names which contain newlines. If you're running on linux, you could handle all possible file names by adding -print0 to find, -z to sort, and -0 to perl, like this:

find . -type f -print0 | sort -z |
/usr/bin/perl -ln0e '
   open FH, "<", $_ or do { print " error: $_"; next };
   $pos = sysseek FH, 0, 2;                     # seek to EOF
   if (!defined $pos)     { print " error: $_"; next }
   if ($pos == 0)         { print " empty: $_"; next }
   $pos = sysseek FH, -1, 1;                    # seek to last char
   if (!defined $pos)     { print " error: $_"; next }
   $cnt = sysread FH, $c, 1;
   if (!$cnt)             { print " error: $_"; next }
   if ($c eq "\n")        { print "   EOL: $_"; next }
   else                   { print "no EOL: $_"; next }
'

Of course, you'd still have to come up with a way of encoding the file names with newlines in the output (left as an exercise for the reader).

The output could be filtered, if desired, to append a newline to those files which don't have one, most simply with

 echo >> $filename

Lack of a final newline can cause bugs in scripts since some versions of shell and other utilities will not properly handle a missing final newline when reading such a file.

In my experience, the lack of a final newline is caused by using various Windows utilities to edit files. I have never seen vim cause a missing final newline when editing a file, although it will report on such files.

Finally, there are much shorter (but slower) scripts which can loop over their file name inputs to print those files which do not end in newline, such as:

/usr/bin/perl -ne 'print "$ARGV\n" if /.\z/' -- FILE1 FILE2 ...
share|improve this answer

To recursively sanitize a project I use this oneliner:

for f in $(find . -type fls); do tail -n1 $f | read -r _ || echo >> $f; done
share|improve this answer

If you just want to quickly add a newline when processing some pipeline, use this:

outputting_program | { cat ; echo ; }

it's also POSIX compliant.

Then, of course, you can redirect it to a file.

share|improve this answer
    
The fact that I can use this in a pipeline is helpful. This allows me to count the number of rows in a CSV file, excluding the header. And it helps get an accurate line count on windows files that don't end with a newline or carriage return. cat file.csv | tr "\r" "\n" | { cat; echo; } | sed "/^[[:space:]]*$/d" | tail -n +2 | wc -l – Kyle Tolle Dec 22 '15 at 16:54

To apply the accepted answer to all files in the current directory (plus subdirectories):

$ find . -type f -exec sed -i -e '$a\' {} \;

This works on Linux (Ubuntu). On OS X you probably have to use -i '' (untested).

share|improve this answer
2  
Note that find . lists all files, including files in .git. To exclude: find . -type f -not -path './.git/*' -exec sed -i -e '$a\' {} \; – friederbluemle Jul 14 '15 at 7:59
    
Wish I would have read this comment/thought about it before I ran it. Oh well. – kstev Nov 9 '15 at 4:02

A simple, portable, POSIX-compliant way to add an absent, final newline to a would be text file:

[ -n "$(tail -c1 file)" ] && echo >> file

This approach does not need to read the entire file; it can simply seek to EOF and work from there.

This approach also does not need to create temp files behind your back (e.g. sed -i), so hardlinks aren't affected.

echo appends a newline to the file only when the result of the command substitution is a non-empty string. Note that this can only happen if the file is not empty and the last byte is not a newline.

If the last byte of the file is a newline, tail returns it, then command substitution strips it; the result is an empty string. The -n test fails and echo does not run.

If the file is empty, the result of the command substitution is also an empty string, and again echo does not run. This is desirable, because an empty file is not an invalid text file, nor is it equivalent to a non-empty text file with an empty line.

share|improve this answer
    
Note that it doesn't work with yash if the last character in the file is a multi-byte character (in UTF-8 locales for instance), or if the locale is C and the last byte in the file has the 8th bit set. With other shells (except zsh), it would not add a newline if the file ended in a NUL byte (but then again, that would mean the input would be non-text even after a newline is added). – Stéphane Chazelas Feb 19 at 11:39

Add newline regardless:

echo >> filename

Here is a way to check if a newline exists at the end before adding one, by using Python:

f=filename; python -c "import sys; sys.exit(open(\"$f\").read().endswith('\n'))" && echo >> $f
share|improve this answer
    
I wouldn't use the python version in any sort of loop because of the slow python startup time. Of course you could do the loop in python if you wanted. – Kevin Cox Nov 9 '13 at 14:03
    
The startup time for Python is 0.03 seconds here. Do you really consider that to be problematic? – Alexander Nov 10 '13 at 11:48
    
That's quite a bit faster than mine, but it adds up quick. I remember using python to do some math in a script once and it really bogged it down. Maybe they have improved it since then and I just have haunting memories. – Kevin Cox Nov 10 '13 at 15:36
    
Startup time does not add up quick, it's a one time cost. And yes, python is faster now than before and if needed you can compile it to native with Nuitka or Shedskin. There is also the "psyco" module for Python 2. In addition to this, there are libraries like numpy and scipy, which are reasonably fast for maths. There are also OpenCL bindings if you wish to use the GPU. So yes, I think your experience sounds outdated and/or misguided. – Alexander Nov 11 '13 at 12:16
    
Startup time does matter if you call python in a loop, that is why I said consider doing the loop in python. Then you only incur the startup cost once. For me, half the cost the startup is more than half of the time of the whole snipit, I would consider that substantial overhead. (Again, irrelevant if only doing a small number of files) – Kevin Cox Nov 11 '13 at 16:35

This works in AIX ksh:

lastchar=`tail -c 1 *filename*`
if [ `echo "$lastchar" | wc -c` -gt "1" ]
then
    echo "/n" >> *filename*
fi

In my case, if the file is missing the newline, the wc command returns a value of 2 and we write a newline.

share|improve this answer
    
Feedback will come in forms of up- or downvotes, or you will be asked in comments to outline your answers/questions more, no point in asking for it in the answer's body. Keep it to the point, welcome to stackexchange! – k0pernikus Feb 12 '15 at 2:45

The vi/vim/ex editors automatically add <EOL> at EOF unless file already has it.

So try either:

vi -ecwq foo.txt

which is equivalent to:

ex -cwq foo.txt

Testing:

$ printf foo > foo.txt && wc foo.txt
0 1 3 foo.txt
$ ex -scwq foo.txt && wc foo.txt
1 1 4 foo.txt

To correct multiple files, check: How to fix 'No newline at end of file' for lots of files? at SO

Why this is so important? To keep our files POSIX compatible.

share|improve this answer

A lot of great suggestions here but one idea would be to remove a newline and add one so you know you aren't continually adding them:

Take file "foo":

Remove a new line if there is one:

  truncate -s $(($(stat -c '%s' foo)-1)) foo

Then add one:

  sed -i -e '$a\' foo

Hence foo will always contain at least one new line..

or tail the file and look for new line, if it doesn't contain add one ..

  grep -q "[^0-9a-z-]" <<< $(tail -1 ./foo) && echo " " >> ./foo
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.