Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Using version control systems I get annoyed at the noise when the diff says No newline at end of file.

So I was wondering: How to add a newline at the end of a file to get rid off those messages?

share|improve this question
    
see also so/q/10082204/155090 –  RubyTuesdayDONO Feb 9 at 5:11

6 Answers 6

up vote 53 down vote accepted

Here you go:

sed -i -e '$a\' file

And alternatively for OS X sed:

sed -i '' -e '$a\' file

This adds \n at the end of the file only if it doesn’t already end with a newline. So if you run it twice, it will not add another newline:

$ cd "$(mktemp -d)"
$ printf foo > test.txt
$ sed -e '$a\' test.txt > test-with-eol.txt
$ diff test*
1c1
< foo
\ No newline at end of file
---
> foo
$ echo $?
1
$ sed -e '$a\' test-with-eol.txt > test-still-with-one-eol.txt
$ diff test-with-eol.txt test-still-with-one-eol.txt
$ echo $?
0
share|improve this answer
1  
While this does work, please note that the documentation at your link is misleading. It says '$' denotes the end of file, but in this case $ is matching the end of line (for every line), and is taking advantage of the fact that sed will add the newline if its not there. Actually, removing the . from the regex works fine, too. –  jwd Feb 17 '12 at 17:28
1  
A better version, which seems like what the original author was trying for, would be sed -i -e '$a\' file, which really does match only the last line, and is possibly more efficient. –  jwd Feb 17 '12 at 17:30
    
@jwd: From man sed: $ Match the last line. But maybe it works only by accident. Your solution also works. –  l0b0 Feb 20 '12 at 11:54
9  
There are two different meanings of $. Inside a regex, such as with the form /<regex>/, it has the usual "match end of line" meaning. Otherwise, used as an address, sed gives it the special "last line in file" meaning. The code works because sed by default appends a newline to its output if it is not already there. The code "$a\" just says "match the last line of the file, and add nothing to it." But implicitly, sed adds the newline to every line it processes (such as this $ line) if it is not already there. –  jwd Feb 22 '12 at 19:07
3  
For those of you that are trying this on a Mac, the -i parameter needs to be specified as -i ''. –  M. Scott Ford Oct 17 '13 at 17:41

Have a look:

$ echo -n foo > foo 
$ cat foo
foo$
$ echo "" >> foo
$ cat foo
foo

so echo "" >> noeol-file should do the trick. (Or did you mean to ask for identifying these files and fixing them?)

edit removed the "" from echo "" >> foo (see @yuyichao's comment) edit2 added the "" again (but see @Keith Thompson's comment)

share|improve this answer
1  
the "" is not necessary (at least for bash) and tail -1 | wc -l can be used to find out the file without a new line at the end –  yuyichao Feb 17 '12 at 14:42
1  
@yuyichao: The "" isn't necessary for bash, but I've seen echo implementations that print nothing when invoked without arguments (though none of the ones I can find now do this). echo "" >> noeol-file is probably slightly more robust. printf "\n" >> noeol-file is even more so. –  Keith Thompson Feb 17 '12 at 17:17

You're better off correcting the editor of the user who last edited the file. If you are the last person to have edited the file - what editor are you using, I'm guessing textmate ..?

share|improve this answer
    
Vim is the editor in question. But in general, you are right, I should not only fix the symptons ;) –  k0pernikus Feb 17 '12 at 13:46
3  
for vim, you have to go out of your way and do the binary-file-on-save dance to get vim to not add a new line at the end of the file - just don't do that dance. OR, to simply correct existing files open them in vim and save the file and vim will 'fix' the missing newline for you (can be easily scripted for multiple files) –  AD7six Feb 17 '12 at 13:50
    
My emacs do not add a newline at end of file. –  enzotib Feb 20 '12 at 17:45
    
Thanks for the comment @AD7six, I keep getting phantom reports from diffs when I commit things, about how the original file does not have a newline at the end. No matter how I edit a file with vim i can't get it to not put a newline there. So it's just vim doing it. –  Steven Lu Jun 21 '13 at 19:39

Another solution using ed. This solution only affect the last line and only if \n is missing:

ed -s file <<< w

It essentially works opening the file for editing through a script, the script is the single w command, that write the file back to disk. It is based on this sentence found in ed(1) man page:

LIMITATIONS
       (...)

       If  a  text (non-binary) file is not terminated by a newline character,
       then ed appends one on reading/writing it.  In the  case  of  a  binary
       file, ed does not append a newline on reading/writing.
share|improve this answer
    
This does not add a newline for me. –  Olhovsky Apr 12 '13 at 1:46

Although it doesn't directly answer the question, here is a related script I wrote to detect files which do not end in newline. It is very fast.

find . -type f | # sort |        # sort file names if you like
/usr/bin/perl -lne '
   open FH, "<", $_ or do { print " error: $_"; next };
   $pos = sysseek FH, 0, 2;                     # seek to EOF
   if (!defined $pos)     { print " error: $_"; next }
   if ($pos == 0)         { print " empty: $_"; next }
   $pos = sysseek FH, -1, 1;                    # seek to last char
   if (!defined $pos)     { print " error: $_"; next }
   $cnt = sysread FH, $c, 1;
   if (!$cnt)             { print " error: $_"; next }
   if ($c eq "\n")        { print "   EOL: $_"; next }
   else                   { print "no EOL: $_"; next }
'

The perl script reads a list of (optionally sorted) file names from stdin and for every file it reads the last character to determine if the file ends in a newline or not. It is very fast because it avoids reading the entire contents of each file. It outputs one line for each file it reads, prefixed with "error:" if some kind of error occurs, "empty:" if the file is empty (doesn't end with newline!), "EOL:" ("end of line") if the file ends with newline and "no EOL:" if the file doesn't end with newline.

Note: the script doesn't handle file names which contain newlines. If you're running on linux, you could handle all possible file names by adding -print0 to find, -z to sort, and -0 to perl, like this:

find . -type f -print0 | sort -z |
/usr/bin/perl -ln0e '
   open FH, "<", $_ or do { print " error: $_"; next };
   $pos = sysseek FH, 0, 2;                     # seek to EOF
   if (!defined $pos)     { print " error: $_"; next }
   if ($pos == 0)         { print " empty: $_"; next }
   $pos = sysseek FH, -1, 1;                    # seek to last char
   if (!defined $pos)     { print " error: $_"; next }
   $cnt = sysread FH, $c, 1;
   if (!$cnt)             { print " error: $_"; next }
   if ($c eq "\n")        { print "   EOL: $_"; next }
   else                   { print "no EOL: $_"; next }
'

Of course, you'd still have to come up with a way of encoding the file names with newlines in the output (left as an exercise for the reader).

The output could be filtered, if desired, to append a newline to those files which don't have one, most simply with

 echo >> $filename

Lack of a final newline can cause bugs in scripts since some versions of shell and other utilities will not properly handle a missing final newline when reading such a file.

In my experience, the lack of a final newline is caused by using various Windows utilities to edit files. I have never seen vim cause a missing final newline when editing a file, although it will report on such files.

Finally, there are much shorter (but slower) scripts which can loop over their file name inputs to print those files which do not end in newline, such as:

/usr/bin/perl -ne 'print "$ARGV\n" if /.\z/' -- FILE1 FILE2 ...
share|improve this answer

Add newline regardless:

echo -n '\n' >> filename

Here is a way to check if a newline exists at the end before adding one, by using Python:

f=filename; python -c "import sys; sys.exit(open(\"$f\").read().endswith('\n'))" && echo -n '\n' >> $f
share|improve this answer
    
I wouldn't use the python version in any sort of loop because of the slow python startup time. Of course you could do the loop in python if you wanted. –  Kevin Cox Nov 9 '13 at 14:03
    
The startup time for Python is 0.03 seconds here. Do you really consider that to be problematic? –  Alexander Nov 10 '13 at 11:48
    
That's quite a bit faster than mine, but it adds up quick. I remember using python to do some math in a script once and it really bogged it down. Maybe they have improved it since then and I just have haunting memories. –  Kevin Cox Nov 10 '13 at 15:36
    
Startup time does not add up quick, it's a one time cost. And yes, python is faster now than before and if needed you can compile it to native with Nuitka or Shedskin. There is also the "psyco" module for Python 2. In addition to this, there are libraries like numpy and scipy, which are reasonably fast for maths. There are also OpenCL bindings if you wish to use the GPU. So yes, I think your experience sounds outdated and/or misguided. –  Alexander Nov 11 '13 at 12:16
    
Startup time does matter if you call python in a loop, that is why I said consider doing the loop in python. Then you only incur the startup cost once. For me, half the cost the startup is more than half of the time of the whole snipit, I would consider that substantial overhead. (Again, irrelevant if only doing a small number of files) –  Kevin Cox Nov 11 '13 at 16:35

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.