Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am running the following 2 sed commands. The first one adds newline characters where I want them, the second also adds newline characters where I want them, BUT also adds an extra one at the end of the file where there wasn't one before.

sed -e 's|\<LIST_G_STATEMENT>|&\
|g' ${XMLDIR}/statement_tmp_1.xml > ${XMLDIR}/statement_tmp_2.xml

sed -e 's|\</LIST_G_STATEMENT>|&\
|g' ${XMLDIR}/statement_tmp_2.xml > ${XMLDIR}/statement_tmp_3.xml

Using od -c on all 3 of the files gives the following output.

statement_tmp_1.xml (no \n at end of file)

1314700    T   A   T   E   M   E   N   T   >   <   /   L   I   S   T   _
1314720    G   _   S   T   A   T   E   M   E   N   T   >   <   /   G   _
1314740    S   E   T   U   P   >   <   /   L   I   S   T   _   G   _   S
1314760    E   T   U   P   >   <   /   A   R   X   S   G   P   O   >
1314777

statement_tmp_2.xml (no \n at end of file)

1314700    S   T   A   T   E   M   E   N   T   >   <   /   L   I   S   T
1314720    _   G   _   S   T   A   T   E   M   E   N   T   >   <   /   G
1314740    _   S   E   T   U   P   >   <   /   L   I   S   T   _   G   _
1314760    S   E   T   U   P   >   <   /   A   R   X   S   G   P   O   >
1315000

statement_tmp_3.xml (\n at end of file - where did it come from?)

1314700    S   T   A   T   E   M   E   N   T   >   <   /   L   I   S   T
1314720    _   G   _   S   T   A   T   E   M   E   N   T   >  \n   <   /
1314740    G   _   S   E   T   U   P   >   <   /   L   I   S   T   _   G
1314760    _   S   E   T   U   P   >   <   /   A   R   X   S   G   P   O
1315000    >  \n
1315002

I am running AIX 5.3

Basically, I either want it to stop adding the extra \n, or find a way of removing it.

share|improve this question
    
Just a question: why are you using a literal newline in your substitution pattern when you could have used s|...|&\n| just as well? –  Joseph R. Nov 4 '13 at 10:59
    
@JosephR. \n in the right hand side is not portable. –  Stéphane Chazelas Nov 4 '13 at 11:03
    
@StephaneChazelas That's weird. Is it a CR vs CRLF thing? –  Joseph R. Nov 4 '13 at 11:05
2  
A file which doesn't end in a newline character is not a text file, so the behaviour with text utilities on them is unspecified. Use perl or other tool that can deal with binary data. –  Stéphane Chazelas Nov 4 '13 at 11:06
4  
@JosephR. No, \<LF> is the traditional and POSIX way to add a LF character. \n would typically substitute a n character in anything but GNU sed. –  Stéphane Chazelas Nov 4 '13 at 11:07

3 Answers 3

You should consider yourself lucky that AIX sed added that missing newline characters.

A non-empty file that doesn't end in a newline character is not a text file (at least as per the POSIX definition of a text file) as a text file is meant to contain lines and lines are a (not-too-long) sequence of characters terminated by a newline character, so the behaviour of text utilities like sed on it is unspecified and in practice varies from implementation to implementation.

Some sed implementation would have dismissed those spurious character after the last line.

AFAIK, xml files are meant to be text files, so that means sed just fixed it for you.

If you do need that file not to end in a newline character, then you could use perl or other tools that can cope with non-text data.

perl -pe 's|<LIST_G_STATEMENT>|$&\n|g'
share|improve this answer

Here's a way to remove the final newline from a file using dd:

printf "" | dd  of='/path/to/file' seek=<filesize_in_bytes - 1> bs=1 count=1

To test whether a file ends with a newline you could use:

tail -c 1 /path/to/file | tr -dc '\n' | wc -c

And to get the file size in bytes use:

wc -c < /path/to/file
share|improve this answer

According to this AIX manual IBM's tail does -reverse - which looks pretty cool. So long as your file is under 20KB the following should work:

tail -r <file | dd bs=1 skip=1 | tail -r >file.new
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.