Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Is there an issue with sed and new line character?
I have a file test.txt with the following contents

aaaaa  
bbbbb  
ccccc  
ddddd  

The following does not work:
sed -r -i 's/\n/,/g' test.txt

I know that I can use tr for this but my question is why it seems not possible with sed.

Update
If this is a side effect of processing the file line by line I would be interested in why this happens. I think grep removes new lines. Does sed do the same?

share|improve this question

4 Answers 4

up vote 8 down vote accepted
sed -i ':a;N;$!ba;s/\n/,/g' test.txt

From http://stackoverflow.com/questions/1251999/sed-how-can-i-replace-a-newline-n :

  1. create a label via :a
  2. append the current and next line to the pattern space via N
  3. if we are before the last line, branch to the created label $!ba ($! means not to do it on the last line (as there should be one final newline)).
  4. finally the substitution replaces every newline with a comma on the pattern space (which is the whole file).
share|improve this answer
    
This seems to indicate that the problem is that sed reads line by line.But I can't understand why is this an issue.It could just read the line and replace the new line character (or last character) with a , –  Jim Feb 12 at 20:27
1  
@jim It looks like it is not in the buffer to be matched, but I am not fluent with sed, maybe someone else can shed a light on that. I think you should extend your Q with that specific info, so people are more likely to read it, and hopefully answer. –  Anthon Feb 12 at 20:30

From Oracle's web site:

The sed utility works by sequentially reading a file, line by line, into memory. It then performs all actions specified for the line and places the line back in memory to dump to the terminal with the requested changes made. After all actions have taken place to this one line, it reads the next line of the file and repeats the process until it is finished with the file.

Basically this means that because sed is reading line by line the newline character is not matched.

The solution from http://stackoverflow.com/questions/1251999/sed-how-can-i-replace-a-newline-n is:

sed ':a;N;$!ba;s/\n/ /g'

An explanation into how that works is provided on that page.

share|improve this answer
1  
That will get you spaces between the lines, not commas –  Anthon Feb 12 at 20:33

Alternatively, you can use a slightly simpler syntax:

sed ':a;N;s/\n/,/g;ba'

...just changing sequence order.

share|improve this answer

sed always removes the trailing \newline just before populating pattern space, and then appends one before writing out the results of its script. A \newline can be had in pattern-space by various means - but never if it is not the result of an edit. This is important - \newlines in sed's pattern space always reflect a change, and never occur in the input stream. \newlines are the only delimiter a sedder can count on with unknown input.

If you want to replace all \newlines with commas and your file is not very large, then you can do:

sed 'H;1h;$!d;x;y/\n/,/'

That appends every input line to hold space - except the first, which instead overwrites hold space - following a \newline character. It then deletes every line not the $!last from output. On the last line Hold and pattern spaces are exchanged and all \newline characters are y///translated to commas.

For large files this sort of thing is bound to cause problems - sed's buffer on line-boundaries, that can be easily overflowed with actions of this sort.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.