I have following lines with ^M (return carriage). I want to remove ^M+proceeding line. You can notice there is a new line after each ^M, which is not desired. I purpose is to remove control character and make one line CSV.

    '1.0';'';'';'';'Mon Jul 04 00:00:00 CEST 2016';'To cash Pls Acc 4142^M
To cash Pls Acc 4142';'To cash Pls Acc 4142^M

I tried following but didn't work. NOTE: ^M = CONTROL v + CONTRL m

cat file.csv | sed 's/[\^M\n]//g' > new_file.csv

above lines removed ^M but the line is still broken with new line.

expected output

 '1.0';'';'';'';'Mon Jul 04 00:00:00 CEST 2016';'To cash Pls Acc 4142To cash Pls Acc 4142';'To cash Pls Acc 4142 Money';'236159';'236159';'-2000.0';'';'2000.0'
  • I believe you should replace the end of the line as well. Try adding $ to the left side of the substitution
    – Dani_l
    Jan 2 '17 at 11:29
  • There are several tool available to replace Windows newlines with Unix newlines. Look for dos2unix for example.
    – Kusalananda
    Jan 2 '17 at 11:34
  • 1
    Please add your desired output for that sample input to your question.
    – Cyrus
    Jan 2 '17 at 11:48
  • @Cyrus added expected out put
    – amjad
    Jan 2 '17 at 12:41

This works for your case:

sed -i ':a;N;$!ba;s/\r\n//g' file.csv

From: Can sed replace new line characters?

Or this one:

sed -e :a -e '/\r$/N; s/\r\n//; ta' file.csv

From: http://www.catonmat.net/blog/sed-one-liners-explained-part-one/

Note that you don't need to use cat, and you can directly edit your file with the option -i if you want.

  • I am still getting output file ^M after running above. sed -i ':a;N;$!ba;s/\^M\n//g' file.csv > file_without_m.csv.
    – amjad
    Jan 2 '17 at 12:55
  • Try replacing \^M with \r Jan 2 '17 at 13:33
  • You're right, I edited my answer. But remember: use the -i option only if you want to directly modify file.csv. If you just want to redirect output to file_without_m.csv you don't need -i.
    – andreatsh
    Jan 2 '17 at 18:56

It's \r not ^M. The simplest way to do it is:

tr -d '\n-\r' < file.

-d means delete and \n-\r is the character range for truncation.

Note that this code will also remove occurrences of \ns not preceded by "^M"s.

  • About the downvoting. Without an explanation, my only suspicion is that you found my answer redundant. I think the simplest method is a good answer to add.
    – argle
    Jan 3 '17 at 9:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.