1

This question already has an answer here:

I have the exact same question as this one here. Only difference is I have 3 columns only and I am trying to move the 1st column to the end.

The original file looks like this:

col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36

I didn't make the mistake that other questioner did (i.e. not put comma after F or after OFS=). So, my code is

awk -F, '{print $2,$3,$1}' OFS=, old.csv > new.csv

But I am getting the 3rd column (which used to be the first) in a new line:

col1,col2,col3
2015-01-04,23
,2
2015-01-20,36
,196

Why is awk sending the 3rd column data to a new line? I am using awk on Linux Bash Shell (Ubuntu) on Windows, downloaded from here.

marked as duplicate by Rui F Ribeiro, Sparhawk, G-Man, msp9011, Jeff Schaller Aug 8 '18 at 8:35

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 1
    Please paste a sample from old.csv – Sparhawk Aug 7 '18 at 23:44
  • Your output example is showing the columns in the order of 1,2,3 whereas your code specifies printing the columns in the order of 2,3,1. Using that exact same code you have, awk -F , '{print $2,$3,$1}' OFS=, old.csv, I get the columns in the order of 2,3,1 with none of it appearing on a new line. Is there something that you're actually doing differently? – Nasir Riley Aug 8 '18 at 0:04
  • 4
    It might be helpful to include the output of cat -A old.csv so we can see any non-printing characters (including DOS-style line endings for example) – steeldriver Aug 8 '18 at 0:29
  • @steeldriver. Thank you for telling me to check using that command. I found ^M after each line. I removed them using sed and everything is working fine now. – ahmed_m Aug 8 '18 at 3:04
2

It seems like your input file has some additional data in it, such as, DOS style newlines (\r\n), whereas typically on Unix systems, the files only have \n.

For example:

$ cat old.csv
col1,col2,col3
2,2015-01-04,23
196,2015-01-20,36

We can use hexdump to see the actual ASCII of this file:

$ hexdump -C old.csv
00000000  63 6f 6c 31 2c 63 6f 6c  32 2c 63 6f 6c 33 0a 32  |col1,col2,col3.2|
00000010  2c 32 30 31 35 2d 30 31  2d 30 34 2c 32 33 0a 31  |,2015-01-04,23.1|
00000020  39 36 2c 32 30 31 35 2d  30 31 2d 32 30 2c 33 36  |96,2015-01-20,36|
00000030  0a                                                |.|
00000031

Notice the 0a in the HEX output, this is a newline (\n). If I use basically your awk with this file it works as expected:

$ awk -F, '{print $2,$3,$1}' OFS=, old.csv
col2,col3,col1
2015-01-04,23,2
2015-01-20,36,196

If we convert the old.csv file to one that's typically form a Windows/DOS system using a CLI tool unix2dos the modified file, old_dos.csv looks like this:

$ hexdump -C old_dos.csv
00000000  63 6f 6c 31 2c 63 6f 6c  32 2c 63 6f 6c 33 0d 0a  |col1,col2,col3..|
00000010  32 2c 32 30 31 35 2d 30  31 2d 30 34 2c 32 33 0d  |2,2015-01-04,23.|
00000020  0a 31 39 36 2c 32 30 31  35 2d 30 31 2d 32 30 2c  |.196,2015-01-20,|
00000030  33 36 0d 0a                                       |36..|
00000034

Now we see 0d & 0a which is a \r\n. Using awk on this file acts oddly:

$ awk -F, '{print $2,$3,$1}' OFS=, old_dos.csv
,col1col3
,215-01-04,23
,196-01-20,36
  • You are correct. I found ^M at end of each line. Removed those and it's working fine now. – ahmed_m Aug 8 '18 at 3:05

Not the answer you're looking for? Browse other questions tagged or ask your own question.