0

Trying to extract a column (UPC Number) from csv file and adding white spaces at the end in case the length of the file is less than 14. However the output have an ^M at the end which I do not know how to remove.

This command is working for other columns except for this one.

awk -F, '{print $8}' $FILE | awk '{printf("%-14s\n", $0) }' > $TEMP/upc

vi output:

861601000109^M

736040531685^M

cat output:

 61601000109

 36040531685
1

This file was probably opened and saved in a windows text editor.

The end of line character in UNIX based systems are different from that of windows.

Text files created on DOS/Windows machines have different line endings than files created on Unix/Linux. DOS uses carriage return and line feed ("\r\n") as a line ending, which Unix uses just line feed ("\n"). You need to be careful about transferring files between Windows machines and Unix machines to make sure the line endings are translated properly.

Source

When you cat the file, the EOL character will not be displayed, however, when you read the file using vi, it will be.

The easiest way to remove ^M from the file, is probably to use the stream editor sed to remove the ^M characters. Type this command: % sed -e "s/^M//" filename > newfilename

To enter ^M, type CTRL-V, then CTRL-M. That is, hold down the CTRL key then press V and M in succession.

  • 2
    You can also use the nearly ubiquitous tool dos2unix to convert CRLF line-endings to LF line-endings. (older) Mac-sourced files have a similar issue, as they had used CR as their line-endings. – DopeGhoti Dec 20 '16 at 22:08
  • 1
    ^M can also be written \r. And if you don't have the dos2unix utility, vim can be of some help: vim -c 'set fileformat=unix' -c 'x' file – xhienne Dec 20 '16 at 22:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.