19

Everybody knows :-) that in Windows plain text files lines are terminated with CR+LF, and in Unix&Linux - with LF only. How can I quickly convert all my source code files from one format to another and back?

4 Answers 4

25

That depends: if the files are under version control, this could be a rather unpopular history-polluting decision. Git has the option to automagically convert line endings on check-out.

If you do not care and want to quickly convert, there are programs like fromdos/todos and dos2unix/unix2dos that do this for you. You can use find: find . -type f -name '*.php' -exec dos2unix '{}' +.

2
  • This will run dos2unix on all files, the ones that already have Unix line-breaks (LF), as well as the ones in need to convert the Windows line-breaks (CRLF) to LF. This should be fine, I think. But it's worth a comment. :-) Mar 6, 2023 at 13:11
  • Also, I prefer to start with find . -type f -name '*.php' -exec echo dos2unix '{}' \; in order to see what files will be affected before going through with the actual conversion. Of course, you might not want to follow my suggestion if the number of files affected is huge. Mar 6, 2023 at 13:24
13

There are several dedicated programs, including

  • dos2unix and unix2dos from the dos2unix tools
  • todos and fromdos from the tofrodos tools

Simply pick the tool for the appropriate direction and pass the names of the files to convert on the command line.


If you don't have either, but have Linux or Cygwin:

sed -i -e 's/\r\+$//' filename             # dos|unix -> unix
sed -i -e 's/\r*$/\r/' filename            # dos|unix -> dos

If you have perl:

perl -i -pe 's/\r+$//' filename            # dos|unix -> unix
perl -i -pe 's/\r*$/\r/' filename          # dos|unix -> dos

With only POSIX tools (including BusyBox), to go from unix to dos, you'll need to pass the CR character literally in the sed command.

cr=$(echo | tr '\n' '\r')
sed -e "s/$cr*\$/$cr/" <filename >filename.dos
mv filename.dos filename

In the other direction, you can simply delete all CRs:

tr -d '\r' <filename >filename.dos
mv filename.dos filename

You can use wildcards to convert many files in the same directory at once, e.g.

sed -i -e 's/\r\+$//' *.txt

To convert all files in the current directory and its subdirectories, if your shell is zsh, you can use **/, e.g.

sed -i -e 's/\r\+$//' **/*.txt

You can use **/ in bash ≥4, but you need to run shopt -s globstar first (you can put this line in your ~/.bashrc). You can use **/ in ksh93, but you need to run set -o globstar first (you can put this line in your ~/.kshrc.

If you can only use the tools that require a redirection, use a for loop.

for x in *.txt; do
  tr -d '\r' <"$x" >"$x.dos"
  mv -- "$x.dos" "$x"
done

If you don't have **/ or need more complex matching to select which files to convert, use the find command. Here's a Linux/Cygwin example which converts all files under the current directory and its subdirectories recursively, except for files called .o and under subdirectories called bin.

find -name 'bin' -type d -prune -o \
     \! -name '*.o' \
     -exec sed -i -e 's/\r\+$//' {} +

Here's a POSIX example. We tell find to start a shell that can perform the necessary redirection.

find -name 'bin' -type d -prune -o \
     \! -name '*.o' \
     -exec sh -c '
       tr -d '\r' <"$0" >"$0.dos"
       mv -- "$0.dos" "$0"
' {} \;

You can make the find method slightly faster, at the expense of more complex code, by using a loop in the shell command.

find -name 'bin' -type d -prune -o \
     \! -name '*.o' \
     -exec sh -c '
       for x; do
         tr -d '\r' <"$x" >"$x.dos"
         mv -- "$x.dos" "$x"
       done
' _ {} +
2

winscp will also auto-convert, if told so.

If your files reside all in the same directory:

DIRECTORY=/your/directory
unix2dos $DIRECTORY/*
1
  • WinSCP doesn't seem to autoconvert files in the current directory - it's more about automatically converting files as part of a transfer process. Even vanilla ftp did that. Dec 31, 2020 at 12:35
0

Changing from ASCII text, with CRLF line terminators to ASCII text

VIM - Vi IMproved 8.0 (2016 Sep 12, compiled Apr 10 2018 21:31:58)


All files in current directory

for i in *; do vi -c "set fileformat=unix | wq" "${i}"; done

Specific mask (all filenames in current directory starting with word "file")

for i in file*; do vi -c "set fileformat=unix | wq" "${i}"; done

Another mask (all filenames in current directory that contain the word "file")

for i in *file*; do vi -c "set fileformat=unix | wq" "${i}"; done

To convert files back to CRLF just change fileformat=unix to fileformat=dos:

for i in *file*; do vi -c "set fileformat=dos | wq" "${i}"; done

This is the quickest way if using vim to convert fileformat.


Alternatively sed can also be used to remove all ^M by pressing CTRL+V and then CTRL+M

for i in *; do sed -i s/^M// "$i"; done

or

for i in *; do sed -i s/^M// "${i}"; done

This will also change all CRLF files in current directory to ASCII text.

1
  • 2
    Your vi suggestions will only work when vi is aliased to vim (the set fileformat option is not part of standard vi) Dec 31, 2020 at 12:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .