Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I need to periodically run a command that ensures that some text files are kept in Linux mode. Unfortunately dos2unix always modifies the file, which would mess file's and folder's timestamps and cause unnecessary writes.

The script I write is in Bash, so I'd prefer answers based on Bash.

share|improve this question

If the goal is just to avoid affecting the timestamp, dos2unix has a -k or --keepdate option which will keep the timestamp the same. It will still have to do a write to make the temporary file and rename it, but your timestamps will not be affected.

If any modification of the file is unacceptable, you can use the following solution from this answer.

find . -not -type d -exec file "{}" ";" | grep CRLF
share|improve this answer
    
Do you mean you literally write CRLF as 4 characters C, R, L and F? – bodacydo Dec 3 '15 at 3:19
2  
Do you also mean that grep can take CR and LF just like that? – bodacydo Dec 3 '15 at 3:19
    
@bodacydo It's explained in the answer he links to, and now also in Scott's edit of BertS' answer here unix.stackexchange.com/a/79708/59699 . – dave_thompson_085 Dec 3 '15 at 5:14
    
@dave_thompson_085 I dont see explanation. It only mentions CRLF but doesnt explain what it is. – bodacydo Dec 3 '15 at 6:05
    
@bodacydo stackoverflow.com/questions/73833/… says that find ... -exec file ... | grep CRLF for a file with DOS line endings (i.e. bytes 0D 0A) "will get you something like: ./1/dos1.txt: ASCII text, with CRLF line terminators As you can see this contains the actual string CRLF and therefore is matched by grep looking for the simple string CRLF. – dave_thompson_085 Dec 4 '15 at 8:40

You can use dos2unix as a filter and compare its output to the original file:

dos2unix < myfile.txt | cmp -s - myfile.txt
share|improve this answer
1  
Very smart and useful, because it tests the complete file and not only the first or a few line. – halloleo May 19 '15 at 0:04
1  
Maybe you could replace test by myfile.txt twice in your example to avoid confusion with /usr/bin/test. – Peterino May 30 at 7:51
    
@Peterino: good idea, thanks! – Samuel Edwin Ward Jun 3 at 20:46

You could try to grep for CRLF code, octal:

grep -U $'\015' myfile.txt

or hex:

grep -U $'\x0D' myfile.txt
share|improve this answer
    
Of course, the assumption is that this is a text file. – mdpc Jun 17 '13 at 17:24
    
I like this grep usage because it allows me to easily list all such files in the directory with grep -lU $'\x0D' * and pass the output to xargs. – Melebius Apr 30 '15 at 6:19

First method (grep):

Count the lines that contain a carriage return:

[[ $(grep -c $'\r' myfile.txt) -gt 0 ]] && echo dos

Count the lines that end with a carriage return:

[[ $(grep -c $'\r$' myfile.txt) -gt 0 ]] && echo dos

These will typically be equivalent; a carriage return in the interior of a line (i.e., not at the end) is rare.

More efficient:

grep -q $'\r' myfile.txt && echo dos

This is more efficient

  1. because it doesn't need to convert the count to an ASCII string, and then convert that string back to an integer, and compare it to zero, and
  2. because grep -c needs to read the entire file, to count all the occurrences of the pattern, while grep -q can exit upon seeing the first occurrence of the pattern.

Notes:

  • Throughout the above, you may need to add the -U option (i.e., use -cU or -qU), because GNU grep guesses whether the file is a text file.  If it thinks the file is text, it ignores carriage returns at the ends of lines, in an attempt to make $ in regular expressions work "correctly" — even if the regular expression is \r$!  Specifying -U (or --binary) overrules this guesswork, causing grep to treat the file(s) as binary and pass the data to the matching mechanism verbatim, with CR-endings intact.
  • Do not do grep … $'\r\n' myfile.txt, because grep treats \n as a pattern delimiter.  Just as grep -E 'foo|' looks for lines containing foo or a null string, grep $'\r\n' looks for lines containing \r or a null string, and every line matches a null string.

Second method (file):

[[ $(file myfile.txt) =~ CRLF ]] && echo dos

because file reports something like:

myfile.txt: UTF-8 Unicode text, with CRLF line terminators

Safer variant:

[[ $(file -b - < myfile.txt) =~ CRLF ]] && echo dos

where

  • file -b outputs only the file type, and not the file name.  Without this, a file whose name included the characters CRLF would trigger a false positive.
  • file - < filename works even if filename begins with -See Bash script: check if a file is a text file.

Beware that checking the output from file might not work in a non-English locale.

share|improve this answer
    
You can replace "$(echo -e '\r')" with the much simpler $'\r', although personally I'd use $'\r\n' to reduce the number of false positives. – rici Jun 17 '13 at 17:03
    
@rici grep $'\r\n' seems to match all files on my system... – depquid Jun 17 '13 at 17:09
    
@rici: good catch. I edited my answer according to your suggestion. — depquid: Maybe you are on Windows? :-) rici's tip works here. – BertS Jun 17 '13 at 17:11
    
@depquid (and BertS): Actually, I think the correct invocation is grep -U $'\r$', to prevent grep trying to second-guess line-endings. – rici Jun 17 '13 at 17:18
    
Also, you can use -q to just set the return code if a match is found, instead of -c which requires an additional check. Personally I like your second solution, although it's highly dependent on the whims of file and might not work in a non-English locale. – rici Jun 17 '13 at 17:22

a bash function for you:

# return 0 (true) if first line ends in CR
isDosFile() {
    [[ $(head -1 "$1") == *$'\r' ]]  
}

Then you can do stuff like

streamFile () {
    if isDosFile /tmp/foo.txt; then
        sed 's/\r$//' "$1"
    else
        cat "$1"
    fi
}

streamFile /tmp/foo.txt | process_lines_without_CR
share|improve this answer
3  
You don't have to use isDosFile() in your example: streamFile() { sed 's/\r$//' "$1" ; }. – Evan Teitelman Jun 17 '13 at 18:59
1  
I think this is the most elegant solution; it doesn't read whole file, just the first line. – Adam Ryczkowski Jun 22 '13 at 20:50

Since version 7.1 dos2unix has an -i, --info option to get information about line breaks. You can use dos2unix itself to test which files need conversion.

Example:

dos2unix -ic *.txt | xargs dos2unix
share|improve this answer
    
Here is link to the changelog itself waterlan.home.xs4all.nl/dos2unix/NEWS.txt – Adam Ryczkowski Sep 23 '15 at 14:16

If a file has DOS/Windows-style CR-LF line endings, then if you look at it using a Unix-based tool you'll see CR ('\r') characters at the end of each line.

This command:

grep -l '^M$' filename

will print filename if the file contains one or more lines with Windows-style line endings, and will print nothing if it doesn't. Except that the ^M has to be a literal carriage return character, typically entered in the terminal by typing Ctrl+V followed by Enter (or Ctrl+V and then Ctrl+M). The bash shell lets you write a literal carriage return as $'\r' (documented here), so you can write:

grep -l $'\r$' filename

Other shells may provide a similar feature.

You can use another tool instead:

awk '/\r$/ { exit(1) }' filename

This will exit with a status of 1 (setting $? to 1) if the file contains any Windows-style line endings, and with a status of 0 if it doesn't, making it useful in a shell if statement (note the lack of [ brackets ]):

if awk '/\r$/ { exit(1) }' filename ; then
    echo filename has Unix-style line endings
else
    echo filename has at least one Windows-style line ending
fi

A file can contain a mixture of Unix-style and Windows-style line endings. I'm assuming here that you want to detect files that have any Windows-style line endings.

share|improve this answer
1  
You can encode a carriage return on the command line in bash (and some other shells) by typing $'\r', as mentioned in other answers to this question. – Scott Dec 2 '15 at 6:41
    
@Scott: Thanks, updated. – Keith Thompson Dec 2 '15 at 19:04

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.