3

I test content of a set of data files whether they contain at least one of a set of characters which consists of printing and non-printing characters.

My last issue is detecting whether the file contains a line-feed. My GNU grep 3.0 states every input contains a LF... Why is that?

echo -n "test" | grep -UF -e $'\x0a'

Any ideas? I suspect some implicit EOL/EOF interference.

10

grep is line-oriented - if the input does not end with a newline, it still considers the text past the last newline (or start of file) as a line.

Line-oriented programs are a poor fit for directly processing binary files - they will often have pathological cases if a binary file has a particularly long "line".

Instead, consider something like a combination of tr and cmp:

$ echo -n foo | tr -d -c $'\n' | cmp /dev/null - 
# no output and exits with status 0

$ echo foo | tr -d -c $'\n' | cmp /dev/null -
cmp: EOF on /dev/null which is empty
# exits with status 1

This method also has the advantage of needing to read the input only up to the first newline character (plus buffering).

  • Just a side note: man tr mentions the way how to specify the non-printing characters - via octal value \NNN – Yuri Jul 26 '18 at 11:08
  • But buffering will actually mean that it needs to read something like 4096 lines before tr starts to output anything (see seq 5000 | strace -e write tr -cd '\n' > /dev/null) – Stéphane Chazelas Jul 26 '18 at 14:14
3

From the POSIX specification of grep:

Regular expression matching shall be based on text lines. Since a <newline> separates or terminates patterns [...], regular expressions cannot contain a <newline>. Similarly, since patterns are matched against individual lines (excluding the terminating <newline> characters) of the input, there is no way for a pattern to match a <newline> found in the input.

To check whether input contains a linefeed character, which is the same as a newline character on Unix, you may count the number of lines with wc -l:

lines=$( wc -l <inputfile )
if [ "$lines" -gt 0 ]; then
    echo 'Input had a linefeed character in it'
fi

Or, if you have a string:

lines=$( printf '%s' "$string" | wc -l )
# etc.
  • @StephenKitt Check the manual page. Among other things, "This option has no effect on platforms other than MS-DOS and MS-Windows" makes it pretty clear that this is not the case. – Vladimir Panteleev Jul 26 '18 at 9:47
  • Oh right sorry, it only strips CRs, right. – Stephen Kitt Jul 26 '18 at 9:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.