Trying to use
grep today, I ran into the familiar problem of the Byte Order Mark (BOM) in a Unicode file (UTF-8, in this case). Specifically, I was trying to find a file beginning with
XYZ with the pattern
grep '^XYZ', but of course
grep treated the BOM as three separate characters and did not match the first line of the file if the first line started with
XYZ. I even tried to update the regular expression to ignore spaces (
'^[[:space:]]*XYZ'), but to no avail.
Other questions have dealt with converting files or targeting the BOM specifically, but I want to know if POSIX tools have a general option to handle Unicode files correctly. If
grep handled the Unicode file correctly, it would consider the file contents to start after the BOM and match
XYZ on the first line just like any other line.