24

How can I check if a UTF-8 text file has a BOM from command line?

file command shows me:

UTF-8 Unicode text

But, I don't know if it means there is no BOM in the file.

I'm using Ubuntu 12.04.

1
  • Note that there is no BOM in UTF-8: that's a feature of UTF-16. An UTF-8 file may start with the U+FEFF character, but in that case it's a zero-width space. Dec 2, 2014 at 14:16

2 Answers 2

41

file will tell you if there is a BOM. You can simply test it with:

printf '\ufeff...\n' | file -
/dev/stdin: UTF-8 Unicode (with BOM) text

Some shells such as ash or dash have a printf builtin that does not support \u, in which case you need to use printf from the GNU coreutils, e.g. /usr/bin/printf.


Note: according to the file changelog, this feature existed already in 2007. So, this should work on any current machine.

3
  • 1
    Thanks for answer. My file version is file-5.09 and the result was /dev/stdin: ASCII text. Is it depends on version of file?
    – ironsand
    Dec 1, 2014 at 3:55
  • Thank you for that addition! I am using POSIX printf and completely missed that, sorry. Cheers. Feb 20 at 11:57
  • 1
    @LinuxSecurityFreak POSIX does not specify the \u escape sequence (at least, not yet). It specifies \ddd with a 3-digit octal number, so that a portable version could be: printf '\357\273\277...\n' | file - (but it is rather difficult to remember).
    – vinc17
    Feb 20 at 19:55
8

If you execute stat fileName it should give you exact the three characters. When I opened the file in the editor, I was unable to see anything. So noticing that the file size was 3 gave me clarity that it has a BOM.

Also, the post here was helful in my case.

hexdump -n 3 -C 2.txt
00000000 ef bb bf
ef bb bf // YES

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.