18

How can I check if a utf-8 text file have a BOM from command line?

file command shows me UTF-8 Unicode text.

But I don't know it means there is no BOM in the file.

I'm using Ubuntu 12.04.

  • Note that there is no BOM in UTF-8: that's a feature of UTF-16. An UTF-8 file may start with the U+FEFF character, but in that case it's a zero-width space. – Gilles Dec 2 '14 at 14:16
29

file will tell you if there is a BOM. You can test:

$ /usr/bin/printf "\ufeff...\n" | file -
/dev/stdin: UTF-8 Unicode (with BOM) text

Note: according to the file changelog, this feature existed already in 2007. So, this should work on any current machine.

  • 1
    Thanks for answer. My file version is file-5.09 and the result was /dev/stdin: ASCII text. Is it depends on version of file? – ironsand Dec 1 '14 at 3:55
  • @Tetsu I tested with file 5.04 (Debian squeeze) and it was OK. Even in 2007, according to the changelog (not tested). But you can do the test I've given in my answer. – vinc17 Dec 1 '14 at 3:57
  • I just installed language-pack-ja from apt, then your example works perfectly. Thanks for your help! – ironsand Dec 1 '14 at 4:18
4

If you execute stat fileName it should give you exact the three characters. When I opened the file in the editor, I was unable to see anything. So noticing that the file size was 3 gave me clarity that it has a BOM.

Also, the post here was helful in my case.

hexdump -n 3 -C 2.txt
00000000 ef bb bf
ef bb bf // YES

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.