2

I have a Java class which the compiler refuses to compile due to \ufeff at the start of the file. I can view the fact that the BOM is present by vim -b file.java, but neither xxd nor hexdump show the two bytes. Is there some way to make them do so?

3

The U+FEFF character is encoded in UTF-8 over 3 bytes: ef bb bf.

xxd or hexdump shows you the byte content, so those 3 bytes, not the character that those 3 bytes encode like vim -b does.

To remove that BOM (which doesn't make sense in UTF-8) and fix other idiosyncrasies of Microsoft text files (which is likely the source of your problem), you can use dos2unix.

$ printf '\ufefffoobar\r\n' | hd
00000000  ef bb bf 66 6f 6f 62 61  72 0d 0a                 |...foobar..|
0000000b
$ printf '\ufefffoobar\r\n' | uconv -x name
\N{ZERO WIDTH NO-BREAK SPACE}\N{LATIN SMALL LETTER F}\N{LATIN SMALL LETTER O}\N{LATIN SMALL LETTER O}\N{LATIN SMALL LETTER B}\N{LATIN SMALL LETTER A}\N{LATIN SMALL LETTER R}\N{<control-000D>}\N{<control-000A>}
$ printf '\ufefffoobar\r\n' | uconv -x hex
\uFEFF\u0066\u006F\u006F\u0062\u0061\u0072\u000D\u000A
$ printf '\ufefffoobar\r\n' | dos2unix | hd
00000000  66 6f 6f 62 61 72 0a                              |foobar.|
00000007
  • Silly me, I was looking for the bytes fe ff, but now I realise that that's the codepoint, not the utf8 code. – oarfish Apr 25 '17 at 14:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.