The text in your file is pages = {1113},
, yes it looks like the number 1113
but actually there is a different character after the first 1
. And, yes, you can copy-paste the string from the edit link for this web page to get the encoded character.
We can look inside the string with some tools:
$ a='pages = {1113},'
Or, to make it explicitly clear and allow an easy copy-paste without using the edit page:
$ a=$(printf 'pages = {1\xc2\x96113},')
$ echo "$a" | od -An -tx1c
70 61 67 65 73 20 3d 20 7b 31 c2 96 31 31 33 7d
p a g e s = { 1 302 226 1 1 3 }
2c 0a
, \n
$ echo "$a" | sed -n l
pages = {1\302\226113},$
$ echo "$a" | xxd
00000000: 7061 6765 7320 3d20 7b31 c296 3131 337d pages = {1..113}
00000010: 2c0a
So, the character is two bytes values c2 96
(in hex) or 302 226
(in octal).
It probably is the UTF-8 encoding of a byte value of 96
, or expressed as an Unicode character: U-0096
.
That value, in present times UTF-8, or better yet, in ISO-8859-1, is a control character in the C1 region of control characters(Wikipedia page) and (Unicode PDF) that goes from 128 to 159 in decimal. In specific, the U-0096 is called "START OF GUARDED AREA" or SPA.
That value (dec 150) is beyond the ASCII range (0-127) and was (in older times) used to represent several characters depending on the code-page used. It seems reasonable to assume that is was previously a dash (to mark the range 1-113) as encoded in Windows-1252 (Microsoft page) (Wikipedia 1252) and called an en dash (which is the smaller of the two dashes en and em) (Wikipedia en dash) or simply, in layman terms, a dash (-
).
Q1: Is there anything wrong with this file?
Not really, control characters are valid characters, rarely used but valid none-the-less.
But you may replace them with a dash to make editing easier.
<file.txt sed 's/\xc2\x96/-/'
Q2 - How can I search for other occurrences of it inside the same file?
sed -n '/\xc2\x96/p' # will print lines that contain that character.
Or, grep could search for the character (the color highlight will not be visible as the character is non-printable) and print the line.
c="$(printf "\U96")" ; grep "$c" file.txt
Or more broad, find all characters in that control character range and list the files that contain such characters:
grep -rlP "[\x80-\x9f]"
Q3 - How can I grep for other files that may contain the same problem/character?
This will list (-l
) the files that match the character.
grep -rlP "\x96"
hexdump -C filename
and look at the encoding of what is "visible" to you as<96>
. Context should help to pinpoint it.hexdump -C
showsc2 96
. How can I search for other occurrences of the same thing?gedit
when you have set it to manage your language and UTF-8. In this case you can remove that character.