The tag has no wiki summary.

learn more… | top users | synonyms

4
votes
4answers
2k views

How to convert hex chars to normal chars?

I tried, but I'm stuck at "escaping" the "sed's": sed -i 's/\\x0/NUL/g' $1 sed -i 's/\\x1/SOH/g' $1 sed -i 's/\\x2/STX/g' $1 sed -i 's/\\x3/ETX/g' $1 sed -i 's/\\x4/EOT/g' $1 sed -i 's/\\x5/ENQ/g' ...
1
vote
1answer
1k views

How can I change locale/encoding to avoid getting weird characters in terminal?

I have installed tree on my ubuntu box, when i connect to it from Putty and I launch tree i get this : $ tree âââ html.vim -> xml.vim âââ js.vim -> xml.vim âââ xml.vim instead of this : $ ...
16
votes
3answers
5k views

bulk rename (or correctly display) files with special characters

I have a bunch of directories and subdirectories that contain files with special characters, like this file: robbie@phil:~$ ls test�sktest.txt test?sktest.txt Find reveals an escape sequence: ...
4
votes
3answers
1k views

russian letters in gentoo console as '?'

I managed to get russian language working in my Gnome apps, but neither in the console (Alt-Shift-F2 and such) nor in the Gnome Terminal valyagentoo va1en0k # locale LANG=ru_RU.UTF-8 ...
0
votes
1answer
2k views

Script failing with “command not found: ^M”

When I try to run the following script in zsh, via the command /bin/zsh ~/.set_color_scheme.sh I get the following error: command not found: ^M The script has u+x permissions and it used to work on ...
8
votes
2answers
685 views

Why are wc -m and wc -c different?

As a C programmer, I was surprised to see that wc -c (which count the number of bytes), and wc -m (which counts the number of characters) output very different results for a long, text file of mine. I ...
2
votes
1answer
840 views

file, awk and iconv to automatically adapt tex-file encoding

I have a set of tex-files with mixed encodings, e.g. (subset of output of file -i *.tex) f1.tex: text/plain; charset=utf-8 f2.tex: text/plain; charset=utf-8 f3.tex: text/x-tex; charset=us-ascii ...
3
votes
1answer
339 views

tr not replacing apostrophe

I want to convert all apostrophes in this file to X: Bob's book Bob’s book Bob′s book # (Might look the same but actually different) The first apostrophe is replaced as expected: $ cat file | tr ...
1
vote
2answers
273 views

How do I set the mysql server character set in FreeBSD 8.2?

My mysql server's current server character set is latin1. My server is started on startup via rc.conf setting mysql_enable="YES". How to set my server to use utf8 as the server character set?
2
votes
2answers
172 views

Is it possible to add CP-1252 support to CentOS?

We have a CentOS backend running server software that is processing files generated by our clients. Many of our clients are Windows users, and generate these files on their Windows machines in the ...
1
vote
0answers
190 views

Force mysqldump encoding

I have two mysql servers, viewed from the same mysql client, right_server is rendering the french characters correctly, wrong_server is not. Encoding is the same : $ mysql -h right_server -NBe "SHOW ...
7
votes
2answers
3k views

Printing decimal to ascii character, my command does not output as intended

I wanted to output a string of all the ascii characters with the following command for i in `seq 32 127`; do printf "%c" $i; done The output of the above command is: ...
4
votes
1answer
575 views

opendir and readdir encoding strings behind my back?

(You can skip the details to the last couple of lines if you're able to answer the question :) ) I'm on an Ubuntu 12.04. I'm trying to resolve an old issue that I've posted about in the past (if ...
2
votes
1answer
211 views

Qt based apps have wrong encoding?

I have Ubuntu 11.04. Yesterday I installed VirtualBox and the encoding was wrong (I couldn't read anything). I know its UI very well so I tried changing the UI display language but it didn't work as ...
1
vote
1answer
661 views

Is Native Language Support (NLS) kernel support still necessary?

Does ext4 / cifs system need kernel NLS support ? I'm not sure if it's being handled by user-space program (decoding / encoding) ?
7
votes
2answers
2k views

Several questions about file-system character encoding on linux

Due to a lot of file exchange works between Windows (GBK encoding) and Linux (UTF-8 encoding), it will encounter character encoding issues easily, such as: zip/tar files whose name contains chinese ...
13
votes
3answers
2k views

Understanding Unix file name encoding

I have a hard time understanding how the file name encoding works. On unix.SE I find contradicting explanations. File names are stored as characters To quote another answer: Several questions about ...
2
votes
2answers
3k views

terminal: non-english characters

I used to be able to copy&paste (or type if I load respective keyboard layout) any kind of characters (eg. é ö ñ ß و 好) in my console/terminal. Now I bought a new computer and it does not work ...
7
votes
2answers
360 views

Does OS X have its own line format?

Obviously there are at least two newline types: DOS and Unix. But does OS X have its own plaintext 'format'? I opened a text file in nano and was surprised to see: [ Read 26793 lines (Converted from ...
0
votes
2answers
990 views

Terminal Character Encoding Issue BitchX

I ran BitchX on three different Linux OS-es because I got this weird � (question mark) on two of them (Linux OpenSUSE 11.4 and BackTrack 5 R1) using yakuake/konsole (OpenSUSE) and terminator/tilda (BT ...
3
votes
1answer
1k views

How to use grep/ack with files in arbitrary encoding?

On my Linux desktop I have a UTF-8 locale. When I try to search some KOI8-R encoded files with grep (ack), it fails. If I manually encode the pattern into KOI8-R and pass that as an argument, it ...
3
votes
2answers
1k views

Font encoding in Terminal

I have a problem with my font encoding in terminator When I first lauch it and then launch vim nerdTree the small ▸ are displayed well. But if I quit vim and then type reset in my terminator they are ...
7
votes
1answer
4k views

How to fix character encoding in bash (arch linux)

I have some issues with the encoding of german umlauts in bash. It mostly is working fine, but when i am typing a invalid command, the error message generated by bash contains strange escape sequences ...
1
vote
1answer
725 views

TUI interfaces not displaying correctly

I have an Ubuntu 11.04 server on Amazon EC2 which cannot display TUI interfaces correctly, like on Midnight Commander or tree. I'm quite sure that this problem was not there when I set it up, and I ...
1
vote
2answers
3k views

Converting text into ASCII/ISO-8859-1

I'm looking for some tool that can convert text, ideally from UTF-8 (but ISO-8859-2 and WINDOWS-1250 would be fine) into ASCII/ISO-8859-1? I have seen some online transliteration tools but I need ...
4
votes
1answer
736 views

How to produce a file encoded in Mac OS Roman

Inside a bash script I have a line that creates and then inside a loop, append strings to a text file, something like echo $myTextString >> file.txt I need this final file to be formatted ...
6
votes
2answers
1k views

working with filenames in a different encoding over ssh

I'm ssh'ing to a remote system where a different encoding for the filenames (and for the users' locales) has been used. And this causes some problems. Problems solved by matching the locale settings ...
2
votes
2answers
376 views

Remove copyright symbol

I'm trying to parse a RSS feed on the command line. The code works so far but the feed contains a copyright symbol which I try to remove (it is latin1 encoded). How do I remove the copyright symbol ...
5
votes
3answers
3k views

Most common encoding for strings in C++ in Linux (and Unix?)

For creating a C++ program that is source code level portable between Windows and Linux and handles internationalization well, there are IMHO three main encodings to consider: The encoding of the ...
6
votes
2answers
8k views

How can I test the encoding of a text file… Is it valid, and what is it?

I have several .htm files which open in Gedit without any warning/error, but when I open these same file in Jedit, it warns me of invalid UTF-8 encoding... The html meta tag states ...
21
votes
2answers
13k views

How can I set VIM's default encoding to UTF-8?

I'd like to contribute to an open source project providing translated strings. One of their requirements is that contributors must use UTF-8 as the encoding for the PO files. I'm using VIM 7.3 on ...
3
votes
4answers
3k views

How do I re-encode a mixed encoded text file

I've got a log file that is ASCII, except for a few UTF-8 characters (which I can fix for a future version). For the moment, I need to figure out how to get this file to a ...
2
votes
1answer
187 views

XML text file with `^@` characters in it?

I have an XML file that I need to parse. When I open it in nano, nano give me the message (converted from Mac format). However between each character, there is a ^@ sequence, like so: ^@t^@h^@e^@ ...
1
vote
0answers
220 views

How to configure character encoding in OpenFT?

When transferring files over: http://ts.fujitsu.com/products/software/openseas/openft.html then the charset of files get's scrumbled...I can't seem to find any character encoding settings, how to ...
5
votes
2answers
2k views

special character in filename (\#033OA)

I've got a slight problem with a very stubborn error during an rsync. It's caused by a file with a special character in its filename. There's been others but I could sort that out by doing some ...
6
votes
2answers
145 views

What is the connection between a gedit bug and a Unix-&-Linux Q/A href?

While answering a Unix-&-Linux question, I observed that Gedit and two other editors, Leafpad and Medit (I tested 12 editors altogether) exhibit the a certain bug. As it turns out, the bug is ...
6
votes
1answer
406 views

`^M` at the end of each line of text files generated under Windows

I was wondering why if you open a textfile made in Windows notepad under unix you will find that it has ^M where there should be a new line? My understanding is that in Windows, every line is ended ...
4
votes
2answers
813 views

Encoding changed after running cat

After running cat on some files I get the infamous 'bushes': ␌⎺└⎻⎼␊⎽⎽␊␍ ├▒⎼ °␋┌␊. B␊├├␊⎼ ┤⎽␊ ├▒⎼'⎽ ≥ which I believe are result of encoding being changed. Is there any way to recover the terminal ...
6
votes
1answer
4k views

Foreign characters won't display in SSH

I'm having some problems with using ssh and a remote server (which I do not have admin access to) - Specifically, there are a couple of folders on there with Korean and Cyrillic text. When I display ...
5
votes
3answers
2k views

How to print all printable ASCII chars in CLI?

How can I list all the printable ASCII characters in the terminal?
3
votes
4answers
2k views

How to convert to HTML code?

Are there any scripts that can convert between text (e.g. <hi>) and the html entities version (&lt;hi&gt;) like this website does? Or at least a PHP file?
2
votes
2answers
664 views

sed, grep, or tr command that returns only Latin characters from a UTF-8 file

I am working with the text of the 300 tang poems, which unfortunately is a single file containing both Chinese and English. Since I am interested in 'extracting' the English, I hope to use sed, ...
11
votes
2answers
6k views

Filtering invalid utf8

I have a text file in an unknown or mixed encoding. I want to see the lines that contain a byte sequence that is not valid UTF-8 (by piping the text file into some program). Equivalently, I want to ...
16
votes
3answers
9k views

What charset encoding is used for filenames and paths on Linux?

Does it depend on what file system I use? For example, ext2/ext3/ext4 but also what happens when I insert one of those "joliet" CD-ROMs with ISO 9660? I've heard that POSIX contains some sort of spec ...
5
votes
1answer
540 views

why does my emacs whitespace mode add “” to my buffer?

I'm using Putty to ssh into a RedHat box, and I'm running GNU emacs inside the putty ssh window. When I turn on whitespace-mode using M-x whitespace mode, I notice I see a "Â" before each space or ...
1
vote
1answer
555 views

How to build-compile a .c file

I have little experience with LINUX . I am using Debian. It has a library glibc which has several useful programs. iconv is the program that i want to use to do several charset conversions... However ...
2
votes
1answer
767 views

What encoding does my Konsole support?

How can I check what encoding (ASCII, UTF-8, UTF-16 etc) my Terminal uses? It shouldn't matter, but I am using Konsole on KDE3.5.