The tag has no wiki summary.

learn more… | top users | synonyms

4
votes
0answers
34 views

How to declare accented variables in Python [migrated]

I need to write few basic scripts for students in Python, like this one: #!/usr/bin/python # -*- coding: utf-8 -*- mia_età = 31 print mia_età But apparently I cannot use accented characters while ...
1
vote
1answer
21 views

Can not get the file names with convmv

I have just uploaded some files on a FreeBSD server. Some files have the ??? into their names. The files came from a windows machine, so I try this: $ convmv -f cp850 -t utf-8 * Starting a dry run ...
2
votes
1answer
19 views

Cyrillic file names broken after upgrading Slackware

After upgrading glibc and some other libraries on Slackware instead of cyrillic names I see the garbage: | Was before | Is now | |------------+------------------| | Вождение | ...
2
votes
1answer
51 views

Convert TeX accents to UTF-8 in Python

edit below First of all, I am quite sure this is a duplicated question. But I have no idea how to search for it. I spent quite a long time already. I am reading with python from a .txt file the path ...
0
votes
0answers
42 views

LMDE2 / MATE locale charset inconsistency

I have a machine with Linux Mint Debian Edition 2 "Betsy" (installed as RC, with all available updates applied) and MATE desktop environment. My problem is that in some applications special characters ...
0
votes
1answer
34 views

make `tar` archive from NTFS with non-ascii file names

Loaded a live disk into a Windows based machine with NTFS files. Being a Linux user for some years, using tar came to my mind instantly and made some backup archives from partitions and stored them ...
4
votes
4answers
221 views

Make tail not modify the charset of the terminal

In a linux terminal (CentOS) I am using the command tail --follow=name my-rolling-file.log in order to see the logs of my application. Sometimes in the log there is some binary data dumped (I dump ...
0
votes
1answer
33 views

Error not found [closed]

I wrote this script to encode / encrypt data from a sql query by changing the names of tables and fields (I played on the ASCII character codes). #! /bin/bash chr() { [ "$1" -lt 256 ] || return 1 ...
28
votes
2answers
809 views

find(1): how is the star wildcard implemented for it to fail on some filenames?

In a file system where filenames are in UTF-8, I have a file with a faulty name; it is displayed as: D�sinstaller, actual name according to zsh: D$'\351'sinstaller, Latin1 for Désinstaller, itself a ...
0
votes
1answer
33 views

iconv can't convert circled digits to/from Japanese encodings

The unicode range of circled digits (U+2460 .. U+2468) cannot be converted to, or from, any of the Japanese encodings (EUC-JP, Shift-JIS, ISO-2022-JP), even though they exist there, and I run across ...
0
votes
2answers
93 views

Check whether text contains non-ASCII characters in a shell script

I have a file, I want to determine if it contains only English ASCII characters. I thought this is a very common question, but when I googled it, there is no direct answer or related. According to ...
1
vote
1answer
87 views

cp: cannot stat error - when filename has Asian characters

I am simply trying to copy files using cp -r /home/user/source/ /home/user/destination/ but it throws me the cp: cannot stat /source/filename.xxx error for some of the files. When I searched for this ...
1
vote
4answers
98 views

How to recognize which ASCII character in hex is this?

We have a textfile that we want to clear from "bad" characters. If we open it with vim (with ":set number"): 57000044 zo¥<9a>¥ge¥o¥graph¥i¥cal¥ly 39999999 pariá¹<83>Å<9b>a The ...
1
vote
2answers
45 views

How to only leave in given characters in a file?

Purpose: tr -cd '\12\40-\176' but with accents, ex.: á, í, ű, ő, ö, ü, ű, ú, ó, é. In octal: "á" is 341 "\n" is 12 So I tried: [user@notebook ~]$ printf 'XXXá\nXXX' | tr -cd '\12\341' | ...
1
vote
1answer
39 views

Any way to interpret pstree?

-bash-3.2$ pstree 29838 RBcron.shâââjavaâââ15*[{java}] -bash-3.2$ pstree 24423 bashââ¬âRBcron.shâââjavaâââ15*[{java}] ââvim -bash-3.2$ Any way to interpret this? Why would this happen?
2
votes
1answer
91 views

Question marks instead of national letters in console

I have Netgear Stora MS2000 (which uses a modified version of Red Hat) and have strange font issues while connecting by SSH. Note that this occurs both when connecting by PuTTY from Windows and by ...
1
vote
1answer
40 views

How do I edit text in Cyrillic / Russian on OpenBSD?

I'm trying to make it possible to easily edit text in Cyrillic / Russian on a remote OpenBSD 5.2 amd64. I'm using emacs-21.4p18-no_x11, which doesn't show any Cyrillic characters whatsoever when I ...
1
vote
2answers
379 views

How to convert unknown-8bit file to utf8

I have a .srt file that displays as gibberish when I open it in gEdit in ubuntu. So I want to convert it to utf8 to be able to read it. When I try to figure out what the encoding it give: file -i ...
1
vote
0answers
70 views

Rename files with non-utf8 characters using detox

I am trying to use detox in the following way detox -s utf_8-only -r -v -n to detect invalid filenames on my system. The command performs mostly ok. Unfortunately it falsely detects also German ...
4
votes
2answers
112 views

Characters visible in vi, but not in cat.

I have a 300 lines file with ^@ character between each characters in the file. (I cannot post the entire contents for security reasons, so I am pasting only the first line) [mercury@app01 ftp_logs]$ ...
16
votes
5answers
2k views

Generating a random password; why isn't this portable?

I want to generate a random password, and am doing it like so: </dev/urandom tr -dc [:print:] | head -c 64 On my laptop, which runs Ubuntu, this produces only printable characters, as intended. ...
1
vote
1answer
104 views

Special characters in a web feed

I am running a script to check my gmail from the command line and display the first 35 characters in the title. curl -u username:password --silent "https://mail.google.com/mail/feed/atom" | \ grep ...
1
vote
1answer
97 views

script to parse integer and convert it to char code

I am noob on Unix. I've a very unique problem where and really appreciate your help. I need to parse string containing hash code value and convert the hash code is equivalent char representation. Here ...
0
votes
2answers
54 views

Find out which encoding method less uses to display the content of a text file?

There is a plain text file, which Emacs can't display correctly, but less can. I wonder if less automatically detects the encoding method for each individual text file, or just use a single default ...
3
votes
1answer
163 views

How do I view cp1251 text file in UTF-8 console?

Attempt 1: $ less subs.srt "subs.srt" may be a binary file. See it anyway? <C8><F2><E0><EB><FC><FF><ED> ... Attempt 2: $ LANG=ru_RU.CP1251 less subs.srt ...
1
vote
0answers
62 views

How do I configure medit to display UTF-8?

My environment is set to use UTF-8. I produce a file in UTF-8 encoding. It works correctly on the terminal. For example: $ echo ééé > test $ cat test $ echo é | od -t x1 0000000 c3 a9 0a ...
1
vote
1answer
104 views

iconv terminated by signal 7

I was running a little bash script on some text files: find . -name "*.dat" -exec iconv --from-code='UTF-8' --to-code='ASCII//TRANSLIT' {} --output={} \; My machine is a Ubuntu 14.04 LTS. After a ...
2
votes
0answers
72 views

iconv cannot replace Ø

It appears that iconv cannot, for example, replace the letter Ø. It was also noted in the second answer to http://stackoverflow.com/questions/3371697/replacing-accented-characters-php I have two ...
3
votes
2answers
86 views

Why can't I grep this command line option?

The output of man icc contains -Ldir Tells the linker to search for libraries... Doing man icc | grep "\-L" gives back the "-Ldir" line while man icc | grep "\-Ldir" does not. What am ...
2
votes
2answers
144 views

Check files for conventions

I need a script that checks and lists all text files for project conventions. With conventions I mean for example: UTF-8 encoding No trailing white spaces Newline at the end of file No not-ascii ...
1
vote
1answer
248 views

Converting a UTF-8 file to ASCII (best-effort)

I have a file in UTF-8 that contains texts in multiple languages. A lot of it are people's names. I need to convert it to ASCII and I need the result to look as decent as possible. There are many ...
2
votes
1answer
185 views

$Unix character set conversion

I'm confused by character-sets in Unix. I have a CSV file downloaded via SFTP: $ file -ib myfile text/plain; charset=us-ascii The purpose for this character-set quest is that the data within file ...
2
votes
1answer
63 views

Change Text File Encoding without knowning the source encoding

I want to change the charset encoding for a file in unix with a single command but since this will be an automated process it's impossible for me to know the source encoding. So I want a command that ...
0
votes
0answers
39 views

jumbled character set

In my Linux env, for some unknown! reason when I process a raw unformatted file which I copied from Win7prof (was encoded in utf-16), the resulting file is all jumbled. It will print OK in Linux ...
5
votes
1answer
160 views

Some apps doesn't accept some characters from the «Compose» key

The problem is that the compose key works fine, but some application doesn't accept some characters from it. E.g. I can paste the character ∞ in Emacs (Compose+8+8), but this won't work in FireFox, ...
1
vote
1answer
105 views

File weird character encoding [closed]

I'm on ubuntu 13.10 and am writing a program in eclipse that's supposed to output my result in a file. I'm building an LZW compression and decompression program that uses character conversion to its ...
1
vote
0answers
80 views

Why doesn't this sed substitution for a non-ASCII byte work?

I tried this substitution with GNU sed on OS X (4.2.2 installed through Homebrew). But it doesn't work. printf "Hello\x92 World" | gsed -r "s/[\x92]/'/g" The expected output is: Hello' World The ...
0
votes
0answers
22 views

Change encoding of a text file in-place: iconv alternative? [duplicate]

Apparently, iconv is unable to use the same file as input and output file. Other tools, e.g. sed, have an -i switch that allows "updating" a file in-place. (Under the hood, temporary files are ...
5
votes
1answer
213 views

Can not use `cut -c` (`--characters`) with UTF-8?

The command cut has an option -c to work on characters, instead of bytes with the option -b. But that does not seem to work, in en_US.UTF-8 locale: The second byte gives the second ASCII character ...
3
votes
1answer
164 views

How to remove Russian and Arabic strings from a text file

Problem: I have text file with approximately 1M lines, each line consists of multiple words Some lines contain Russian or Arabic words (lets call them "bad lines"), I want to remove only these bad ...
0
votes
2answers
121 views

Can CentOS be set to pure EBCDIC?

I need to directly manipulate files from a mainframe. Text would be encoded in EBCDIC. Data can contain binary and packed decimal fields. I know I can deal with the data. I am trying to avoid the ...
0
votes
0answers
177 views

A quick and dirty way of finding invalid characters in a text file by gedit

In vonbrand's comment to How do I scan for invalid characters on gedit? A quick and dirty way would be to edit the file (probably adding and deleting something will be enough to make gedit ...
0
votes
1answer
33 views

Mixture of encodings

For a text which is a mixture of different encodings, is there no way to find all encodings? And convert them to a single encoding such as utf8? I copied from different sources possibly with ...
1
vote
2answers
82 views

Emacs shows octal escapes for some characters in a UTF-8 file

I copied some math formulas into some files under gedit, using the default settings of gedit from websites, such as math.stackexchange.com, wikipedia, ... When I open the files now under emacs, the ...
0
votes
2answers
141 views

How does rsync encode filenames?

By using strace on the active rsync process, I can see output like: read(4, "9\0\0\7\1\10\0\1\10\0\1\10\0\1\10\0\1\10\0\1\10\0\1\10\0\1\10\0\1\10\0\1"..., 16384) = 61 write(3, ...
1
vote
1answer
75 views

What does “position” mean in iconv error message?

I get error when converting a large file: $ iconv -f GB2312 -t UTF-8 2001.txt -o 2001_u.txt iconv: illegal input sequence at position 245256667 What does the position mean in this error? I tried it ...
1
vote
1answer
1k views

Converting documents to text with UTF-8 encoding

unoconv uses LibreOffice to convert any file that LibreOffice can convert. Now I need export to text with the UTF-8 character encoding. I use the following command: unoconv -f txt -e ...
11
votes
4answers
601 views

support for utf-8 encoding with lpr

While trying to send a text file to the printer via lpr from xterm, the content was corrupted beyond recognition, the cause of which was ultimately traced to the encoding of the file. If I instead ...
4
votes
2answers
1k views

How do I rename files with strange characters

I have file with greek or cyrillic characters. It is not owned by me, but by the web server user (www). I cannot use the shell as the web server user (www) or as root, but I've used a script ...
5
votes
1answer
2k views

How to display Chinese characters correctly on remote Red-Hat machine?

I am using Ubuntu14.04 to connect to a remote host. Which its version is: Linux version 2.6.32-431.11.5.el6.yyyzzz.x86_64 (gcc version 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC) ) #1 SMP Thu Jul 3 ...