Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm currently fiddling around with a dying harddisk, and while trying to cp data from it, I get errors like

cp: error reading ‘brokenFile’: Input/output error
# comparison: backtick: `, apostrophe '

I now try to copy the failed files again by using sed to convert the error messages to new cp invocations (I was successfull some times already). But: what are the funny quotation marks? They are even different at start/end. It is no backtick and no apostrophe. I copy/pasted it into my regex, but is there a better way? Maybe using compose?

share|improve this question
1  
It probably generates some Unicode characters for English quotes. You should get error messages more friendly to sed (and automated processing in general) if you set the environment variable LANG to C. –  celtschk Apr 25 at 14:05
    
They appear to be either wide or unicode characters. This is what i get when I try to put them in my terminal. ` ▒~@~Y ▒~@~X ` –  Livinglifeback Apr 25 at 14:07
1  
OK, I now searched for them in the character map. They are U+2018 LEFT SINGLE QUOTATION MARK and U+2019 RIGHT SINGLE QUOTATION MARK. –  celtschk Apr 25 at 14:08
    
@celtschk LANG=C works like a charm. I usually have en_US.utf8. –  Jasper Apr 25 at 14:19
    
... and sice I asked for compose key usage: the german wikipedia says Comp+<+' for left and Comp+>+' for right quotation mark. @celtschk do you want to write that up as an answer? –  Jasper Apr 25 at 14:24
show 1 more comment

2 Answers 2

up vote 3 down vote accepted

The characters generated are U+2018 LEFT SINGLE QUOTATION MARK () and U+2019 RIGHT SINGLE QUOTATION MARK (). Those are typographical single quotes for the English language, and are generated because of your current locale. If your current keyboard layout has a compose key, you can enter them with Compose<' (left quote) and Compose>' (right quote).

However, if you want to process the output of a command with other tools (like, in your case, sed), it is usually easier to change the local by setting the environment variable LANG to C. That way, programs will output error messages (and more generally, all output meant to be human-readable) in pure ASCII, which is generally more easily handled using text-based tools.

share|improve this answer
    
You can set the locale "locally" to each command by just adding "LC_ALL=C" in front of the command itself. ex: LC_ALL=C cp something somewhere . Only cp here will have that locale forced. BUT don't forget to add similar forcing to any interpreting commands as well (sed, etc). [otherwise things like [A-Z] ranges may turn out to be all letters except 'a', for example... read for example gnu.org/software/gawk/manual/html_node/Ranges-and-Locales.html . Added bonus : limiting to C range makes ranges faster, as containing far less characters in them than their unicode counterparts) –  Olivier Dulac Apr 25 at 17:15
add comment

A useful command to find out about a given character is GNU recode:

$ echo -n ‘’ | recode ..dump
UCS2   Mne   Description

2018   '6    left single quotation mark
2019   '9    right single quotation mark

Or the unicode command from Debian:

$ unicode ‘’
U+2018 LEFT SINGLE QUOTATION MARK
UTF-8: e2 80 98  UTF-16BE: 2018  Decimal: &#8216;
‘
Category: Pi (Punctuation, Initial quote)
Bidi: ON (Other Neutrals)

U+2019 RIGHT SINGLE QUOTATION MARK
UTF-8: e2 80 99  UTF-16BE: 2019  Decimal: &#8217;
’
Category: Pf (Punctuation, Final quote)
Bidi: ON (Other Neutrals)
share|improve this answer
    
I get from echo -n ‘ | recode ..dump the following: 1820 lettre mongole a ... –  Jasper Apr 25 at 16:07
    
@Jasper, on what system? I can't really see how the 18 and 20 could be revered. What's the output of locale charmap for you? Do you get the same with recode u8..dump? –  Stéphane Chazelas Apr 25 at 17:51
    
locale charmap --> UTF-8. Same output with u8..dump. I'm running ArchLinux. –  Jasper Apr 25 at 18:03
    
on an x86_64 CPU. I didn't realize until now that my numbers are just swapped... endianness-issue?! –  Jasper Apr 25 at 18:18
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.