I have a script I am writing, that includes passing across a connection string to socat.

When I paste this into the command line it is fine, but when I open a file and paste it into there (pico filename.sh), the characters are converted to something else, and the connection string fails.

The code in question is:

conTest=$(echo -e -n "@\0\0'{abcde\0fghij\0Ç\x01Ó|" | socat STDIO tcp4:

The characters with an issue are:


which get converted to:


Is there a way to stop them being converted?

I am connecting to this machine (Debian) using putty (ssh) from my Windows 7 PC.

My initial text is in a file in notepad++ on my Windows PC. I then copy / paste it from there into the command line on my Debian Server. When I run this it works fine.

If I open a file using pico pico test1.sh and copy / paste the text in, then it appears to be ok. But as soon as I try and save it (ctrl-O) the characters change.

If I then run file test1.sh it says test1.sh: Bourne-Again shell script, ISO-8859 text executable.

If I run iconv -f ASCII -t UTF-8 test1.sh I get illegal input sequence at position 263 which is the Ç character.

If I try it with nano nano test2.sh then the copy / paste doesn't work and I get additional characters after the problem characters. Again, as soon as I try and save it, the same happens (the characters change).

Next I tried to copy the file across using sftp. I copied it as test3.sh, and then once on the server I opened it with vi using vi test3.sh. This showed the same problem in that the problem characters got replaced.

However, running file test3.sh gave me test3.sh: Bourne-Again shell script, UTF-8 Unicode text executable

I copied the file back via sftp and opened it on my windows pc, and the characters are all fine, suggetsing that the sftp process isn't a problem.

  • 1
    Apparently, your pico can't deal with non-ASCII characters. Is there any specific reason you need to use that editor? Have you tried changing your font to a UTF-8 font? – terdon Mar 30 '16 at 11:13
  • there's no reason, it's just what I am used to. I kind of presumed that if it worked on the command line, it would work elsewhere... – IGGt Mar 31 '16 at 7:44
  • nah, it depends on the program you're using. Did you try changing the font? – terdon Mar 31 '16 at 8:22
  • I tried setting :set encoding=utf8 in vi (I couldn't see how to change it in pico), but it made no difference. As soon as I open the file, the characters are converted. – IGGt Apr 4 '16 at 9:49
  • Well, pico really is quite limited. Have you tried nano? It's basically an enhanced version of pico and should be installed by default. Although, that said, both pico and nano can read a file with 0Ç\x01Ó correctly on my system. What is the output of running file filename.sh? Is the file's encoding UTF8? – terdon Apr 4 '16 at 9:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.