1

I can't seem to eliminate this ° symbol.

I have tried:

  sed 's/°//g' file.txt

and

  sed 's/\°//g' file.txt

but both did not work.

I'm running ubuntu 16.

The symbol does not seem to be detected - I did copy and paste.

This did not do anything either to find the symbol

  sed '/°/!d' file.txt

Any suggestions?

2

Already the first should work. This symbol has no special meaning, although it is a non-ascii character. No \ is needed, don't be opportunist, if it should work without a \, make it working on this way! It is like you would try

sed 's/ő//g'

In such cases, the most probable reasons are:

  1. Encoding problem. Check, in which encoding is the file (file -szL file.txt), and in which encoding are you using (echo $LANG). It is because the the "°" can have a different byte sequence in the different encodings. Best if both are in utf-8. Extension: Ubuntu 16 is good utf-8 friendly by default, at most in the case of US-only, very old, continuously upgraded system, can I imagine such an encoding problem. It is much more realistic, that your text file is not in utf-8. The file command will say it to you.
  2. You may filter for a similar character, but not for the exact one. The are thousands of displayable characters on a not very old, utf-8 system, many of them are special hieroglyphs like this, any of them look very similar. The solution is easy also in this case: cat the file, and copy-paste the glyph into your command line.

If you followed these, 95% that you found and fixed the problem on this point. The remainder is for the 10% of the cases.

If none of these work, you can debug the things on the deepest byte-level. In your case I would do this:

  1. I would first start this sed command, but I would give it the /dev/zero as source: sed 's/°//g' </dev/zero >/dev/null. This will result that the sed will work in an infinite loop, working on an infinite long zero-byte stream, and don't exits on the spot. So, you can debug it.
  2. Suspend its execution with a ctrl/z.
  3. Check for its pid with a pidof sed. You get an at most 5-digit integer.
  4. With a hexcat /proc/12345/cmdline you will be able to see the byte-level data of this sed command line.
  5. You can check the byte-level coding of the "°" by doing the same hexcat file.txt.
  6. The two must match. If they don't, there is some evil in the picture, try (2) or ask a new question.

Many linux distros simply removed hexcat, one of their most useful tool, from their distrib, because they think nobody used it. But there are some other tool, as far I can remember, maybe xxd is one of them. If you can't make hexa dumps with anything, ask it in a new question.

0

Well try this easy aproach:

echo "°" | xxd

This will return something like:

0000000: c2b0 0a

You need the code c2b0, I guess 0a is the new line character.

Then try:

sed -e 's/\xc2\xb0//' file.txt

Hope that works.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.