How to remove duplicate characters?

Question

If I am having a line as:

Thhiisss iisss mmyyy nameeee

I want to print this as:

This is my name

What is the unix command for this?

Answer 1

With tr:

echo "Thhiisss iisss mmyyy nameeee" | tr -s 'a-z'

Explanation: The -s switch of tr "squeezes" repeat characters. As shown, the switch can be used with a range of characters: a to z.

Answer 2

One way with sed:

sed ':X;s/\(.\)\1/\1/g;tX'

or even simpler:

sed 's/\(.\)\1*/\1/g'

(thanks Costas and mikeserv for comments).

Answer 3

On a GNU system you'll need to use sed or similar if your locale uses multibyte characters (as jimmij suggests) because GNU tr can only reference a character per byte. In an ASCII locale you can remove all duplicates w/ tr like:

LC_ALL=C tr -s '\0-\255' <input

So...

echo Thhiisss iisss mmyyy nameeee|
LC_ALL=C tr -s '\0-\255'

...prints...

This is my name

You can also do it selectively by referring to your targets by range:

echo TThhiisss iisss mmyyy nameeee|
LC_ALL=C tr -s '\101-\132'

...or...

echo TTTThhiisss iisss mmyyy nameeee|
LC_ALL=C tr -s '[:upper:]'

...which work out to be the same thing, and which both print:

Thhiisss iisss mmyyy nameeee

...or use [:punct:], [:digit:], [:lower:], [:alpha:] or whatever you would like. You can also negate the selection w/ -c so...

echo 'TTTThhiisss     iisss mmyyy nameeee' |
LC_ALL=C tr -cs '[:upper:]'

...prints...

TTTThis is my name

Answer 4

Try tr:

echo "Thhiisss iisss mmyyy nameeee" | tr -s 'hismye'

Answer 5

echo "Thhiisss iisss mmyyy nameeee" | grep -o . | uniq | tr -d '\n'