When I used cut today, I found out it doesn't treat a UTF-8 character as a character, but 3 characters because it's 3 bytes long.

This seems to be generally true for many tools.

Are there versions of the coreutils that are UTF-8 aware?

My locale output:


Here is when cut doesn't work

echo 哈哈 | cut -c 2-

The right output should be

if cut -c worked with multibyte characters.

  • Did you set your locale properly? What's the reading of locale (w/o arguments) command invocation?
    – alex
    Jul 2, 2011 at 5:00
  • I have updated with the locale.
    – Chao Xu
    Jul 2, 2011 at 5:13
  • OK, can you also add some example of your cut command line?
    – alex
    Jul 2, 2011 at 5:40
  • Can confirm this on Ubuntu 10.04 with echo ßßßß | cut -c 2- -> �ßßß (LANG=en_US.UTF-8) Jul 2, 2011 at 8:52
  • The sad thing is that, 3 years later, this is still true in Ubuntu 13.10...
    – Dr. Mike
    Mar 22, 2014 at 10:54

2 Answers 2


GNU coreutils do understand UTF-8 in general. For example echo 哈哈 | wc -m correctly outputs 3 in a UTF-8 locale (note that the option is -m, not -c which for historical reasons means bytes).

This is a bug in cut. Looking at the source of cut, cut on characters is simply not implemented: the -c option is treated as a synonym of -b.

A workaround is to use awk. GNU awk copes with UTF-8 just fine.

awk '{print substr($0,2,length)}'
  • 9
    It would be worth filing a bugreport (even if it ends up being a duplicate) to annoy the coreutils maintainers into actually fixing bugs like this, rather than just adding stupid hacks and toy utilities that nobody needs... Jul 3, 2011 at 16:27

This looks like a bug in your build/version of coreutils. I can reproduce this on Ubuntu 10.10 Maverick Meerkat but not on Fedora 15.

[patches@holocene ~]$ cat /etc/fedora-release 
Fedora release 15 (Lovelock)
[patches@holocene ~]$ rpm -q coreutils
[patches@holocene ~]$ echo 哈哈 | cut -c 2-
[patches@holocene ~]$ sudo chroot /mnt/maverick
root@holocene:/# grep DISTRIB_DESC /etc/lsb-release
root@holocene:/# dpkg-query -s coreutils | grep Version
Version: 8.5-1ubuntu3
root@holocene:/# echo 哈哈 | cut -c 2-

If you're also using Ubuntu, you can report a bug to the Ubuntu coreutils packagers by running the following command:

apport-bug coreutils

Update: Gilles points out in the comments that this is a bug in the upstream version of coreutils that Fedora has patched. You can find their patch here if you would like to try to patch it yourself to get it working.

  • I just looked at the source, and it's an upstream bug that Fedora apparently patched. The upstream source simply makes -c an alias of -b. Jul 2, 2011 at 10:44
  • @Gilles: Interesting. I linked to Fedora's patch in my answer in case someone wants to try and fix it for themselves.
    – Patches
    Jul 2, 2011 at 11:00
  • Link is broken.
    – corvus_192
    Jan 7, 2017 at 19:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.