Simple sed replacement of tabs mysteriously failing

Question

This ought to be really simple, but for some reason it is not working:

sed -i.bak -E 's/\t/  /' file.txt

Instead of replacing tab characters, it's replacing t characters. I've tried every variation on this I could think of, playing with quoting, etc. I've Googled and found everyone else using pretty similar expressions and they seem to work for them.

The -E is an OS X thing. I thought the failure might be a result of some weird quirk of OS X's sed, so I tried it with Ruby as well (without the -i), and got the same result:

ruby -pe '$_.gsub!(/\t/,"  ")' < file.txt > file.new

I'm using Bash 3.2.51 on OS X, and iTerm, although I can't see how any of those could be terribly relevant. I haven't set any weird environment variables, though I can post any that you think might be relevant.

What could be wrong?

UPDATE: I must have made some other mistake or typo when I tried the Ruby version, since Gilles points out that it does work (and I've never had him steer me wrong!). I'm not sure what happened, but I'm pretty sure it must have been my mistake.

Answer 1

The syntax \t for a tab character in sed is not standard. That escape is a GNU sed extension. You find a lot of examples online that use it because a lot of people use GNU sed (it's the sed implementation on non-embedded Linux). But OS X sed, like other *BSD sed, doesn't support \t for tab and instead treats \t as meaning backslash followed by t.

There are many solutions, such as:

• Use a literal tab character.

  sed -i.bak 's/  /  /' file.txt
  

• Use tr or printf to produce a tab character.

  sed -i.bak "s/$(printf '\t')/  /" file.txt
  sed -i.bak "s/$(echo a | tr 'a' '\t')/  /" file.txt
  

• Use bash's string syntax allowing backslash escapes.

  sed -i.bak $'s/\t/  /' file.txt
  

• Use Perl, Python or Ruby. The Ruby snippet that you posted does work.

Answer 2

Use a Bash specific quoting which lets you use strings like in C, so that a real tab character is passed to sed, not an escape sequence:

sed -i.bak -E $'s/\t/  /' file.txt

Answer 3

sed -i $'s/\t/  /g' file.txt 

works for me on OS X and is the same command i use on linux all the time.

Answer 4

As noted, not all sed implementations support the notation of \t as a horizontal tab.

You can easily achieve your substitution with:

 perl -pi.old -e 's{\t+}{ }g' file.txt

This performs an in situ replacment which preserves your original file as "*.old". Perl allows alternate delimiters for the classic / making the expression much more readable (i.e. devoid of the "leaning toothpick" syndrome).

The + says one or more repetitions of a tab character are to be replaced. The g modifier enables global replacements throughout the end of each line.

Answer 5

You can also use echo inside sed:

sed -i "s/$(echo '\t')//g"

Answer 6

If you want a more powerful sed (supporting \t and more) than the one on OS X, install GNU sed.

Answer 7

If it is okay to require bash or zsh as a shell, then this is the easiest solution I can think of:

sed "s/$(echo -n -e "\t")/ /" file.txt

Note however, that echo flags (-n and -e) are undefined in POSIX, so a POSIX conform shell doesn't require to understand theses flags, yet many will for compatibility reasons.

Answer 8

Drawing from the above answers, here is a simple copy-and-pasteable version that works on OS X and you don't have to think about.

mysql -h myhost.com -u username -p  -e "select * from users;" -B --raw  | sed $'s/\t/,/g' > filename.csv

The $ in the sed command is the magic that makes it work.

Answer 9

I'm surprised no one suggested the very simple solution of: sed -i.bak -E 's/\\\t/ /' file.txt That should do the trick.

You need to escape the escape (hence the 3 \s) to allow sed to understand that you are trying to use a \t character in the regular expression when everything is substituted...

Answer 10

This worked for me.

sed -e 's/[\t]/ /g'