sed behaves different on FreeBSD and on Linux?

Question

I use both Linux and FreeBSD (specifically, I use Debian Linux and PC-BSD), and I found something weird about sed.

I frequently need to convert "tab separated values" files to "comma separated values". The simplest way I know is to use sed, like this:

sed 's/\t/,/g' inputFile.txt > outputFile.csv

This works perfectly on Linux: It replaces every tab with a comma... but on FreeBSD, it doesn't replace anything!!!

Am I missing something? Is there a syntax with FreeBSD sed that is different to the one on Linux?

Answer 1

Maybe you should use the -E option(or -r as explained on the manual) to keep compatibility with GNU Sed. In your case, you could install Gnu Sed if you are used to it (port gsed on FreeBSD), or it will take a long effort to port scripts.

And remember. If some command on BSD does not act like the gnu version of that utility, it doesn´t mean it´s broken ;)

Answer 2

Yes, there are various differences, the behavior of -i being the only one I know of off the top of my head.

I've never used BSD so I can't really help with the details but a workaround might be to use tr instead:

tr '\t' , < inputFile.txt > outputFile.csv

A pleasant side effect is that tr should be significantly faster. I tested that on my Linux using a test file with 50000 lines, each of which had 2 tabs:

$ time tr '\t' , < foo.txt > /dev/null 

real    0m0.004s
user    0m0.000s
sys     0m0.000s

$ time sed 's/\t/,/g' foo.txt > /dev/null 

real    0m0.039s
user    0m0.036s
sys     0m0.000s

Answer 3

Yes, unlike GNU sed FreeBSD sed does not interpret ANSI C escape sequences such as \t in regular expressions.

One way to get a least common denomiator in this case is to use printf.

tab="$(printf '\t')"
printf '\t\n' | sed 's/'"${tab}"'/,/g'
printf '\t\n' | sed 's/'"$(printf '\t')"'/,/g'

The behaviour of sed -i for in-place file edits can be made compatible if a switch or option immediately follows the -i switch, e. g. sed -i -e 's/x/X/g' file works for both GNU sed as well as FreeBSD sed.

Recent versions of FreeBSD sed (FreeBSD 8.1 or newer) have the -r switch to increase compatibility with GNU sed.

(In addition, the use of POSIX character classes in sed regular expressions is a good way to ensure compatibility as well).

For an alternative, POSIX-conformant sed implementation see: minised - a smaller, cheaper, faster SED implementation.

Answer 4

You should use a literal TAB character instead of \t:

sed 's/    /,/g' inputFile.txt > outputFile.csv

See this comment by Stephane on another question.

The following article may also interest you:

• The Differences in Sed Versions

I quote the relevant part:

Regex Differences

Regular expression syntax differs subtly between the different versions of SED. Most of the differences involve special escape patterns used to match non-printing characters, such as the ASCI bell and form feeds.

Answer 5

After login I see the next announcement and save it. Hope it will useful for other too

Want to use sed(1) to edit a file in place? Well, to replace every 'e' with an 'o', in a file named 'foo', you can do:

sed -i.bak s/e/o/g foo

And you'll get a backup of the original in a file named 'foo.bak', but if you want no backup:

sed -i '' s/e/o/g foo