Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

file :

hi hello,new york,      ,  ,brazil site   ,brazil
january,month is feb   ,      , ,indiana jones

task:

  • delete all horizontal space (spaces and tab) between any two comas (like , ,) if there is no character other than space. so it would look like :

    hi hello,new york,,,brazil site   ,brazil
    january,month is feb   ,,,indiana jones
    
  • also, delete any trailing or leading horizontal space (space and tab) for each parameter so that it looks like this.

    final output:

    hi hello,new york,,,brazil site,brazil
    january,month is feb,,,indiana jones
    

NOTE: The parameters itself has spaces like hi hello and indiana jones etc. It should not be affected and become like hihello or indianajones. If possible a solution with awk, otherwise sed will also do.

I am unable to achive the first task with sed like this:

sed -n 's/,[[:blank:]],/,,/gp' file

but i get this as output:

january,month is feb   ,      ,,indiana jones
share|improve this question
1  
Probably I misunderstand you, but why not just sed 's/ *, */,/g' file? –  manatwork Feb 27 '13 at 8:11
    
thanks @manatwork it works, u got this this was easy and i have been thinking a lot. –  munish Feb 27 '13 at 8:14
    
@manatwork, better sed 's/[[:space:]]*,[[:space:]]*/,/g to catch tabs too. –  vonbrand Feb 27 '13 at 12:27
    
Right. Or sed 's/\s*,\s*/,/g' file to type less. –  manatwork Feb 27 '13 at 13:14
add comment

2 Answers 2

up vote 3 down vote accepted

There are three possibilities with sed:

  1. Simplest one (matching a space character zero or more times before a comma, and a space character zero or more times after the comma, replacing it by just a comma—and this many times, thus using the option g for global):

    sed 's/ *, */,/g' file
    

    Time for this command: 3.056s, with perl -p -e instead of sed 5.932s

  2. [:space:] means all whitespace characters, thus being the same as [ \t\r\n\v\f]. This is POSIX standard:

    sed 's/[[:space:]]*,[[:space:]]*/,/g' file
    

    Time for this command: 10.365s, with perl -p -e instead of sed 9.060s

  3. And finally the abbreviated version with \s, which stands for the same as [:space:] but is Perl syntax:

    sed 's/\s*,\s*/,/g' file
    

    Time for this command: 10.507s, with perl -p -e instead of sed 6.126s

All of them result in

hi hello,new york,,,brazil site,brazil
january,month is feb,,,indiana jones

The time was taken for a file with 2M lines (containing the questions example text 1M times). The file was 97MBytes big.

The first command seems to be the fastest with sed. The last command is the fastest when using perl and taking into account, that it replaces not only spaces but also tabstops, linefeed and so on.

share|improve this answer
    
All 3 regular expressions work in perl's replacement mode too =) perl -pi -e 's/\s*,\s*/,/g' file –  mtm Mar 1 '13 at 9:49
    
To have the same as sed you should have written perl -p -e to be non in place editing. Your example perl -pi -e is the same as sed -i. –  erik Mar 1 '13 at 11:20
    
ah yes. was thinking file replacement from the question. –  mtm Mar 1 '13 at 12:53
add comment

So, you are asking awk:

awk '{$1=$1};gsub(" ,",",")'

Will this do? Note that this have the side-effect of removing duplicated space in your data inside commas, which might not be a problem for English.

share|improve this answer
    
May you please explain the command? What is this $1=$1 for? –  erik Mar 1 '13 at 6:42
1  
'$1=$1' is a refreshing, that tells awk to refresh the data it reads from input, after that $0 is changed, and output is re-structured with OFS(the out put seperater) –  MeaCulpa Mar 1 '13 at 6:55
    
Time for this command: 5.038s and for the easier to understand and less to write version awk 'gsub(" +,",",")' only 3.061s. Same time as for the simple sed command in my answer. –  erik Mar 1 '13 at 11:37
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.