I have files that are edited by people other than me. They contain lines that look like this:

<E> This is text </E>

I use some Bash shell scripts do a bunch of text replacement functions on these lines of text. However, in order for the text replacements to work, the format has to be exactly correct. Most of the time they are, but of course I can't rely on the editors I'm getting these files from to not make any typos.

Part of that format is that there be one space after <E>, and one space before </E>. No more, no less. So these are all incorrect:

<E>This is text </E>
<E> This is text</E>
<E> This is text     </E>
<E>   This is text </E>

I know I can use sed to search for specific problems, like two spaces before </E> (using # instead of / since the text I'm acting on also contains / characters):

sed -i '$ s#  </E>#" </E>#g' *.txt

... but I don't know how to use it to search for an unknown number. Also, in cases where there is no space, then the character adjacent to <E> or </E> might be anything.

Bottom line, how can I search for instances of both zero spaces and of two or more spaces separated the text and <E> and </E> tags, and convert them to one space?

up vote 3 down vote accepted
sed -e 's!<E> *!<E> !g' -e 's! *</E>! </E>!g'

(Note: I've used ! rather than # or / as my regular expression delimiter. Personal preference.)

sed can be passed more than one command to run on its input, as long as each is prefixed with the -e flag.

The * following the space in the regular expressions above means "match 0 or more space characters". The re_format man page gives more information on such repetition:

An atom followed by '*' matches a sequence of 0 or more matches of the atom. An atom followed by '+' matches a sequence of 1 or more matches of the atom. An atom followed by '?' matches a sequence of 0 or 1 matches of the atom.

where an "atom" is the sub-pattern before the *, +, or ?.

Running this sed command on your examples:

reedm@www:/tmp $ cat > example.txt
<E>This is text </E>
<E> This is text</E>
<E> This is text     </E>
<E>   This is text </E>
reedm@www:/tmp $ sed -e 's!<E> *!<E> !g' -e 's! *</E>! </E>!g' example.txt 
<E> This is text </E>
<E> This is text </E>
<E> This is text </E>
<E> This is text </E>

echo $STRING | tr -s " "

should eliminate multiple consecutive space characters and reduce them to a single space. Then it is up to you to decide how you want to handle that single white space.

hope this helps

"at least one space" is / \+/

"zero or more spaces" is / */

"not a space" is /[^ ]/

"two or more spaces" is either /   */ or /  \+/
  • sed -i 's#<E>[^ ]#<E> #g' deletes whatever character is after </E> and replaces it with a space. So <E>blah </E> becomes <E> lah </E>. Are you sure that's the right syntax for "no space"? – Questioner Feb 21 '13 at 3:18
  • Not "whatever character" but "everything but a space". ">[^ ]" means "there is no space after >". It's not my fault that you confuse matching with replacing. You either have to limit the replacement to respective lines (have a look at man sed, the block about addresses) or have to include the matched content in the replacement: s#<E>\([^ ]\)#<E> \1# – Hauke Laging Feb 21 '13 at 3:34
  • Chill out man. I wasn't blaming you for anything, just asking. There's no "fault" here. – Questioner Feb 21 '13 at 3:38

Look at the regular expressions handled by sed(1). Under the many options you have there are \+ to repeat the preceding 1 or more times. So, sed -i -e 's; \+</E>;</E>;g' *.txt will delete any number of spaces before </E>. Look at the info manual for GNU sed (I find pinfo(1) is the nicest reader, but YMMV), the man page is definitely lacking. Under the detailed node listing check "Regular expressions".

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.