Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Our input looks something like

2012-04-17  [GBPGBP]
2012-04-13  [GBP GBP]
2012-04-13  [GBP]
2012-04-11  [GBPGBP]
2012-04-11  [GBP GBP]
2012-04-10  [GBPGBP]
2012-04-06  [GBP GBP GBP]
2012-04-17  [GBPGBP]
2012-04-13  [GBP CDN]
2012-04-13  [GBP]
2012-04-11  [GBPCDN]
2012-04-11  [GBP DL DL]
2012-04-10  [PSGBP]
2012-04-06  [PS PS]

And we'd like to get output like

2012-04-17  [GBP]
2012-04-13  [GBP]
2012-04-13  [GBP]
2012-04-11  [GBP]
2012-04-11  [GBP]
2012-04-10  [GBP]
2012-04-06  [GBP]
2012-04-17  [GBP]
2012-04-13  [GBP CDN]
2012-04-13  [GBP]
2012-04-11  [GBPCDN]
2012-04-11  [GBP DL]
2012-04-10  [PSGBP]
2012-04-06  [PS]

Basically delete any repeated string within the brackets. Any suggestions?

share|improve this question
    
Define string. The first line suggests that a simple set of nested loops will suffice (slowly), but the second line shows you expect white space to be treated as special. What do you really want? Do you have a dictionary of 'words' that can appear? –  ams Apr 18 '12 at 20:12
    
@ams Actually yes: it'd be GBP, CDN or PS in every case. I really want the output brackets to give me the unique set of that dictionary that appears within the input brackets, ignoring white space. If the brackets have GBP multiple times, regardless of separation I want it condensed to a single GBP. I wish I could do something like tr -s "GBP" |tr -s "CDN"|tr -s "PS" but tr -s only works for single characters. –  Chris Apr 18 '12 at 21:22

1 Answer 1

up vote 5 down vote accepted
sed -e ': a' -e 's/\(\[[^][]*\)\([A-Z][A-Z][A-Z]*\)\([^][]*\)\2/\1\2\3/' -e 't a'
  • : a sets a label at the beginning of the script.
  • s/\(wibble\)\(foo\)\(bar\)\2/\1\2\3/ replaces wibblefoobarfoo by wibblefoobar.
  • [A-Z][A-Z][A-Z]* matches two or more letters
  • t a loops back to the label a if the previous s command made a replacement.
share|improve this answer
    
I ended up going with sed -e ': a' -e 's/\(\[[^][]*\)\([A-Z][A-Z][A-Z]\)\([^][]*\)\2/\1\2\3/' -e 't a' since the wild card seemed to be only deleting the last two letters of a repeat. I changed other code to ensure that all the matches would be three characters long. Thanks! –  Chris Apr 23 '12 at 16:27

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.