Is it possible to remove specific repeated character of a list using regex?

Question

I have a list of one column that contain ~ 100 lines, in which some lines are repeated, and my purpose is to get rid of a specific duplicate lines and leave only one copy, while the other lines kept untouched.

An extract of the files that I'm working on :

V(Mn9)   
V(C1,H3) 
V(Mn6)   
V(Mn6)   
V(C4,H6) 
V(Mn9)   
V(Mn9)   
V(C1,Mn6)
V(C4,Mn9)
V(Mn6)   
V(C1,C4) 
C(Mn9)   
C(Mn6)   
C(C1)    
C(C4)    
C(Mn9)   
C(Mn6)   
V(C1,H2) 
V(Mn9)   
V(Mn6)   
V(C4,H5)

My purpose is to remove all the duplicate lines contain C(Xx0-9) and leave one of them and kept the V(Xxx..).

The result I seek :

V(Mn9)   
V(C1,H3) 
V(Mn6)   
V(Mn6)   
V(C4,H6) 
V(Mn9)   
V(Mn9)   
V(C1,Mn6)
V(C4,Mn9)
V(Mn6)   
V(C1,C4) 
C(C1)    
C(C4)    
C(Mn9)   
C(Mn6)   
V(C1,H2) 
V(Mn9)   
V(Mn6)   
V(C4,H5)

I used the command :

sed '0,/C(Mn9)/{/C(Mn9)/d}' inputfile.txt | sed '0,/C(Mn6)/{/C(Mn6)/d}'

and it's working, but it's not good enough for the whole file, because there is a lot of C(Xx1-50), I thinked to use regular expression, but I don't know how, that's why I need your help.

Answer 1

$ awk '!(/^C\(..[0-9])$/ && seen[$0]++)' file
V(Mn9)
V(C1,H3)
V(Mn6)
V(Mn6)
V(C4,H6)
V(Mn9)
V(Mn9)
V(C1,Mn6)
V(C4,Mn9)
V(Mn6)
V(C1,C4)
C(Mn9)
C(Mn6)
C(C1)
C(C4)
V(C1,H2)
V(Mn9)
V(Mn6)
V(C4,H5)

The above assumes you have no white space before/after the visible characters in your sample input. If you do then remove them, e.g.:

$ awk '{gsub(/^[[:space:]]+|[[:space:]]+$/,"")} !(/^C\(..[0-9])$/ && seen[$0]++)' file
V(Mn9)
V(C1,H3)
V(Mn6)
V(Mn6)
V(C4,H6)
V(Mn9)
V(Mn9)
V(C1,Mn6)
V(C4,Mn9)
V(Mn6)
V(C1,C4)
C(Mn9)
C(Mn6)
C(C1)
C(C4)
V(C1,H2)
V(Mn9)
V(Mn6)
V(C4,H5)

Answer 2

I suggest to use sed to collect lines in the hold space to check whether they appeared before:

 sed -n 'H;G;/^\(C([^)]*)\).*\1 *\n/!P'

• H appends the current line to the hold space
• G appends the hold space with all lines we ever saw to the pattern space
• C([^)]*) is one of those C(…) patterns, the ^ anchors it to the beginning of the line and it's surrounded by \(…\), so it can be backreferenced as \1 later. We need \1 *\n as pattern, with the newline (after possible whitespaces) to avoid matching the freshly appended line at the end. So the whole pattern /^\(C([^)]*)\).*\1 *\n/ matches a line with a duplicate C(…), so only if this ! doesn't match,
• Print everything before the first newline (= without the appended hold space), while default output is suppressed by the -n option

Note that depending on you sed version and file size, this may fail because over the time, all lines will be in memory.