Search through line for a string and print that line only once to another file

Question

I have a text file that has unique lines by timestamps. This below is the example of the lines in that text file:

111.111.11.11: Jun 18 08:01:34 2017: abc\john.doe1 Windows...

111.111.11.11: Jun 18 09:01:34 2017: abc\john.doe2 Windows...

111.111.11.11: Jun 18 09:30:34 2017: abc\john.doe2 Windows...

111.111.11.11: Jun 18 10:01:34 2017: abc\john.doe3 Windows...

111.111.11.11: Jun 18 11:01:34 2017: abc\john.doe1 Windows...

111.111.11.11: Jun 18 12:01:34 2017: abc\john.doe3 Windows...

111.111.11.11: Jun 18 13:01:34 2017: abc\john.doe3 Windows...

...

These lines go a long way. I need to move the line where the first occurrence of john.doe1 happened to another text file, and ignore every other line that has john.doe1 in it. I have to do the same for the whole file, that has many, many john.does.

My output to another file should look like this:

111.111.11.11: Jun 18 08:01:34 2017: abc\john.doe1 Windows...

111.111.11.11: Jun 18 09:01:34 2017: abc\john.doe2 Windows...

111.111.11.11: Jun 18 10:01:34 2017: abc\john.doe3 Windows...

Problem is john.does are usually real names, and I do not know them before, I just know that they appear between the characters '\' and ' '.

Answer 1

Extension of the familiar Awk de-duplication idiom to use a matched substring instead of a delimited field:

awk 'match($0, /\\[^ ]*/) && !seen[substr($0,RSTART,RLENGTH)]++' file