1

I have a text file that has unique lines by timestamps. This below is the example of the lines in that text file:

111.111.11.11: Jun 18 08:01:34 2017: abc\john.doe1 Windows...

111.111.11.11: Jun 18 09:01:34 2017: abc\john.doe2 Windows...

111.111.11.11: Jun 18 09:30:34 2017: abc\john.doe2 Windows...

111.111.11.11: Jun 18 10:01:34 2017: abc\john.doe3 Windows...

111.111.11.11: Jun 18 11:01:34 2017: abc\john.doe1 Windows...

111.111.11.11: Jun 18 12:01:34 2017: abc\john.doe3 Windows...

111.111.11.11: Jun 18 13:01:34 2017: abc\john.doe3 Windows...

...

These lines go a long way. I need to move the line where the first occurrence of john.doe1 happened to another text file, and ignore every other line that has john.doe1 in it. I have to do the same for the whole file, that has many, many john.does.

My output to another file should look like this:

111.111.11.11: Jun 18 08:01:34 2017: abc\john.doe1 Windows...

111.111.11.11: Jun 18 09:01:34 2017: abc\john.doe2 Windows...

111.111.11.11: Jun 18 10:01:34 2017: abc\john.doe3 Windows...

Problem is john.does are usually real names, and I do not know them before, I just know that they appear between the characters '\' and ' '.

4

Extension of the familiar Awk de-duplication idiom to use a matched substring instead of a delimited field:

awk 'match($0, /\\[^ ]*/) && !seen[substr($0,RSTART,RLENGTH)]++' file
1
  • Thank you a lot! – eapoehoe Jul 3 '19 at 22:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.