I have 3 files with millions of records each. I'd like to filter those files in order to find some data more easily. Unfortunately, there is no common format from one record to other. For example:
record1 will contain: A B C D E record2 will contain A B D E record3 will contain B C D
The only common part that I have are the first 23 characters which contain some identifier (from which only last 15 are unique between records).
I'd like to filter those files to contain the identifier and the D part. How can I do that? I know that the delimiter between fields is tab and inside each part, there is comma.
Example of the D part:
ROAMENT,INT,15 ROAMENT - constant part which identifies the field INT - some value which is a must if ROAMENT exists 15 - some other value which is optional