I have a file like this :
1 Record|1111|ABC
2 text in between for record 1
3 text in between for record 1
4 Record|2222|XYZ
5 text in between for record 2
6 Record|3333|XYZ
7 text in between for record 3
8 .
I want to read this file and generate something like
<Record_number> | <start line> | <number of lines> | md5sum(content)
That is:
1111|1|2|md5sum(Record|1111|ABC\ntext in between for record 1\ntext in between for record 1)
2222|4|1|md5sum(Record|2222|XYZ\ntext in between for record 2\n)
etc.
Currently, I am doing this using a two step process:
Step 1:
grep -n -C 0 "Record|" ../test.txt | awk -F[':|'] '{print $3"|"$1}'
will create
1111|1
2222|4
3333|6
Step 2: Read this file line by line and generate md5sum and number of lines through script.
The issue it this two step processing is taking more processing time and the file size is huge (~4GB).
Is there a better way to do this?