0

I have to combine 2 files and remove header from the 2nd file. Output should have new sequence number

ex file1.txt

H20200428  
DSEQ0000001USA  
DSEQ0000002MEXICO  
DSEQ0000003BRAZIL  
T00003

file2.txt

H20200428  
DSEQ0000001USA  
DSEQ0000002MEXICO  
DSEQ0000003BRAZIL  
T00003

output file should look like below

H20200428  
DSEQ0000001USA  
DSEQ0000002MEXICO  
DSEQ0000003BRAZIL  
DSEQ0000004USA  
DSEQ0000005MEXICO  
DSEQ0000006BRAZIL  
T00006

I used awk to combine file and drop the header

awk 'FNR!=NR && FNR==1 {next} 1' file1.txt file2.txt > output.txt

How do I change seq number and include one trailer record with total D type records in the same awk command if possible?

Thanks in advance

  • 1
    If you had 10 values should the output be DSEQ00000010BRAZIL or DSEQ0000010BRAZIL (count the zeros)? Ditto for T000010 vs T00010. – Ed Morton Apr 28 at 12:45
2

You could try the following:

awk 'FNR==1 {if (NR>1) next; print;} \
     /^D/ {seq++; sid=sprintf("SEQ%07d",seq); sub(/SEQ[0-9]+/,sid); print} \
     END {printf("T%05d\n",seq);}' file1.txt file2.txt

This will

  • skip the first line of all files but the first, whose first line is printed "as is"
  • for all lines starting with D, increase a sequence counter seq, replace the existing sequence with the new sequence id, and print the line.

At end-of-file, it will print the total from the last value of seq.

This solution should also work for more than 2 files.

| improve this answer | |
1

With GNU awk for the 3rd arg to match() if you want to print the new sequence number in the same width of 0-padded string as the existing input:

$ cat tst.awk
NR==1 && FNR==1 { print }
FNR > 2 { printf "%s%0*d%s\n", prev[1], length(prev[2]), ++seqNr, prev[3] }
{ match($0,/([^1-9]+)([0-9]+)(.*)/,prev) }
END { printf "%s%0*d\n", prev[1], length(prev[2]), seqNr }

$ awk -f tst.awk file1 file2
H20200428
DSEQ0000001USA
DSEQ0000002MEXICO
DSEQ0000003BRAZIL
DSEQ0000004USA
DSEQ0000005MEXICO
DSEQ0000006BRAZIL
T00006

which with any awk (and keeping the array prev[] instead of using scalars for comparison with the gawk script) would be:

$ cat tst.awk
NR==1 && FNR==1 { print }
FNR > 2 { printf "%s%0*d%s\n", prev[1], lgth2, ++seqNr, prev[3] }
{
    match($0,/[^1-9]+/)
    prev[1] = substr($0,RSTART,RLENGTH)
    match($0,/[^1-9]+[0-9]+/)
    lgth2 = RLENGTH - length(prev[1])
    prev[3] = substr($0,RSTART+RLENGTH)
}
END { printf "%s%0*d\n", prev[1], lgth2, seqNr }

$ awk -f tst.awk file1 file2
H20200428
DSEQ0000001USA
DSEQ0000002MEXICO
DSEQ0000003BRAZIL
DSEQ0000004USA
DSEQ0000005MEXICO
DSEQ0000006BRAZIL
T00006

or if you want to just retain the same number of leading zeros then with GNU awk:

$ cat tst.awk
NR==1 && FNR==1 { print }
FNR > 2 { print prev[1] (++seqNr) prev[2] }
{ match($0,/([^1-9]+)[0-9]+(.*)/,prev) }
END { print prev[1] seqNr }

$ awk -f tst.awk file1 file2
H20200428
DSEQ0000001USA
DSEQ0000002MEXICO
DSEQ0000003BRAZIL
DSEQ0000004USA
DSEQ0000005MEXICO
DSEQ0000006BRAZIL
T00006

and with any awk:

$ cat tst.awk
NR==1 && FNR==1 { print }
FNR > 2 { print prev[1] (++seqNr) prev[2] }
{
    match($0,/[^1-9]+/)
    prev[1] = substr($0,RSTART,RLENGTH)
    sub(/[^0-9]+[0-9]+/,"")
    prev[2] = $0
}
END { print prev[1] seqNr }

$ awk -f tst.awk file1 file2
H20200428
DSEQ0000001USA
DSEQ0000002MEXICO
DSEQ0000003BRAZIL
DSEQ0000004USA
DSEQ0000005MEXICO
DSEQ0000006BRAZIL
T00006
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.