1

I'm trying to join two files removing duplicate head row and take only 1 last row

for example:

File1.txt

head1
data1
data2
tail8

File2.txt

head1
data3
data4
tail9

results desired in file3.txt:

head1
data1
data2
data3
data4
tail8 or rail9 doesn't matter

I try first this to remove duplicate head:

awk '!seen[$0]++' file1.txt file2.txt > file3.txt

second command is:

awk 'NR > 1 { print prev } { prev = $0 }' file3.txt > file4.txt

but result is with tail in the middle of file4.txt, not to the end

head1
data1
data2
tail8
data3
data4

Any idea? Thank you in advance

1
  • Well, do you know for sure that the header is the only duplicated line? Then you do not need to check all other lines with !seen[$0]++ and can do as Paul suggests below.
    – Quasímodo
    Aug 25 '20 at 17:23
0
$ awk 'NR==FNR{ if (NR>1) print prev; prev=$0; next } FNR>1' file1 file2
head1
data1
data2
data3
data4
tail9
5
  • That's works fine... except my files has many colums such "data1, abc, cde etc etc" and this example get only first $1, I change code with $0 and now works fine
    – Darioit
    Aug 25 '20 at 18:25
  • another problems, code fails if one of two file are empty, now I try with sed
    – Darioit
    Aug 25 '20 at 18:27
  • Yeah, I should have used $0 instead of $1, fixed now. Having said that, if your real data has multiple columns then you should provide sample data with multiple columns.
    – Ed Morton
    Aug 25 '20 at 18:31
  • 1
    If your real data could have empty files then you should state that in your question and provide examples where each file is empty. All we have to go on when trying to help you come up with a solution is what you tell/show us in your question.
    – Ed Morton
    Aug 25 '20 at 18:32
  • 1
    I ask, maybe file are not complete empty but head and tail still always present, there is nothing to worry about, thank you
    – Darioit
    Aug 25 '20 at 18:33
3

Why awk? This would work:

{ sed -e '$d' file1.txt; sed -e '1d' file2.txt; } > file3.txt
2
  • yes, but I work in windows environment and I use gawk, I prefer don't install others tools, but it there's no other solution would be great use sed
    – Darioit
    Aug 25 '20 at 17:32
  • 2
    or head -n -1 file1; tail -n +2 file2.
    – Ed Morton
    Aug 25 '20 at 17:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.