This is a problem of essentially performing a 2-way or bi-directional merge between two files syncing the differences between them. My own use-case for doing this was performing upgrades on a GitHub repo and retaining unique data in a variables file to preclude re-keying it on every upgrade!
I'll show you the expression first and the test data used so you can recreate the results to validate (just change the field separator from '=' to an empty space ' ' for your data set):
paste -d'\n' file1.txt file2.txt|awk -F'=' '!seen[$1]++' > file3.txt
My test data looks like this:
file1.txt:
LineA='value1'
LineB='value2'
LineC='value3'
LineD='value4'
#
LineE='value5'
LineF='value6'
#
LineG='value7'
#
LineH='value8'
file2.txt:
LineA=''
LineB=''
NEWVARIABLE1='This only Exists in file2.txt Under LineB'
LineC=''
LineD=''
#
LineE=''
NEWVARIABLE2='This only Exists in file2.txt Under LineE'
LineF=''
#
LineG=''
#
LineH=''
NEWVARIABLE3='This only Exists in file2.txt under LineH'
Output of:
paste -d'\n' file1.txt file2.txt|awk -F'=' '!seen[$1]++' > file3.txt
Combined file3.txt looks like this:
LineA='value1'
LineB='value2'
LineC='value3'
NEWVARIABLE1='This only Exists in file2.txt Under LineB'
LineD='value4'
#
LineE='value5'
LineF='value6'
NEWVARIABLE2='This only Exists in file2.txt Under LineE'
LineG='value7'
LineH='value8'
NEWVARIABLE3='This only Exists in file2.txt under LineH'
Remark that in file2.txt all the values are empty ('') except for NEWVARIABLEx values. Reviewing the output you can see all the unique data from file1.txt has been preserved after the merge.
Also remark that each new "variable" downshifts added to file2.txt (the "updated" file) by 1 line in the combined file3.txt. So if you add 4 new variables to file2.txt, in the combined file3.txt the FOURTH new variable will be 4 lines lower in file3.txt. In my use case this is not a problem however.
Sadly, this will work on your FIRST use case- doing the 2-way file merge. However, if you added a continuous block of test to file2.txt ( again, the "updated file) these would be interleaved with the surrounding lines from file1.txt in the combined file3.txt. So I got you half the way there but my use case only overlaps yours on the first part of your question.
I tried tons of different solutions offered by other posters on this forum and others, but this was really the only one that did the business. HTH-
diff
andpatch
.diff
andpatch
with what options? With normal diffs, the secondpatch
will use the wrong offsets. With context/unified diffs, any additions to the same position in both files will be rejected by the secondpatch
.diff a b > diff
, a slight massage of the resulting diff file (namely removing the deletions), andpatch a < diff
.diff -y a b
might be easier to process.