0

I'm trying to combine multiple files into one final file. Each file has many entries within them, most with overlapping titles. I would like to merge content between both, under the title headings. Think of combining two dictionaries and it makes more sense. Entries for a single word can be found in both, but definitions differ slightly in each. Some entries exist in one and not the other, etc.

For example, I'd like to merge these two files to produce a single output file:

File 1

Entry 1
Green Trees
Entry 3
Orange Fibers

File 2

Entry 1
Red Trees
Entry 2
Spotted Zebras
Entry 3
Blue Fibers

Output File

Entry 1
Green Trees
Red Trees
Entry 2
Spotted Zebras
Entry 3
Orange Fibers
Blue Fibers

Note that Entry 2 did not exist in File 1, but made it to the final product. Likewise, the content of each entry was merged anywhere the entry ID matches.

How can I accomplish this?

EDIT: The above is a simplified version for asking the question. Below is a sample of actual entries in the files.

The $$$00001 is the Entry title.

From File 1

$$$00001
<b><br>- Original: Α<b><br></b></b>- Transliteration: A<b><br></b></b>- Phonetic: al'-fah<b><br></b></b>-...
$$$00002
<b><br>- Original: script<b><br></b></b>- Translitera...

From File 2

$$$00001
<b><br>α<b><br></b></b>a; indeclinable...
$$$00002
<b><br>texts<b><br></b></b>A...
  • Are the headings all of the format Entry <num>? – muru Apr 10 '18 at 8:43
  • Entry <num> is a simplified version of the headings in order to ask the question. More realistically, they will be zero-padded numbered entries with 5 digits. – Matt Zabojnik Apr 10 '18 at 8:47
  • Well, how do we identify the headings then? – muru Apr 10 '18 at 8:47
  • I've updated my question with a real example for clarity. – Matt Zabojnik Apr 10 '18 at 8:53
1

A simple awk one-liner solves your example:

awk '/^Entry/{k=$0;next}{g[k]=g[k]"\n"$0}END{for(k in g)print k g[k]}' file1 file2

I suppose you know that basically awk processes input lines one after another according to a program. This particular awk program is specified as first argument and consists of three statements. Let’s analyze them one by one:

  • /^Entry/{k=$0;next} means: if the processed line matches /^Entry/, store it in the variable k and go to the next cycle ignoring the following statements.

  • {g[k]=g[k]"\n"$0} has no preceding condition, so it is always executed, and means: update the value stored in the dictionary g with the key k: the new value has to be the concatenation of the (possibly empty) previous value g[k], a carriage return "\n", and the current line.

  • END{for(k in g)print k g[k]} has an END condition and is therefore executed when all input lines have been processed. It says: for each key in g, that is, for each title which has appeared in the input files, print the associated value, which is the concatenation of all lines found in input files under that title.

To use it IRL, You have to replace /^Entry/ with the correct pattern (probably /^\$\$\$/).

  • Excellent, this solution worked perfectly. Thank you! Would you mind explaining a little further on what's going on in there? – Matt Zabojnik Apr 10 '18 at 9:46
  • @MattZabojnik Done. – Dario Apr 10 '18 at 10:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.